[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 14:49:58.956605   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39325
I20260709 14:49:58.959262   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 14:49:58.960803   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:49:58.982523   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:49:58.983989   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:49:58.985579   431 server_base.cc:1061] running on GCE node
W20260709 14:49:58.985639   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:00.231383   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:00.231702   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:00.231882   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608600231863 us; error 0 us; skew 500 ppm
I20260709 14:50:00.232823   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:00.241210   431 webserver.cc:533] Webserver started at http://127.0.107.254:39623/ using document root <none> and password file <none>
I20260709 14:50:00.242324   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:00.242569   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:00.243172   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:00.250268   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "74d3e791400b4df286511a6339707352"
format_stamp: "Formatted at 2026-07-09 14:50:00 on dist-test-slave-wg8n"
I20260709 14:50:00.261519   431 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:50:00.268465   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:00.270172   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:50:00.270684   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "74d3e791400b4df286511a6339707352"
format_stamp: "Formatted at 2026-07-09 14:50:00 on dist-test-slave-wg8n"
I20260709 14:50:00.271221   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:00.337452   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:00.339421   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 14:50:00.340032   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:00.453258   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39325
I20260709 14:50:00.453315   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39325 every 8 connection(s)
I20260709 14:50:00.463886   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:00.491868   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: Bootstrap starting.
I20260709 14:50:00.500635   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:00.503180   498 log.cc:826] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: Log is configured to *not* fsync() on all Append() calls
I20260709 14:50:00.510442   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: No bootstrap required, opened a new log
I20260709 14:50:00.535985   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER }
I20260709 14:50:00.536726   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:00.537061   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74d3e791400b4df286511a6339707352, State: Initialized, Role: FOLLOWER
I20260709 14:50:00.538388   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER }
I20260709 14:50:00.538996   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:50:00.539351   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:50:00.539788   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:00.545552   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER }
I20260709 14:50:00.546612   498 leader_election.cc:304] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74d3e791400b4df286511a6339707352; no voters: 
I20260709 14:50:00.548826   498 leader_election.cc:290] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:50:00.550055   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:00.553648   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 1 LEADER]: Becoming Leader. State: Replica: 74d3e791400b4df286511a6339707352, State: Running, Role: LEADER
I20260709 14:50:00.554796   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:50:00.554666   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER }
I20260709 14:50:00.572383   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74d3e791400b4df286511a6339707352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER } }
I20260709 14:50:00.575682   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:00.574779   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74d3e791400b4df286511a6339707352. Latest consensus state: current_term: 1 leader_uuid: "74d3e791400b4df286511a6339707352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d3e791400b4df286511a6339707352" member_type: VOTER } }
I20260709 14:50:00.577492   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:00.580451   511 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:50:00.597330   511 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:50:00.600548   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:50:00.621982   511 catalog_manager.cc:1383] Generated new cluster ID: c9941dd38a154b50a50b9f08477afbb7
I20260709 14:50:00.622339   511 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:50:00.644526   511 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:50:00.646657   511 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:50:00.666882   511 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: Generated new TSK 0
I20260709 14:50:00.668413   511 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:50:00.733055   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:00.746256   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:00.750464   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:00.754983   431 server_base.cc:1061] running on GCE node
W20260709 14:50:00.755702   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:00.756963   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:00.757300   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:00.757516   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608600757498 us; error 0 us; skew 500 ppm
I20260709 14:50:00.758459   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:00.764958   431 webserver.cc:533] Webserver started at http://127.0.107.193:39917/ using document root <none> and password file <none>
I20260709 14:50:00.765866   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:00.766309   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:00.766913   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:00.768869   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "f75c182005714280af142f2b0950c9f0"
format_stamp: "Formatted at 2026-07-09 14:50:00 on dist-test-slave-wg8n"
I20260709 14:50:00.778021   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.009s	sys 0.001s
I20260709 14:50:00.784456   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:00.785619   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 14:50:00.786119   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "f75c182005714280af142f2b0950c9f0"
format_stamp: "Formatted at 2026-07-09 14:50:00 on dist-test-slave-wg8n"
I20260709 14:50:00.786569   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:00.832515   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:00.835445   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:00.838340   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:00.843976   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:00.844264   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:00.844673   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:00.844849   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:00.982821   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43129
I20260709 14:50:00.983023   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43129 every 8 connection(s)
I20260709 14:50:00.993909   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:01.009171   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:01.023864   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:01.025803   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:01.028589   431 server_base.cc:1061] running on GCE node
I20260709 14:50:01.029803   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:01.030109   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:01.030310   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608601030286 us; error 0 us; skew 500 ppm
I20260709 14:50:01.031032   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:01.034914   431 webserver.cc:533] Webserver started at http://127.0.107.194:40855/ using document root <none> and password file <none>
I20260709 14:50:01.035755   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:01.036105   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:01.036441   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:01.038411   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "2ebfda1b7363403da68e5fa26a266f15"
format_stamp: "Formatted at 2026-07-09 14:50:01 on dist-test-slave-wg8n"
I20260709 14:50:01.039142   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39325
I20260709 14:50:01.039757   591 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:01.041330   591 heartbeater.cc:507] Master 127.0.107.254:39325 requested a full tablet report, sending...
I20260709 14:50:01.047472   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:50:01.047340   463 ts_manager.cc:194] Registered new tserver with Master: f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129)
I20260709 14:50:01.051226   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46424
I20260709 14:50:01.054293   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.055693   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 14:50:01.056131   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "2ebfda1b7363403da68e5fa26a266f15"
format_stamp: "Formatted at 2026-07-09 14:50:01 on dist-test-slave-wg8n"
I20260709 14:50:01.056556   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:01.089056   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:01.090950   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:01.092588   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:01.095362   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:01.095594   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.095932   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:01.096123   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.245611   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:45861
I20260709 14:50:01.245893   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:45861 every 8 connection(s)
I20260709 14:50:01.258749   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:01.271670   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:01.275127   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:01.281719   431 server_base.cc:1061] running on GCE node
W20260709 14:50:01.283385   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:01.284240   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39325
I20260709 14:50:01.284685   666 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:01.284899   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:01.285116   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:01.285251   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608601285239 us; error 0 us; skew 500 ppm
I20260709 14:50:01.285785   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:01.285941   666 heartbeater.cc:507] Master 127.0.107.254:39325 requested a full tablet report, sending...
I20260709 14:50:01.289374   463 ts_manager.cc:194] Registered new tserver with Master: 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:01.290982   431 webserver.cc:533] Webserver started at http://127.0.107.195:33723/ using document root <none> and password file <none>
I20260709 14:50:01.291657   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:01.291704   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46428
I20260709 14:50:01.291908   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:01.292356   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:01.293835   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
format_stamp: "Formatted at 2026-07-09 14:50:01 on dist-test-slave-wg8n"
I20260709 14:50:01.302945   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.010s	sys 0.000s
I20260709 14:50:01.307132   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.308259   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:50:01.308665   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
format_stamp: "Formatted at 2026-07-09 14:50:01 on dist-test-slave-wg8n"
I20260709 14:50:01.308987   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:01.335039   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:01.336716   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:01.340507   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:01.344504   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:01.344823   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.345110   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:01.345299   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:01.509732   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33851
I20260709 14:50:01.509912   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33851 every 8 connection(s)
I20260709 14:50:01.535115   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39325
I20260709 14:50:01.535672   740 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:01.537865   740 heartbeater.cc:507] Master 127.0.107.254:39325 requested a full tablet report, sending...
I20260709 14:50:01.541879   463 ts_manager.cc:194] Registered new tserver with Master: 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:01.542953   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.022704676s
I20260709 14:50:01.544286   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46438
I20260709 14:50:01.618434   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:50:01.623461   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:50:01.804344   556 tablet_service.cc:1511] Processing CreateTablet for tablet c2ad604df88241eaadb947764d02e45d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:50:01.803927   553 tablet_service.cc:1511] Processing CreateTablet for tablet 4d56a87183594c8caf86cda36c1e4ba3 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:50:01.801117   555 tablet_service.cc:1511] Processing CreateTablet for tablet a322c0390a314662a84f1c53ab04b00d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:50:01.801206   554 tablet_service.cc:1511] Processing CreateTablet for tablet 90eb9c71a8f442e2b26a3b1394459370 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:50:01.812644   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ad604df88241eaadb947764d02e45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.813397   550 tablet_service.cc:1511] Processing CreateTablet for tablet 400cb3b23ea14886a356218695773ffa (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:50:01.805975   552 tablet_service.cc:1511] Processing CreateTablet for tablet 884c9111d1f249b686aa232a2885d25f (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:50:01.819545   631 tablet_service.cc:1511] Processing CreateTablet for tablet c2ad604df88241eaadb947764d02e45d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:50:01.823782   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a322c0390a314662a84f1c53ab04b00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.825860   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 884c9111d1f249b686aa232a2885d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.825220   630 tablet_service.cc:1511] Processing CreateTablet for tablet a322c0390a314662a84f1c53ab04b00d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:50:01.828202   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400cb3b23ea14886a356218695773ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.815951   551 tablet_service.cc:1511] Processing CreateTablet for tablet 6beecff85e664cc7b70aea711d0ebe55 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:50:01.837193   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ad604df88241eaadb947764d02e45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.837888   549 tablet_service.cc:1511] Processing CreateTablet for tablet a7cbd11606124c0a82fcdae85eebf9a6 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:50:01.840256   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d56a87183594c8caf86cda36c1e4ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.844991   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a322c0390a314662a84f1c53ab04b00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.851635   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beecff85e664cc7b70aea711d0ebe55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.838522   628 tablet_service.cc:1511] Processing CreateTablet for tablet 4d56a87183594c8caf86cda36c1e4ba3 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:50:01.859131   625 tablet_service.cc:1511] Processing CreateTablet for tablet 400cb3b23ea14886a356218695773ffa (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:50:01.856570   627 tablet_service.cc:1511] Processing CreateTablet for tablet 884c9111d1f249b686aa232a2885d25f (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:50:01.844916   629 tablet_service.cc:1511] Processing CreateTablet for tablet 90eb9c71a8f442e2b26a3b1394459370 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:50:01.861583   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eb9c71a8f442e2b26a3b1394459370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.850522   626 tablet_service.cc:1511] Processing CreateTablet for tablet 6beecff85e664cc7b70aea711d0ebe55 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:50:01.877511   624 tablet_service.cc:1511] Processing CreateTablet for tablet a7cbd11606124c0a82fcdae85eebf9a6 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:50:01.888986   704 tablet_service.cc:1511] Processing CreateTablet for tablet a322c0390a314662a84f1c53ab04b00d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:50:01.880906   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cbd11606124c0a82fcdae85eebf9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.890977   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d56a87183594c8caf86cda36c1e4ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.899972   703 tablet_service.cc:1511] Processing CreateTablet for tablet 90eb9c71a8f442e2b26a3b1394459370 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:50:01.894378   705 tablet_service.cc:1511] Processing CreateTablet for tablet c2ad604df88241eaadb947764d02e45d (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:50:01.892721   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a322c0390a314662a84f1c53ab04b00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.910552   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eb9c71a8f442e2b26a3b1394459370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.920912   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eb9c71a8f442e2b26a3b1394459370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.919646   702 tablet_service.cc:1511] Processing CreateTablet for tablet 4d56a87183594c8caf86cda36c1e4ba3 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:50:01.925093   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cbd11606124c0a82fcdae85eebf9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.944281   701 tablet_service.cc:1511] Processing CreateTablet for tablet 884c9111d1f249b686aa232a2885d25f (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:50:01.976126   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ad604df88241eaadb947764d02e45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.981390   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 884c9111d1f249b686aa232a2885d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.980669   548 tablet_service.cc:1511] Processing CreateTablet for tablet 4d2bc34dd22b4481801f97b7b84229e2 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:50:01.982999   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beecff85e664cc7b70aea711d0ebe55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.994994   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d56a87183594c8caf86cda36c1e4ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:01.995082   699 tablet_service.cc:1511] Processing CreateTablet for tablet 400cb3b23ea14886a356218695773ffa (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:50:02.000092   700 tablet_service.cc:1511] Processing CreateTablet for tablet 6beecff85e664cc7b70aea711d0ebe55 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:50:02.009833   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400cb3b23ea14886a356218695773ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.015879   698 tablet_service.cc:1511] Processing CreateTablet for tablet a7cbd11606124c0a82fcdae85eebf9a6 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:50:02.017845   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beecff85e664cc7b70aea711d0ebe55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.023115   548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2bc34dd22b4481801f97b7b84229e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.029248   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 884c9111d1f249b686aa232a2885d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.060714   591 heartbeater.cc:499] Master 127.0.107.254:39325 was elected leader, sending a full tablet report...
I20260709 14:50:02.066936   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cbd11606124c0a82fcdae85eebf9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.102106   623 tablet_service.cc:1511] Processing CreateTablet for tablet 4d2bc34dd22b4481801f97b7b84229e2 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:50:02.106508   623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2bc34dd22b4481801f97b7b84229e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.115001   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400cb3b23ea14886a356218695773ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.140990   697 tablet_service.cc:1511] Processing CreateTablet for tablet 4d2bc34dd22b4481801f97b7b84229e2 (DEFAULT_TABLE table=test-workload [id=44c87b8f39b442aebbe0381cbfdaf7b5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:50:02.149832   697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2bc34dd22b4481801f97b7b84229e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:02.263715   758 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.297420   666 heartbeater.cc:499] Master 127.0.107.254:39325 was elected leader, sending a full tablet report...
I20260709 14:50:02.327126   759 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:02.327296   760 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:02.353264   758 tablet_bootstrap.cc:654] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.383160   760 tablet_bootstrap.cc:654] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.383767   758 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.384341   758 ts_tablet_manager.cc:1403] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.133s	user 0.011s	sys 0.015s
I20260709 14:50:02.381464   759 tablet_bootstrap.cc:654] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.391427   758 raft_consensus.cc:359] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.392331   758 raft_consensus.cc:385] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.392660   758 raft_consensus.cc:740] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.393503   758 consensus_queue.cc:260] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.403704   760 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:02.404485   760 ts_tablet_manager.cc:1403] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.010s
I20260709 14:50:02.410584   759 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:02.411347   759 ts_tablet_manager.cc:1403] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.120s	user 0.023s	sys 0.010s
I20260709 14:50:02.412273   760 raft_consensus.cc:359] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.412922   758 ts_tablet_manager.cc:1434] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 14:50:02.414283   758 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.413184   760 raft_consensus.cc:385] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.417277   760 raft_consensus.cc:740] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.418398   760 consensus_queue.cc:260] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.424839   758 tablet_bootstrap.cc:654] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.416115   759 raft_consensus.cc:359] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.428918   759 raft_consensus.cc:385] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.429373   759 raft_consensus.cc:740] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.429620   760 ts_tablet_manager.cc:1434] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.006s
I20260709 14:50:02.430694   740 heartbeater.cc:499] Master 127.0.107.254:39325 was elected leader, sending a full tablet report...
I20260709 14:50:02.430307   759 consensus_queue.cc:260] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.440045   758 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.440495   759 ts_tablet_manager.cc:1434] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.005s
I20260709 14:50:02.440666   758 ts_tablet_manager.cc:1403] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.011s
I20260709 14:50:02.445478   758 raft_consensus.cc:359] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.446455   758 raft_consensus.cc:385] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.446713   759 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:02.447036   760 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:02.446815   758 raft_consensus.cc:740] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.448170   758 consensus_queue.cc:260] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.452353   758 ts_tablet_manager.cc:1434] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.002s
I20260709 14:50:02.455302   758 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.463752   760 tablet_bootstrap.cc:654] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.464916   758 tablet_bootstrap.cc:654] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.477946   758 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.478372   760 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:02.478555   758 ts_tablet_manager.cc:1403] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.013s
I20260709 14:50:02.478945   760 ts_tablet_manager.cc:1403] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.009s
I20260709 14:50:02.483167   760 raft_consensus.cc:359] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.483717   758 raft_consensus.cc:359] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.484157   760 raft_consensus.cc:385] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.484452   758 raft_consensus.cc:385] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.484476   760 raft_consensus.cc:740] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.485118   758 raft_consensus.cc:740] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.485725   760 consensus_queue.cc:260] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.486284   758 consensus_queue.cc:260] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.488829   760 ts_tablet_manager.cc:1434] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:50:02.489042   758 ts_tablet_manager.cc:1434] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 14:50:02.490423   759 tablet_bootstrap.cc:654] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.490453   758 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.490598   760 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
W20260709 14:50:02.496978   592 tablet.cc:2406] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 14:50:02.501114   760 tablet_bootstrap.cc:654] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.504504   758 tablet_bootstrap.cc:654] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.515061   759 tablet_bootstrap.cc:492] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:02.515830   759 ts_tablet_manager.cc:1403] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.009s
I20260709 14:50:02.517448   758 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.518010   758 ts_tablet_manager.cc:1403] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 14:50:02.519553   759 raft_consensus.cc:359] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.515080   760 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:02.520716   760 ts_tablet_manager.cc:1403] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.014s
I20260709 14:50:02.521209   759 raft_consensus.cc:385] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.521930   767 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.521193   758 raft_consensus.cc:359] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.522095   759 raft_consensus.cc:740] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.522459   767 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.522992   759 consensus_queue.cc:260] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.523998   760 raft_consensus.cc:359] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.524725   760 raft_consensus.cc:385] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.525890   760 raft_consensus.cc:740] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.526849   760 consensus_queue.cc:260] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.530889   760 ts_tablet_manager.cc:1434] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:50:02.534480   767 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.538686   760 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:02.522439   758 raft_consensus.cc:385] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.546787   758 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.548805   758 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.551888   758 ts_tablet_manager.cc:1434] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.009s
I20260709 14:50:02.553069   758 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.554162   764 raft_consensus.cc:493] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.554706   764 raft_consensus.cc:515] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.560731   760 tablet_bootstrap.cc:654] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.562265   764 leader_election.cc:290] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.563248   759 ts_tablet_manager.cc:1434] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.026s
I20260709 14:50:02.567448   759 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:02.578477   765 raft_consensus.cc:493] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.579351   765 raft_consensus.cc:515] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.584790   765 leader_election.cc:290] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
W20260709 14:50:02.596974   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:02.643579   759 tablet_bootstrap.cc:654] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.645000   758 tablet_bootstrap.cc:654] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.667547   765 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.668605   765 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.672464   765 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:02.688115   760 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:02.689316   760 ts_tablet_manager.cc:1403] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.157s	user 0.024s	sys 0.008s
I20260709 14:50:02.713761   758 tablet_bootstrap.cc:492] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.714414   758 ts_tablet_manager.cc:1403] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.162s	user 0.015s	sys 0.017s
I20260709 14:50:02.717943   758 raft_consensus.cc:359] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.721743   758 raft_consensus.cc:385] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.722136   758 raft_consensus.cc:740] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.728811   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
W20260709 14:50:02.732020   532 leader_election.cc:343] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:02.733101   758 consensus_queue.cc:260] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.753222   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:02.751024   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:02.757159   758 ts_tablet_manager.cc:1434] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.005s
I20260709 14:50:02.757586   566 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:02.758805   565 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:02.763535   758 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:02.765136   760 raft_consensus.cc:359] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.766573   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:02.769507   681 leader_election.cc:304] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.769778   564 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 0.
I20260709 14:50:02.770437   765 raft_consensus.cc:2804] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:02.770885   765 raft_consensus.cc:493] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:02.771255   765 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.771345   681 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.767138   760 raft_consensus.cc:385] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.773132   760 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.778242   760 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.794118   759 tablet_bootstrap.cc:492] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:02.803136   759 ts_tablet_manager.cc:1403] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.236s	user 0.024s	sys 0.028s
I20260709 14:50:02.813598   789 raft_consensus.cc:2804] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:02.814358   789 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:02.816210   789 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.814551   759 raft_consensus.cc:359] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.815744   607 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.820266   759 raft_consensus.cc:385] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.821950   759 raft_consensus.cc:740] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
W20260709 14:50:02.823948   607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43129, user_credentials={real_user=slave}} blocked reactor thread for 51014.6us
I20260709 14:50:02.823637   765 raft_consensus.cc:515] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.825620   773 raft_consensus.cc:2804] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:02.828486   759 consensus_queue.cc:260] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.829537   773 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:02.831283   773 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.834671   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:02.837044   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.836717   758 tablet_bootstrap.cc:654] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.844714   764 raft_consensus.cc:493] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.842598   760 ts_tablet_manager.cc:1434] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.153s	user 0.021s	sys 0.013s
I20260709 14:50:02.846215   764 raft_consensus.cc:515] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
W20260709 14:50:02.848117   529 leader_election.cc:343] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:02.848713   529 leader_election.cc:304] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f75c182005714280af142f2b0950c9f0; no voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:02.852499   760 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:02.855357   764 leader_election.cc:290] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.856464   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:02.853178   765 leader_election.cc:290] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:02.855511   759 ts_tablet_manager.cc:1434] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.006s
I20260709 14:50:02.861071   565 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.861229   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:02.860121   764 raft_consensus.cc:2749] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:02.862726   765 raft_consensus.cc:493] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.872740   773 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.868064   765 raft_consensus.cc:515] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.880364   759 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:02.885186   714 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:02.890703   773 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.892402   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:02.895022   564 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.894385   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:02.894516   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:02.896014   566 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:02.893946   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.900228   641 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:02.901386   681 leader_election.cc:304] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.891260   789 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.902020   529 leader_election.cc:304] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.903717   765 leader_election.cc:290] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:02.904037   790 raft_consensus.cc:2804] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:02.904563   715 raft_consensus.cc:2393] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ebfda1b7363403da68e5fa26a266f15 in current term 1: Already voted for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in this term.
I20260709 14:50:02.907099   790 raft_consensus.cc:493] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:02.907742   790 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.912578   789 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:02.909761   791 raft_consensus.cc:2804] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:02.913285   791 raft_consensus.cc:493] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:02.913719   791 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.918660   759 tablet_bootstrap.cc:654] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:02.918118   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.922333   641 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:02.918793   638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.923784   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:02.924856   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.926051   565 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 1.
I20260709 14:50:02.927898   640 raft_consensus.cc:2393] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d954c17b0ce4789aabfa880b6b0ddcf in current term 1: Already voted for candidate 2ebfda1b7363403da68e5fa26a266f15 in this term.
I20260709 14:50:02.927420   764 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:02.928614   638 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:02.934532   681 leader_election.cc:304] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:02.932539   764 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.924525   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:02.936350   639 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.935999   789 raft_consensus.cc:2804] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:02.927737   637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:02.938333   789 raft_consensus.cc:697] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:02.941624   637 raft_consensus.cc:2393] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9d954c17b0ce4789aabfa880b6b0ddcf in current term 1: Already voted for candidate 2ebfda1b7363403da68e5fa26a266f15 in this term.
I20260709 14:50:02.938277   764 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.942310   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:02.942414   789 consensus_queue.cc:237] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
W20260709 14:50:02.945734   532 leader_election.cc:343] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:02.944752   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:02.946625   758 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:02.949331   758 ts_tablet_manager.cc:1403] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.186s	user 0.022s	sys 0.003s
I20260709 14:50:02.947297   678 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0
I20260709 14:50:02.951936   765 raft_consensus.cc:2749] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:02.955695   760 tablet_bootstrap.cc:654] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:02.955111   529 leader_election.cc:343] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:02.957948   529 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f75c182005714280af142f2b0950c9f0; no voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:02.957765   790 raft_consensus.cc:515] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.967379   764 raft_consensus.cc:2749] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:02.968853   791 raft_consensus.cc:515] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.969337   564 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 1.
I20260709 14:50:02.971593   639 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 1.
I20260709 14:50:02.974912   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:02.975035   607 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:02.976035   565 raft_consensus.cc:2393] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9d954c17b0ce4789aabfa880b6b0ddcf in current term 1: Already voted for candidate f75c182005714280af142f2b0950c9f0 in this term.
I20260709 14:50:02.976136   758 raft_consensus.cc:359] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.978471   758 raft_consensus.cc:385] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:02.978984   758 raft_consensus.cc:740] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:02.979842   773 raft_consensus.cc:2804] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:02.980444   773 raft_consensus.cc:697] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:02.980131   758 consensus_queue.cc:260] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.981559   773 consensus_queue.cc:237] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.980386   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:02.976553   790 leader_election.cc:290] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:02.978348   760 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:02.985888   759 tablet_bootstrap.cc:492] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:02.986085   713 raft_consensus.cc:2393] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f75c182005714280af142f2b0950c9f0 in current term 1: Already voted for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in this term.
I20260709 14:50:02.986444   759 ts_tablet_manager.cc:1403] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.014s
I20260709 14:50:02.989049   758 ts_tablet_manager.cc:1434] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 14:50:02.983646   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:02.991565   640 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:02.984215   791 leader_election.cc:290] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:02.993271   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:02.995626   759 raft_consensus.cc:359] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:02.992600   760 ts_tablet_manager.cc:1403] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.142s	user 0.017s	sys 0.006s
I20260709 14:50:03.000975   678 leader_election.cc:304] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0
I20260709 14:50:03.002321   790 raft_consensus.cc:2749] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:02.998104   759 raft_consensus.cc:385] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.003131   758 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:03.002198   760 raft_consensus.cc:359] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.003943   759 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.005214   760 raft_consensus.cc:385] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.007498   760 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.007781   759 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.013522   760 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.019240   759 ts_tablet_manager.cc:1434] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.001s
I20260709 14:50:03.023660   759 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:03.028270   760 ts_tablet_manager.cc:1434] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.006s
I20260709 14:50:03.028105   640 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.031388   760 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:03.031826   529 leader_election.cc:304] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:03.032954   791 raft_consensus.cc:2804] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.033592   791 raft_consensus.cc:697] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:03.034979   791 consensus_queue.cc:237] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.039206   759 tablet_bootstrap.cc:654] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.039207   758 tablet_bootstrap.cc:654] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.045059   462 catalog_manager.cc:5697] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 1 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:03.047845   463 catalog_manager.cc:5697] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.071916   760 tablet_bootstrap.cc:654] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.075947   759 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:03.075946   758 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:03.076749   758 ts_tablet_manager.cc:1403] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.014s
I20260709 14:50:03.076750   759 ts_tablet_manager.cc:1403] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.013s
I20260709 14:50:03.093605   759 raft_consensus.cc:359] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.094791   759 raft_consensus.cc:385] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.095265   759 raft_consensus.cc:740] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.095620   461 catalog_manager.cc:5697] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.096342   759 consensus_queue.cc:260] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.097807   758 raft_consensus.cc:359] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.098788   758 raft_consensus.cc:385] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.099968   758 raft_consensus.cc:740] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.101245   760 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:03.101809   760 ts_tablet_manager.cc:1403] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.013s
I20260709 14:50:03.102506   759 ts_tablet_manager.cc:1434] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.009s
I20260709 14:50:03.103744   759 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:03.105319   760 raft_consensus.cc:359] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.106678   760 raft_consensus.cc:385] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.110325   760 raft_consensus.cc:740] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.110771   758 consensus_queue.cc:260] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.111819   760 consensus_queue.cc:260] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.114439   758 ts_tablet_manager.cc:1434] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 14:50:03.115196   760 ts_tablet_manager.cc:1434] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.005s
I20260709 14:50:03.115720   758 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:03.117100   760 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:03.118471   759 tablet_bootstrap.cc:654] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.148905   760 tablet_bootstrap.cc:654] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.143929   758 tablet_bootstrap.cc:654] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.167598   759 tablet_bootstrap.cc:492] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:03.168205   759 ts_tablet_manager.cc:1403] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.039s
I20260709 14:50:03.172562   759 raft_consensus.cc:359] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.174381   759 raft_consensus.cc:385] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.174713   759 raft_consensus.cc:740] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.175549   759 consensus_queue.cc:260] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.178332   759 ts_tablet_manager.cc:1434] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 14:50:03.179567   759 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:03.190004   759 tablet_bootstrap.cc:654] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.203637   759 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:03.204471   759 ts_tablet_manager.cc:1403] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 14:50:03.209403   759 raft_consensus.cc:359] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.210317   759 raft_consensus.cc:385] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.210650   759 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.211907   759 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.222364   759 ts_tablet_manager.cc:1434] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 14:50:03.226445   759 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:03.228999   790 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.229821   790 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.235603   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.237006   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
W20260709 14:50:03.237217   681 leader_election.cc:343] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:03.237619   759 tablet_bootstrap.cc:654] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.236727   790 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
W20260709 14:50:03.238896   678 leader_election.cc:343] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:03.239392   678 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0
I20260709 14:50:03.240561   791 raft_consensus.cc:493] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.240839   790 raft_consensus.cc:2749] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:03.241056   791 raft_consensus.cc:515] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.245486   791 leader_election.cc:290] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.246029   760 tablet_bootstrap.cc:492] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:03.246809   760 ts_tablet_manager.cc:1403] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.130s	user 0.026s	sys 0.028s
I20260709 14:50:03.247057   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.248993   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:03.249838   640 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:03.252287   529 leader_election.cc:304] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.253252   760 raft_consensus.cc:359] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.253777   791 raft_consensus.cc:2804] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.257753   791 raft_consensus.cc:493] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.258984   791 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.262248   758 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:03.262982   758 ts_tablet_manager.cc:1403] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.148s	user 0.034s	sys 0.058s
I20260709 14:50:03.254330   760 raft_consensus.cc:385] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.264590   760 raft_consensus.cc:740] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.265659   760 consensus_queue.cc:260] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.266688   713 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:03.266674   758 raft_consensus.cc:359] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.267553   758 raft_consensus.cc:385] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.268215   758 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.269747   790 raft_consensus.cc:493] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.269670   758 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.270275   790 raft_consensus.cc:515] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.276562   758 ts_tablet_manager.cc:1434] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 14:50:03.277047   790 leader_election.cc:290] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:03.277730   758 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: Bootstrap starting.
I20260709 14:50:03.280404   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:03.281211   640 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:03.282434   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.283233   565 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:03.285583   758 tablet_bootstrap.cc:654] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.285431   678 leader_election.cc:304] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:03.287372   790 raft_consensus.cc:2804] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.271229   760 ts_tablet_manager.cc:1434] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.006s
I20260709 14:50:03.288849   760 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:03.271445   791 raft_consensus.cc:515] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.270797   759 tablet_bootstrap.cc:492] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:03.291257   759 ts_tablet_manager.cc:1403] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.005s
I20260709 14:50:03.293453   791 leader_election.cc:290] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.294554   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:03.295295   640 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.299538   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:03.300349   713 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.300298   790 raft_consensus.cc:493] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.300942   790 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.301754   758 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: No bootstrap required, opened a new log
I20260709 14:50:03.302307   758 ts_tablet_manager.cc:1403] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.015s
I20260709 14:50:03.309167   759 raft_consensus.cc:359] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.310034   759 raft_consensus.cc:385] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.310312   759 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.311110   759 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.314041   760 tablet_bootstrap.cc:654] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.314250   758 raft_consensus.cc:359] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.315014   758 raft_consensus.cc:385] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.315292   758 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.316053   758 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.319960   758 ts_tablet_manager.cc:1434] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 14:50:03.326040   790 raft_consensus.cc:515] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.326736   759 ts_tablet_manager.cc:1434] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.007s
I20260709 14:50:03.328815   759 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: Bootstrap starting.
I20260709 14:50:03.331828   790 leader_election.cc:290] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:03.333783   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:03.335325   639 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.339146   790 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.341109   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:03.341758   565 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.349778   759 tablet_bootstrap.cc:654] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.353801   565 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 1.
I20260709 14:50:03.359325   681 leader_election.cc:304] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.360323   790 raft_consensus.cc:2804] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.361110   790 raft_consensus.cc:697] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:03.362980   790 consensus_queue.cc:237] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.363977   640 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.365994   760 tablet_bootstrap.cc:492] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:03.365726   529 leader_election.cc:304] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.366685   760 ts_tablet_manager.cc:1403] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.009s
I20260709 14:50:03.367806   791 raft_consensus.cc:2804] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.368359   791 raft_consensus.cc:697] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:03.369239   791 consensus_queue.cc:237] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.371212   759 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: No bootstrap required, opened a new log
I20260709 14:50:03.371917   759 ts_tablet_manager.cc:1403] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.003s
I20260709 14:50:03.371590   760 raft_consensus.cc:359] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.372638   760 raft_consensus.cc:385] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.372996   760 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.373775   760 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.379488   713 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.380761   760 ts_tablet_manager.cc:1434] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 14:50:03.384267   760 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: Bootstrap starting.
I20260709 14:50:03.376974   759 raft_consensus.cc:359] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.387753   759 raft_consensus.cc:385] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.388149   759 raft_consensus.cc:740] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.391845   639 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 1.
I20260709 14:50:03.389724   463 catalog_manager.cc:5697] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 1 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:03.389180   759 consensus_queue.cc:260] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.400861   759 ts_tablet_manager.cc:1434] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.001s
I20260709 14:50:03.403214   760 tablet_bootstrap.cc:654] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:03.406021   773 raft_consensus.cc:493] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.406651   773 raft_consensus.cc:515] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.409091   773 leader_election.cc:290] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.410279   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.412132   565 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 0.
I20260709 14:50:03.413286   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.414525   461 catalog_manager.cc:5697] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.419356   607 leader_election.cc:304] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.414063   713 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 0.
I20260709 14:50:03.423787   773 raft_consensus.cc:2804] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.424229   773 raft_consensus.cc:493] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.424597   773 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.433009   791 raft_consensus.cc:493] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.433686   791 raft_consensus.cc:515] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.442623   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:03.443992   639 raft_consensus.cc:2468] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:03.448455   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.442636   791 leader_election.cc:290] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.452183   760 tablet_bootstrap.cc:492] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: No bootstrap required, opened a new log
I20260709 14:50:03.451998   529 leader_election.cc:304] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.453639   773 raft_consensus.cc:515] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.457391   765 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.459477   790 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:03.463892   532 leader_election.cc:343] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:03.463963   760 ts_tablet_manager.cc:1403] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent bootstrapping tablet: real 0.080s	user 0.005s	sys 0.021s
I20260709 14:50:03.466378   773 leader_election.cc:290] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.464248   790 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.469002   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:03.469853   713 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.470394   790 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:03.472749   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.473779   565 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:03.477118   760 raft_consensus.cc:359] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.478057   760 raft_consensus.cc:385] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:03.478391   760 raft_consensus.cc:740] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Initialized, Role: FOLLOWER
I20260709 14:50:03.479602   760 consensus_queue.cc:260] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.479916   681 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.481550   821 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.481998   821 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.482190   760 ts_tablet_manager.cc:1434] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 14:50:03.482388   821 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.483438   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:03.484228   639 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 0.
I20260709 14:50:03.485116   773 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.486058   773 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.488466   773 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.490801   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:03.492964   564 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.495965   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.499917   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.501510   565 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 0.
I20260709 14:50:03.505232   564 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 1.
I20260709 14:50:03.500170   791 raft_consensus.cc:2804] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.508800   767 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:03.510620   791 raft_consensus.cc:493] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.511122   791 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.508235   821 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.509920   607 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.513823   767 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.514263   767 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.514961   767 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.518139   713 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 1.
I20260709 14:50:03.519460   607 leader_election.cc:304] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.520905   715 raft_consensus.cc:2393] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ebfda1b7363403da68e5fa26a266f15 in current term 1: Already voted for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in this term.
I20260709 14:50:03.527957   773 raft_consensus.cc:2804] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.528939   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:03.531703   791 raft_consensus.cc:515] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.534945   773 raft_consensus.cc:697] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:03.539628   564 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.538719   821 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:03.550437   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:03.551270   715 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.555399   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:03.556111   640 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.536932   773 consensus_queue.cc:237] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.557193   767 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.568394   638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:03.576938   767 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.577069   638 raft_consensus.cc:2393] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9d954c17b0ce4789aabfa880b6b0ddcf in current term 1: Already voted for candidate 2ebfda1b7363403da68e5fa26a266f15 in this term.
I20260709 14:50:03.579598   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:03.579562   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:03.581475   713 raft_consensus.cc:2393] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ebfda1b7363403da68e5fa26a266f15 in current term 1: Already voted for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in this term.
I20260709 14:50:03.556435   791 leader_election.cc:290] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.585076   825 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.585752   825 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.598613   764 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:03.599325   764 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.602386   764 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.588423   767 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 14:50:03.605053   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.608069   713 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:03.611476   607 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15; no voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0
I20260709 14:50:03.613833   564 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 1.
W20260709 14:50:03.611974   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:03.615867   826 raft_consensus.cc:2749] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:03.617158   532 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.618100   791 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:03.618531   791 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:03.618968   791 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.621110   715 raft_consensus.cc:2468] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.621268   825 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.630982   640 raft_consensus.cc:2468] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.637111   791 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.644068   791 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:03.648272   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" is_pre_election: true
I20260709 14:50:03.649559   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15" is_pre_election: true
I20260709 14:50:03.649313   461 catalog_manager.cc:5697] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.651895   640 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 0.
I20260709 14:50:03.653597   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:03.654117   532 leader_election.cc:304] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.654315   639 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.651895   715 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 0.
I20260709 14:50:03.656888   791 consensus_queue.cc:1048] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.659247   791 raft_consensus.cc:2804] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.660257   791 raft_consensus.cc:697] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:03.661324   791 consensus_queue.cc:237] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.667711   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75c182005714280af142f2b0950c9f0" is_pre_election: true
I20260709 14:50:03.668040   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:03.668711   566 raft_consensus.cc:2393] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ebfda1b7363403da68e5fa26a266f15 in current term 1: Already voted for candidate f75c182005714280af142f2b0950c9f0 in this term.
I20260709 14:50:03.668704   714 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:03.672338   764 consensus_queue.cc:1048] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:03.674499   607 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:03.675016   681 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:03.675906   821 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.676433   821 raft_consensus.cc:697] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:03.677325   821 consensus_queue.cc:237] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.684746   639 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.685398   767 raft_consensus.cc:2764] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 14:50:03.688798   714 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 1.
I20260709 14:50:03.691560   532 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:03.692687   764 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:03.698802   764 raft_consensus.cc:697] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:03.715504   764 consensus_queue.cc:237] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:03.747778   463 catalog_manager.cc:5697] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 1 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:03.767854   462 catalog_manager.cc:5697] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.784822   791 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.792275   821 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.829680   764 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:03.830854   790 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:03.843421   463 catalog_manager.cc:5697] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:03.961861   767 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:03.979359   773 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:04.101478   764 consensus_queue.cc:1048] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:04.109885   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:45362
I20260709 14:50:04.110700   640 raft_consensus.cc:606] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:04.118153   764 consensus_queue.cc:1048] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:04.139748   764 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:04.149749   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:55856
I20260709 14:50:04.150583   713 raft_consensus.cc:606] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:04.161741   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:45362
I20260709 14:50:04.161932   837 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:04.162503   640 raft_consensus.cc:606] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:04.176955   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:55856
I20260709 14:50:04.178417   715 raft_consensus.cc:606] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:04.186882   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:55856
I20260709 14:50:04.188447   715 raft_consensus.cc:606] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:04.202219   825 raft_consensus.cc:993] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.202759   773 raft_consensus.cc:1081] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.205384   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42384
I20260709 14:50:04.206067   566 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.206379   566 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.212966   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:04.213752   565 raft_consensus.cc:606] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:04.214026   566 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.217296   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:04.218000   640 raft_consensus.cc:3055] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.218345   640 raft_consensus.cc:740] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.218618   566 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.219341   640 consensus_queue.cc:260] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.221028   640 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.221354   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:04.221583   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.222019   566 raft_consensus.cc:606] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:04.222307   715 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.228991   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:04.230608   566 raft_consensus.cc:606] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:04.230582   640 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.233285   529 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:04.234220   837 raft_consensus.cc:2804] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.234743   837 raft_consensus.cc:697] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.235744   837 consensus_queue.cc:237] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.236817   765 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:04.238994   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:04.239838   566 raft_consensus.cc:606] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Received request to transfer leadership to TS 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:04.256376   461 catalog_manager.cc:5697] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 1 to 2, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.257441   765 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:04.263202   715 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.284382   822 raft_consensus.cc:993] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.285851   822 raft_consensus.cc:1081] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.289505   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42400
I20260709 14:50:04.290270   565 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.290702   565 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.298439   565 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.301257   565 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.303791   566 raft_consensus.cc:1240] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Rejecting Update request from peer 9d954c17b0ce4789aabfa880b6b0ddcf for earlier term 1. Current term is 2. Ops: []
I20260709 14:50:04.304069   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.304824   715 raft_consensus.cc:3055] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.306237   715 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:04.306996   837 raft_consensus.cc:993] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.307574   791 raft_consensus.cc:1081] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.307948   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:04.308610   639 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.309185   790 consensus_queue.cc:1059] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:04.310401   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:04.311148   640 raft_consensus.cc:493] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.311584   640 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.310722   715 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.316498   715 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.318414   639 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.320070   529 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:04.320689   640 raft_consensus.cc:515] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.321285   837 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.321882   837 raft_consensus.cc:697] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.323266   837 consensus_queue.cc:237] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.324414   640 leader_election.cc:290] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.327520   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.328200   713 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.332418   715 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
W20260709 14:50:04.333142   790 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 14:50:04.335726   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cbd11606124c0a82fcdae85eebf9a6" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:04.336629   566 raft_consensus.cc:3055] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.337081   566 raft_consensus.cc:740] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.336002   462 catalog_manager.cc:5697] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 1 to 2, leader changed from 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.338212   713 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.338716   566 consensus_queue.cc:260] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.340171   566 raft_consensus.cc:3060] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.340360   607 leader_election.cc:304] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:04.341305   773 raft_consensus.cc:2804] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.341837   773 raft_consensus.cc:697] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.342798   773 consensus_queue.cc:237] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.349788   566 raft_consensus.cc:2468] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.360437   462 catalog_manager.cc:5697] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 1 to 2, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.397390   791 raft_consensus.cc:993] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.398620   791 raft_consensus.cc:1081] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.400987   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:04.401714   639 raft_consensus.cc:493] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.402140   639 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.408629   639 raft_consensus.cc:515] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.410722   639 leader_election.cc:290] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.411696   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:04.412168   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.412433   566 raft_consensus.cc:3055] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.412791   713 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.412827   566 raft_consensus.cc:740] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.413872   566 consensus_queue.cc:260] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.415308   566 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.420006   713 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.422181   607 leader_election.cc:304] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:04.423120   773 raft_consensus.cc:2804] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.423233   566 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.423774   790 raft_consensus.cc:993] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.423702   773 raft_consensus.cc:697] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.424288   822 raft_consensus.cc:1081] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.425973   773 consensus_queue.cc:237] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.427233   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42400
I20260709 14:50:04.427886   566 raft_consensus.cc:493] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.428262   566 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.436285   566 raft_consensus.cc:515] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.437125   825 raft_consensus.cc:993] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.435856   462 catalog_manager.cc:5697] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 1 to 2, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.437673   825 raft_consensus.cc:1081] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:04.439180   566 leader_election.cc:290] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.439954   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.440117   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42384
I20260709 14:50:04.440644   713 raft_consensus.cc:3055] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.441017   713 raft_consensus.cc:740] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:04.441354   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:04.441692   565 raft_consensus.cc:493] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.442126   565 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.441926   713 consensus_queue.cc:260] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.443437   713 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.441938   639 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.451045   565 raft_consensus.cc:515] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.454376   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:04.455118   640 raft_consensus.cc:3055] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.455468   640 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.456408   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.456357   640 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.457115   715 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.457638   640 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.463801   713 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.454348   639 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.466358   532 leader_election.cc:304] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:04.466897   640 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.467285   837 raft_consensus.cc:2804] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.467909   837 raft_consensus.cc:697] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.468183   565 leader_election.cc:290] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.468863   837 consensus_queue.cc:237] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.470109   715 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 2.
I20260709 14:50:04.470122   529 leader_election.cc:304] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:04.471923   764 raft_consensus.cc:2804] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.472493   764 raft_consensus.cc:697] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.473261   764 consensus_queue.cc:237] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.480984   462 catalog_manager.cc:5697] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 1 to 2, leader changed from 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.497151   462 catalog_manager.cc:5697] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 1 to 2, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.520257   790 raft_consensus.cc:993] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.520802   822 raft_consensus.cc:1081] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.522478   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45346
I20260709 14:50:04.523145   640 raft_consensus.cc:493] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.523532   640 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.532071   640 raft_consensus.cc:515] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.534396   640 leader_election.cc:290] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.535568   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:04.535737   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.536257   565 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.536391   715 raft_consensus.cc:3055] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.536837   715 raft_consensus.cc:740] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:04.537648   715 consensus_queue.cc:260] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.538688   715 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.547010   715 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.547011   565 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.548892   607 leader_election.cc:304] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:04.549703   773 raft_consensus.cc:2804] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.550227   773 raft_consensus.cc:697] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.551208   773 consensus_queue.cc:237] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.560276   462 catalog_manager.cc:5697] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 1 to 2, leader changed from 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.571916   791 raft_consensus.cc:993] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.573299   791 raft_consensus.cc:1081] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:04.575167   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:04.575804   640 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.576171   640 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.583595   640 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.585817   640 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:04.587970   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:04.588001   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:04.588721   565 raft_consensus.cc:3055] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.588778   715 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.589120   565 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.590127   565 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.591355   565 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.597438   715 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.597810   565 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 2.
I20260709 14:50:04.599017   607 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:04.599969   773 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.600483   773 raft_consensus.cc:697] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:04.601348   773 consensus_queue.cc:237] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.609376   461 catalog_manager.cc:5697] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 1 to 2, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:04.627756   715 raft_consensus.cc:1275] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.630756   791 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:04.642411   640 raft_consensus.cc:1275] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.645087   791 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:04.670013   834 raft_consensus.cc:993] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: : Instructing follower 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:04.670527   791 raft_consensus.cc:1081] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Signalling peer 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:04.672959   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:55832
I20260709 14:50:04.673650   715 raft_consensus.cc:493] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:04.674085   715 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.679807   715 raft_consensus.cc:515] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.682106   715 leader_election.cc:290] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 2 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:04.683302   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:04.683421   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d2bc34dd22b4481801f97b7b84229e2" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:04.685101   640 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.685107   565 raft_consensus.cc:3055] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:04.685858   565 raft_consensus.cc:740] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:04.686892   565 consensus_queue.cc:260] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.688429   565 raft_consensus.cc:3060] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:04.695966   640 raft_consensus.cc:2468] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 2.
I20260709 14:50:04.697738   678 leader_election.cc:304] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:04.698719   565 raft_consensus.cc:2468] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 2.
I20260709 14:50:04.698895   822 raft_consensus.cc:2804] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:04.699620   822 raft_consensus.cc:697] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:04.700698   822 consensus_queue.cc:237] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:04.712218   462 catalog_manager.cc:5697] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 1 to 2, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 2 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:04.723628   715 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.725308   791 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:04.739610   640 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.741107   764 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:04.757565   565 raft_consensus.cc:1275] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.759938   773 consensus_queue.cc:1048] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:04.773324   715 raft_consensus.cc:1275] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.774937   773 consensus_queue.cc:1048] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:04.905728   715 raft_consensus.cc:1275] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.907174   773 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:04.917033   565 raft_consensus.cc:1275] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.919471   773 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:04.948074   715 raft_consensus.cc:1275] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.949554   773 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:04.959273   565 raft_consensus.cc:1275] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:04.962527   857 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.013079   640 raft_consensus.cc:1275] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.014772   764 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.015476   715 raft_consensus.cc:1275] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.017264   764 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.029266   713 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.031836   791 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.032516   565 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.033960   857 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:05.045815   714 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.055323   640 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.057209   834 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.057509   825 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:05.283583   640 raft_consensus.cc:1275] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.285247   822 consensus_queue.cc:1048] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:05.297508   566 raft_consensus.cc:1275] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:05.299029   869 consensus_queue.cc:1048] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:07.277091   431 auto_leader_rebalancer-test.cc:378] The leader distribution is 
I20260709 14:50:07.278074   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:07.308214   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:07.308869   639 raft_consensus.cc:606] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Received request to transfer leadership to TS 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:07.320144   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:42418
I20260709 14:50:07.320763   566 raft_consensus.cc:606] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Received request to transfer leadership to TS 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:07.322041   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:50:07.322566   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:50:07.442299   791 raft_consensus.cc:993] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: : Instructing follower 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:07.442915   878 raft_consensus.cc:1081] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Signalling peer 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:07.444680   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:55832
I20260709 14:50:07.445236   714 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:07.445518   714 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.450939   714 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.452917   714 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:07.453833   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:07.454166   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:07.454437   566 raft_consensus.cc:3055] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:07.454780   639 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.454790   566 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:07.455734   566 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.456827   566 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.461009   639 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:07.462380   678 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:07.463050   566 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:07.463486   872 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:07.465061   872 raft_consensus.cc:697] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:07.466046   872 consensus_queue.cc:237] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.473907   462 catalog_manager.cc:5697] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 2 to 3, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 3 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:07.778378   825 raft_consensus.cc:993] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:07.780455   885 raft_consensus.cc:1081] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Signalling peer 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:07.782362   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:55846
I20260709 14:50:07.783010   714 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:07.783349   714 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.788715   714 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.790767   714 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:07.791893   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:07.791826   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:07.792616   639 raft_consensus.cc:3055] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:07.792647   566 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.793133   639 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:07.794003   639 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.796190   639 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:07.801591   566 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:07.803139   681 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:07.803639   639 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:07.803987   872 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:07.804512   872 raft_consensus.cc:697] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:07.806149   872 consensus_queue.cc:237] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:07.813484   462 catalog_manager.cc:5697] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 2 to 3, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 3 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:07.883687   639 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:07.885221   872 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:07.899055   566 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:07.901875   879 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:08.259356   639 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:08.259413   566 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:08.261096   872 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:08.261731   890 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:08.323652   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:08.335114   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:08.335758   566 raft_consensus.cc:606] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:08.341507   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:55856
I20260709 14:50:08.342104   714 raft_consensus.cc:606] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:08.348299   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:08.348964   566 raft_consensus.cc:606] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:08.354676   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:42416
I20260709 14:50:08.355332   566 raft_consensus.cc:606] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:08.367620   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
mode: GRACEFUL
new_leader_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:55856
I20260709 14:50:08.369115   714 raft_consensus.cc:606] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Received request to transfer leadership to TS 2ebfda1b7363403da68e5fa26a266f15
I20260709 14:50:08.396847   872 raft_consensus.cc:993] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.397372   872 raft_consensus.cc:1081] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.399255   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45346
I20260709 14:50:08.399899   639 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:08.400254   639 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.406621   639 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.408520   639 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 4 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:08.409721   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:08.409878   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:08.410439   566 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.410634   714 raft_consensus.cc:3055] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:08.411126   714 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:08.411978   714 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.413259   714 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.417874   566 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 4.
I20260709 14:50:08.419327   607 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:08.419581   714 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 4.
I20260709 14:50:08.420181   885 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:08.421306   885 raft_consensus.cc:697] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:08.422266   885 consensus_queue.cc:237] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.429399   463 catalog_manager.cc:5697] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 3 to 4, leader changed from 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:08.595403   900 raft_consensus.cc:993] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.596076   878 raft_consensus.cc:1081] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.598105   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:08.598713   639 raft_consensus.cc:493] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:08.599145   639 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.603874   639 raft_consensus.cc:515] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.605690   639 leader_election.cc:290] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:08.606709   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:08.607108   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ad604df88241eaadb947764d02e45d" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:08.607316   566 raft_consensus.cc:3055] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:08.607694   566 raft_consensus.cc:740] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:08.607746   714 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.608537   566 consensus_queue.cc:260] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.609548   566 raft_consensus.cc:3060] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.615038   566 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.615109   714 raft_consensus.cc:2468] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.616204   607 leader_election.cc:304] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:08.616894   885 raft_consensus.cc:2804] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:08.617419   885 raft_consensus.cc:697] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:08.618251   885 consensus_queue.cc:237] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.625310   463 catalog_manager.cc:5697] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 2 to 3, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:08.691931   879 raft_consensus.cc:993] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.692487   872 raft_consensus.cc:1081] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.694723   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45346
I20260709 14:50:08.695405   639 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:08.695772   639 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.701095   639 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.702734   639 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 4 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:08.703756   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:08.704037   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:08.704427   566 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.704582   714 raft_consensus.cc:3055] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:08.704931   714 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:08.705806   714 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.707095   714 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:08.711885   566 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 4.
I20260709 14:50:08.715468   607 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:08.715639   714 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 4.
I20260709 14:50:08.716264   885 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:08.716827   885 raft_consensus.cc:697] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:08.717800   885 consensus_queue.cc:237] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.724859   463 catalog_manager.cc:5697] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 3 to 4, leader changed from 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:08.760965   900 raft_consensus.cc:993] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.761458   878 raft_consensus.cc:1081] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.763171   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:08.763754   639 raft_consensus.cc:493] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:08.764081   639 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.769166   639 raft_consensus.cc:515] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.770849   639 leader_election.cc:290] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:08.771884   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:08.772068   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:08.772418   566 raft_consensus.cc:3055] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:08.772634   714 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.772748   566 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:08.773443   566 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.774318   566 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.779970   566 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.779908   714 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.781240   607 leader_election.cc:304] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:08.782059   885 raft_consensus.cc:2804] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:08.782541   885 raft_consensus.cc:697] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:08.783288   885 consensus_queue.cc:237] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.791184   463 catalog_manager.cc:5697] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 2 to 3, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:08.804307   566 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:08.805768   885 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:08.821635   714 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:08.823112   904 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:08.859977   900 raft_consensus.cc:993] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: : Instructing follower 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.860489   878 raft_consensus.cc:1081] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Signalling peer 2ebfda1b7363403da68e5fa26a266f15 to start an election
I20260709 14:50:08.862092   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
 from {username='slave'} at 127.0.0.1:45332
I20260709 14:50:08.862627   639 raft_consensus.cc:493] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:08.862931   639 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.867969   639 raft_consensus.cc:515] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.869781   639 leader_election.cc:290] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:08.870724   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:08.871465   566 raft_consensus.cc:3055] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:08.871773   566 raft_consensus.cc:740] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:08.871644   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d56a87183594c8caf86cda36c1e4ba3" candidate_uuid: "2ebfda1b7363403da68e5fa26a266f15" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:08.872380   714 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.872686   566 consensus_queue.cc:260] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.873677   566 raft_consensus.cc:3060] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:08.878221   714 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.878324   566 raft_consensus.cc:2468] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ebfda1b7363403da68e5fa26a266f15 in term 3.
I20260709 14:50:08.879635   607 leader_election.cc:304] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebfda1b7363403da68e5fa26a266f15, 9d954c17b0ce4789aabfa880b6b0ddcf; no voters: 
I20260709 14:50:08.880398   904 raft_consensus.cc:2804] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:08.880897   904 raft_consensus.cc:697] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Becoming Leader. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:08.881740   904 consensus_queue.cc:237] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:08.890334   460 catalog_manager.cc:5697] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 reported cstate change: term changed from 2 to 3, leader changed from f75c182005714280af142f2b0950c9f0 (127.0.107.193) to 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "2ebfda1b7363403da68e5fa26a266f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:09.011397   714 raft_consensus.cc:1275] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.014815   904 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:09.030023   566 raft_consensus.cc:1275] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.032711   904 consensus_queue.cc:1048] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:09.222810   714 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:09.224215   904 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:09.227804   566 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.229800   885 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:09.236390   565 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:09.238278   904 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:09.252269   713 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.253738   885 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:09.278352   566 raft_consensus.cc:1275] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.280049   904 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:09.292347   713 raft_consensus.cc:1275] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Refusing update from remote peer 2ebfda1b7363403da68e5fa26a266f15: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:09.295861   896 consensus_queue.cc:1048] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:11.378175   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:11.387033   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:11.387758   639 raft_consensus.cc:606] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:11.390586   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:11.391115   639 raft_consensus.cc:606] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Received request to transfer leadership to TS 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:11.393476   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:11.393947   639 raft_consensus.cc:606] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:11.396384   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:11.396919   639 raft_consensus.cc:606] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Received request to transfer leadership to TS 9d954c17b0ce4789aabfa880b6b0ddcf
I20260709 14:50:11.399210   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
mode: GRACEFUL
new_leader_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:45368
I20260709 14:50:11.399669   639 raft_consensus.cc:606] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Received request to transfer leadership to TS f75c182005714280af142f2b0950c9f0
I20260709 14:50:11.400610   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:50:11.401175   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:50:11.683133   904 raft_consensus.cc:993] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:11.683666   916 raft_consensus.cc:1081] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Signalling peer 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:11.685429   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:55846
I20260709 14:50:11.686017   714 raft_consensus.cc:493] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:11.686365   714 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:11.692423   714 raft_consensus.cc:515] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.694136   714 leader_election.cc:290] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 4 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:11.695258   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:11.695255   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eb9c71a8f442e2b26a3b1394459370" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:11.696157   565 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:11.696209   639 raft_consensus.cc:3055] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:11.696715   639 raft_consensus.cc:740] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:11.697649   639 consensus_queue.cc:260] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.699097   639 raft_consensus.cc:3060] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:11.705148   565 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 4.
I20260709 14:50:11.706647   681 leader_election.cc:304] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:11.707043   639 raft_consensus.cc:2468] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 4.
I20260709 14:50:11.707692   872 raft_consensus.cc:2804] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:11.708734   872 raft_consensus.cc:697] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:11.709690   872 consensus_queue.cc:237] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.717113   463 catalog_manager.cc:5697] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 3 to 4, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 4 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:11.727103   904 raft_consensus.cc:993] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.727806   916 raft_consensus.cc:1081] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.730141   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42384
I20260709 14:50:11.730760   565 raft_consensus.cc:493] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:11.731238   565 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.738198   565 raft_consensus.cc:515] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.739964   565 leader_election.cc:290] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 3 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:11.741228   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:11.741178   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "884c9111d1f249b686aa232a2885d25f" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:11.741909   714 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.742025   639 raft_consensus.cc:3055] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:11.742375   639 raft_consensus.cc:740] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:11.743125   639 consensus_queue.cc:260] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.744066   639 raft_consensus.cc:3060] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.748729   714 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 3.
I20260709 14:50:11.750480   639 raft_consensus.cc:2468] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 3.
I20260709 14:50:11.750545   532 leader_election.cc:304] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:11.752573   923 raft_consensus.cc:2804] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:11.753683   923 raft_consensus.cc:697] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 3 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:11.754547   923 consensus_queue.cc:237] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.765707   463 catalog_manager.cc:5697] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 2 to 3, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:11.799520   904 raft_consensus.cc:993] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:11.800050   904 raft_consensus.cc:1081] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Signalling peer 9d954c17b0ce4789aabfa880b6b0ddcf to start an election
I20260709 14:50:11.801756   714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55"
dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
 from {username='slave'} at 127.0.0.1:55846
I20260709 14:50:11.802289   714 raft_consensus.cc:493] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:11.802594   714 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.811921   714 raft_consensus.cc:515] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.814271   714 leader_election.cc:290] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Requested vote from peers f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129), 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861)
I20260709 14:50:11.815215   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f75c182005714280af142f2b0950c9f0"
I20260709 14:50:11.815562   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beecff85e664cc7b70aea711d0ebe55" candidate_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:11.815893   565 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.816242   639 raft_consensus.cc:3055] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:11.816660   639 raft_consensus.cc:740] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:11.817500   639 consensus_queue.cc:260] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.818743   639 raft_consensus.cc:3060] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:11.819646   904 raft_consensus.cc:993] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.820145   916 raft_consensus.cc:1081] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.821954   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42384
I20260709 14:50:11.822515   566 raft_consensus.cc:493] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:11.822880   566 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.823383   565 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:11.824591   681 leader_election.cc:304] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:11.825364   872 raft_consensus.cc:2804] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:11.825799   639 raft_consensus.cc:2468] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d954c17b0ce4789aabfa880b6b0ddcf in term 3.
I20260709 14:50:11.825842   872 raft_consensus.cc:697] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Becoming Leader. State: Replica: 9d954c17b0ce4789aabfa880b6b0ddcf, State: Running, Role: LEADER
I20260709 14:50:11.827527   872 consensus_queue.cc:237] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.831226   566 raft_consensus.cc:515] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.834101   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:11.834509   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a322c0390a314662a84f1c53ab04b00d" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:11.834698   639 raft_consensus.cc:3055] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:50:11.835135   714 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.835078   566 leader_election.cc:290] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 5 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:11.835107   639 raft_consensus.cc:740] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:11.836172   463 catalog_manager.cc:5697] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf reported cstate change: term changed from 2 to 3, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195). New cstate: current_term: 3 leader_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:11.836845   639 consensus_queue.cc:260] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.838040   639 raft_consensus.cc:3060] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.841737   714 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 5.
I20260709 14:50:11.843335   532 leader_election.cc:304] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:11.843765   639 raft_consensus.cc:2468] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 5.
I20260709 14:50:11.844154   923 raft_consensus.cc:2804] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:50:11.844713   923 raft_consensus.cc:697] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 5 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:11.845642   923 consensus_queue.cc:237] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.855422   463 catalog_manager.cc:5697] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 4 to 5, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:11.868273   904 raft_consensus.cc:993] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: : Instructing follower f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.868752   904 raft_consensus.cc:1081] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Signalling peer f75c182005714280af142f2b0950c9f0 to start an election
I20260709 14:50:11.870088   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400cb3b23ea14886a356218695773ffa"
dest_uuid: "f75c182005714280af142f2b0950c9f0"
 from {username='slave'} at 127.0.0.1:42384
I20260709 14:50:11.870579   566 raft_consensus.cc:493] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:11.870929   566 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.876696   566 raft_consensus.cc:515] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.878350   566 leader_election.cc:290] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 5 election: Requested vote from peers 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861), 9d954c17b0ce4789aabfa880b6b0ddcf (127.0.107.195:33851)
I20260709 14:50:11.879211   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2ebfda1b7363403da68e5fa26a266f15"
I20260709 14:50:11.879508   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400cb3b23ea14886a356218695773ffa" candidate_uuid: "f75c182005714280af142f2b0950c9f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf"
I20260709 14:50:11.879928   639 raft_consensus.cc:3055] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:50:11.880141   714 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.880295   639 raft_consensus.cc:740] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2ebfda1b7363403da68e5fa26a266f15, State: Running, Role: LEADER
I20260709 14:50:11.881196   639 consensus_queue.cc:260] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.882474   639 raft_consensus.cc:3060] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:11.887140   714 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 5.
I20260709 14:50:11.887894   639 raft_consensus.cc:2468] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f75c182005714280af142f2b0950c9f0 in term 5.
I20260709 14:50:11.888500   532 leader_election.cc:304] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d954c17b0ce4789aabfa880b6b0ddcf, f75c182005714280af142f2b0950c9f0; no voters: 
I20260709 14:50:11.889310   923 raft_consensus.cc:2804] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:50:11.889741   923 raft_consensus.cc:697] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 5 LEADER]: Becoming Leader. State: Replica: f75c182005714280af142f2b0950c9f0, State: Running, Role: LEADER
I20260709 14:50:11.890581   923 consensus_queue.cc:237] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } }
I20260709 14:50:11.897738   463 catalog_manager.cc:5697] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 reported cstate change: term changed from 4 to 5, leader changed from 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194) to f75c182005714280af142f2b0950c9f0 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "f75c182005714280af142f2b0950c9f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:12.149055   714 raft_consensus.cc:1275] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:12.150573   923 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:12.162026   639 raft_consensus.cc:1275] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:12.163672   923 consensus_queue.cc:1048] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:12.173291   640 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:12.174727   872 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:12.185297   566 raft_consensus.cc:1275] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:12.187651   917 consensus_queue.cc:1048] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:12.271857   640 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:12.273627   923 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:50:12.286991   714 raft_consensus.cc:1275] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:12.291245   939 consensus_queue.cc:1048] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:50:12.305748   566 raft_consensus.cc:1275] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:12.307344   917 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75c182005714280af142f2b0950c9f0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:12.321151   714 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:12.326807   939 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d954c17b0ce4789aabfa880b6b0ddcf" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:50:12.335816   640 raft_consensus.cc:1275] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Refusing update from remote peer 9d954c17b0ce4789aabfa880b6b0ddcf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:12.339735   942 consensus_queue.cc:1048] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:12.349421   639 raft_consensus.cc:1275] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Refusing update from remote peer f75c182005714280af142f2b0950c9f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:12.354095   924 consensus_queue.cc:1048] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebfda1b7363403da68e5fa26a266f15" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:50:12.402237   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:12.409660   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 14:50:12.443141   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:12.444178   431 tablet_replica.cc:333] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.445132   431 raft_consensus.cc:2243] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.445765   431 raft_consensus.cc:2272] T 4d2bc34dd22b4481801f97b7b84229e2 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:12.471750   607 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:50:12.475566   607 consensus_peers.cc:597] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 -> Peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Couldn't send request to peer f75c182005714280af142f2b0950c9f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:12.525010   431 tablet_replica.cc:333] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.526118   431 raft_consensus.cc:2243] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.526886   431 raft_consensus.cc:2272] T 90eb9c71a8f442e2b26a3b1394459370 P f75c182005714280af142f2b0950c9f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.530066   431 tablet_replica.cc:333] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.530650   431 raft_consensus.cc:2243] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 5 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.531577   431 raft_consensus.cc:2272] T 400cb3b23ea14886a356218695773ffa P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.533754   431 tablet_replica.cc:333] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.534327   431 raft_consensus.cc:2243] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.534745   431 raft_consensus.cc:2272] T 6beecff85e664cc7b70aea711d0ebe55 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.537285   431 tablet_replica.cc:333] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.537817   431 raft_consensus.cc:2243] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.538301   431 raft_consensus.cc:2272] T a7cbd11606124c0a82fcdae85eebf9a6 P f75c182005714280af142f2b0950c9f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.540683   431 tablet_replica.cc:333] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.541320   431 raft_consensus.cc:2243] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 5 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.542272   431 raft_consensus.cc:2272] T a322c0390a314662a84f1c53ab04b00d P f75c182005714280af142f2b0950c9f0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.544703   431 tablet_replica.cc:333] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.545536   431 raft_consensus.cc:2243] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.546404   431 raft_consensus.cc:2272] T 4d56a87183594c8caf86cda36c1e4ba3 P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.548946   431 tablet_replica.cc:333] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.549605   431 raft_consensus.cc:2243] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.550256   431 raft_consensus.cc:2272] T c2ad604df88241eaadb947764d02e45d P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:12.551517   681 consensus_peers.cc:597] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf -> Peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Couldn't send request to peer f75c182005714280af142f2b0950c9f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:12.552510   431 tablet_replica.cc:333] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0: stopping tablet replica
I20260709 14:50:12.553077   431 raft_consensus.cc:2243] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.553941   431 raft_consensus.cc:2272] T 884c9111d1f249b686aa232a2885d25f P f75c182005714280af142f2b0950c9f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.698066   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:50:12.698807   607 consensus_peers.cc:597] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 -> Peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Couldn't send request to peer f75c182005714280af142f2b0950c9f0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:12.699082   681 consensus_peers.cc:597] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf -> Peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Couldn't send request to peer f75c182005714280af142f2b0950c9f0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:12.708928   681 consensus_peers.cc:597] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf -> Peer f75c182005714280af142f2b0950c9f0 (127.0.107.193:43129): Couldn't send request to peer f75c182005714280af142f2b0950c9f0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:12.724116   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 14:50:12.761750   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:12.762491   431 tablet_replica.cc:333] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.763268   431 raft_consensus.cc:2243] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.763770   431 raft_consensus.cc:2272] T 4d2bc34dd22b4481801f97b7b84229e2 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.765827   431 tablet_replica.cc:333] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.766477   431 raft_consensus.cc:2243] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.767441   431 raft_consensus.cc:2272] T 4d56a87183594c8caf86cda36c1e4ba3 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.769321   431 tablet_replica.cc:333] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.769896   431 raft_consensus.cc:2243] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.770754   431 raft_consensus.cc:2272] T a7cbd11606124c0a82fcdae85eebf9a6 P 2ebfda1b7363403da68e5fa26a266f15 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.772768   431 tablet_replica.cc:333] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.773357   431 raft_consensus.cc:2243] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.773808   431 raft_consensus.cc:2272] T a322c0390a314662a84f1c53ab04b00d P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.776182   431 tablet_replica.cc:333] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.776736   431 raft_consensus.cc:2243] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.777179   431 raft_consensus.cc:2272] T 400cb3b23ea14886a356218695773ffa P 2ebfda1b7363403da68e5fa26a266f15 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.779474   431 tablet_replica.cc:333] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.780018   431 raft_consensus.cc:2243] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.780449   431 raft_consensus.cc:2272] T 90eb9c71a8f442e2b26a3b1394459370 P 2ebfda1b7363403da68e5fa26a266f15 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.782816   431 tablet_replica.cc:333] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.783417   431 raft_consensus.cc:2243] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.783860   431 raft_consensus.cc:2272] T 6beecff85e664cc7b70aea711d0ebe55 P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.785953   431 tablet_replica.cc:333] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.786495   431 raft_consensus.cc:2243] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.787036   431 raft_consensus.cc:2272] T 884c9111d1f249b686aa232a2885d25f P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.789393   431 tablet_replica.cc:333] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15: stopping tablet replica
I20260709 14:50:12.789913   431 raft_consensus.cc:2243] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.790715   431 raft_consensus.cc:2272] T c2ad604df88241eaadb947764d02e45d P 2ebfda1b7363403da68e5fa26a266f15 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.823468   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 14:50:12.841809   678 consensus_peers.cc:597] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf -> Peer 2ebfda1b7363403da68e5fa26a266f15 (127.0.107.194:45861): Couldn't send request to peer 2ebfda1b7363403da68e5fa26a266f15. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:45861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:12.848696   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 14:50:12.885842   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:12.886768   431 tablet_replica.cc:333] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.887579   431 raft_consensus.cc:2243] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.888571   431 raft_consensus.cc:2272] T 4d2bc34dd22b4481801f97b7b84229e2 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.890641   431 tablet_replica.cc:333] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.891359   431 raft_consensus.cc:2243] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.891887   431 raft_consensus.cc:2272] T 4d56a87183594c8caf86cda36c1e4ba3 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.893932   431 tablet_replica.cc:333] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.894548   431 raft_consensus.cc:2243] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.895012   431 raft_consensus.cc:2272] T a7cbd11606124c0a82fcdae85eebf9a6 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.896940   431 tablet_replica.cc:333] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.897486   431 raft_consensus.cc:2243] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.897890   431 raft_consensus.cc:2272] T a322c0390a314662a84f1c53ab04b00d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.900287   431 tablet_replica.cc:333] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.900851   431 raft_consensus.cc:2243] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.901332   431 raft_consensus.cc:2272] T c2ad604df88241eaadb947764d02e45d P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.903174   431 tablet_replica.cc:333] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.903688   431 raft_consensus.cc:2243] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.904094   431 raft_consensus.cc:2272] T 884c9111d1f249b686aa232a2885d25f P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.906638   431 tablet_replica.cc:333] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.907232   431 raft_consensus.cc:2243] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.908115   431 raft_consensus.cc:2272] T 90eb9c71a8f442e2b26a3b1394459370 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.910414   431 tablet_replica.cc:333] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.911067   431 raft_consensus.cc:2243] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:50:12.911991   431 raft_consensus.cc:2272] T 6beecff85e664cc7b70aea711d0ebe55 P 9d954c17b0ce4789aabfa880b6b0ddcf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.914580   431 tablet_replica.cc:333] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf: stopping tablet replica
I20260709 14:50:12.915172   431 raft_consensus.cc:2243] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:12.915637   431 raft_consensus.cc:2272] T 400cb3b23ea14886a356218695773ffa P 9d954c17b0ce4789aabfa880b6b0ddcf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:12.945822   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:50:12.967121   431 master.cc:562] Master@127.0.107.254:39325 shutting down...
I20260709 14:50:13.206300   518 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 14:50:13.211359   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:13.212083   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:13.212471   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 74d3e791400b4df286511a6339707352: stopping tablet replica
I20260709 14:50:13.238781   431 master.cc:584] Master@127.0.107.254:39325 shutdown complete.
2ebfda1b7363403da68e5fa26a266f15  4
9d954c17b0ce4789aabfa880b6b0ddcf  1
f75c182005714280af142f2b0950c9f0  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14337 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 14:50:13.290409   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:41327
I20260709 14:50:13.291903   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:13.300022   953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.299795   950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.299898   951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:13.301229   431 server_base.cc:1061] running on GCE node
I20260709 14:50:13.302300   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:13.302559   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:13.302742   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608613302722 us; error 0 us; skew 500 ppm
I20260709 14:50:13.303424   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:13.307341   431 webserver.cc:533] Webserver started at http://127.0.107.254:37907/ using document root <none> and password file <none>
I20260709 14:50:13.308213   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:13.308471   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:13.308796   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:13.310086   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "1ddb3440e4e647da8a0d1a6656140959"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.316149   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.008s	sys 0.000s
I20260709 14:50:13.320587   958 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.321724   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 14:50:13.322103   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "1ddb3440e4e647da8a0d1a6656140959"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.322602   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:13.347326   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:13.348954   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:13.395344   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:41327
I20260709 14:50:13.395397  1009 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:41327 every 8 connection(s)
I20260709 14:50:13.401926  1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:13.415052  1010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959: Bootstrap starting.
I20260709 14:50:13.420764  1010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:13.426204  1010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959: No bootstrap required, opened a new log
I20260709 14:50:13.428918  1010 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER }
I20260709 14:50:13.429416  1010 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:13.429605  1010 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddb3440e4e647da8a0d1a6656140959, State: Initialized, Role: FOLLOWER
I20260709 14:50:13.430178  1010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER }
I20260709 14:50:13.430686  1010 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:50:13.431015  1010 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:50:13.431260  1010 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:13.496450  1010 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER }
I20260709 14:50:13.497164  1010 leader_election.cc:304] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddb3440e4e647da8a0d1a6656140959; no voters: 
I20260709 14:50:13.498616  1010 leader_election.cc:290] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:50:13.499703  1013 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:13.501559  1013 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 1 LEADER]: Becoming Leader. State: Replica: 1ddb3440e4e647da8a0d1a6656140959, State: Running, Role: LEADER
I20260709 14:50:13.502287  1010 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:50:13.502519  1013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER }
I20260709 14:50:13.508071  1014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ddb3440e4e647da8a0d1a6656140959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER } }
I20260709 14:50:13.509089  1014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:13.508966  1015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddb3440e4e647da8a0d1a6656140959. Latest consensus state: current_term: 1 leader_uuid: "1ddb3440e4e647da8a0d1a6656140959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddb3440e4e647da8a0d1a6656140959" member_type: VOTER } }
I20260709 14:50:13.509970  1015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:13.514968  1021 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:50:13.524055  1021 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:50:13.525147   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:50:13.534466  1021 catalog_manager.cc:1383] Generated new cluster ID: 07a4d64ba9184bb2b2068e2536e0cbae
I20260709 14:50:13.534899  1021 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:50:13.561611  1021 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:50:13.563095  1021 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:50:13.578939  1021 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959: Generated new TSK 0
I20260709 14:50:13.579679  1021 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:50:13.592708   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:13.600158  1031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.601190  1032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.603930  1034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:13.604045   431 server_base.cc:1061] running on GCE node
I20260709 14:50:13.605189   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:13.605460   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:13.605654   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608613605634 us; error 0 us; skew 500 ppm
I20260709 14:50:13.606249   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:13.609213   431 webserver.cc:533] Webserver started at http://127.0.107.193:44851/ using document root <none> and password file <none>
I20260709 14:50:13.609839   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:13.610080   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:13.610378   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:13.611642   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "9fef998d350141c895e2167984302880"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.616971   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 14:50:13.621006  1039 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.622009   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:50:13.622395   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "9fef998d350141c895e2167984302880"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.622752   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:13.650399   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:13.652001   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:13.653752   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:13.656454   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:13.656737   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.657150   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:13.657410   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.707599   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44667
I20260709 14:50:13.707757  1101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44667 every 8 connection(s)
I20260709 14:50:13.713424   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:13.721887  1106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.725546  1107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.727041  1109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:13.729220   431 server_base.cc:1061] running on GCE node
I20260709 14:50:13.730350   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:13.730580   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:13.730716   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608613730703 us; error 0 us; skew 500 ppm
I20260709 14:50:13.731328   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:13.735500   431 webserver.cc:533] Webserver started at http://127.0.107.194:42055/ using document root <none> and password file <none>
I20260709 14:50:13.736320   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:13.736663   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:13.737000   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:13.737248  1102 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41327
I20260709 14:50:13.737649  1102 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:13.738454   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "25e30a3a1cc646798a918dc1d90bb813"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.738513  1102 heartbeater.cc:507] Master 127.0.107.254:41327 requested a full tablet report, sending...
I20260709 14:50:13.741405   975 ts_manager.cc:194] Registered new tserver with Master: 9fef998d350141c895e2167984302880 (127.0.107.193:44667)
I20260709 14:50:13.743919   975 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36112
I20260709 14:50:13.744591   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:50:13.748816  1114 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.750029   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:50:13.750416   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "25e30a3a1cc646798a918dc1d90bb813"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.750780   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:13.772652   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:13.774119   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:13.776412   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:13.779122   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:13.779397   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.779660   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:13.779853   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.834038   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40325
I20260709 14:50:13.834184  1176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40325 every 8 connection(s)
I20260709 14:50:13.839860   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:13.849177  1180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.849671  1181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:13.854068  1183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:13.855279  1177 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41327
I20260709 14:50:13.855826  1177 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:13.856730  1177 heartbeater.cc:507] Master 127.0.107.254:41327 requested a full tablet report, sending...
I20260709 14:50:13.856961   431 server_base.cc:1061] running on GCE node
I20260709 14:50:13.858135   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:13.858364   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:13.858580   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608613858558 us; error 0 us; skew 500 ppm
I20260709 14:50:13.859359   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:13.859689   975 ts_manager.cc:194] Registered new tserver with Master: 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:13.861847   975 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36118
I20260709 14:50:13.862732   431 webserver.cc:533] Webserver started at http://127.0.107.195:43583/ using document root <none> and password file <none>
I20260709 14:50:13.863590   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:13.863850   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:13.864197   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:13.865753   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.871485   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 14:50:13.876277  1188 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.877401   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.002s
I20260709 14:50:13.877740   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
format_stamp: "Formatted at 2026-07-09 14:50:13 on dist-test-slave-wg8n"
I20260709 14:50:13.878027   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:13.899612   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:13.901059   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:13.902771   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:13.905490   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:13.905771   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.906046   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:13.906242   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:13.959968   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:42491
I20260709 14:50:13.960134  1250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:42491 every 8 connection(s)
I20260709 14:50:13.977447  1251 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41327
I20260709 14:50:13.977950  1251 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:13.978775  1251 heartbeater.cc:507] Master 127.0.107.254:41327 requested a full tablet report, sending...
I20260709 14:50:13.981389   975 ts_manager.cc:194] Registered new tserver with Master: 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:13.981585   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017715271s
I20260709 14:50:13.983198   975 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36126
I20260709 14:50:14.013592   975 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36128:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:50:14.017234   975 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:50:14.115054  1216 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3254117a704799bdbf446c04c070f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:50:14.116125  1142 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3254117a704799bdbf446c04c070f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:50:14.116890  1216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3254117a704799bdbf446c04c070f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.116467  1141 tablet_service.cc:1511] Processing CreateTablet for tablet 42adb749413c4474a1ce87707146f923 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:50:14.117295  1140 tablet_service.cc:1511] Processing CreateTablet for tablet c61ed07ed88a45efad0fc152edd82509 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:50:14.117848  1142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3254117a704799bdbf446c04c070f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.118669  1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61ed07ed88a45efad0fc152edd82509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.121640  1215 tablet_service.cc:1511] Processing CreateTablet for tablet 42adb749413c4474a1ce87707146f923 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:50:14.121930  1139 tablet_service.cc:1511] Processing CreateTablet for tablet 642bb41f622740c8a71312a693908c7f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:50:14.123430  1215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42adb749413c4474a1ce87707146f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.128310  1214 tablet_service.cc:1511] Processing CreateTablet for tablet c61ed07ed88a45efad0fc152edd82509 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:50:14.130119  1214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61ed07ed88a45efad0fc152edd82509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.136615  1138 tablet_service.cc:1511] Processing CreateTablet for tablet 5c01fa23f0d64a108fe80f75c150093f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:50:14.131852  1213 tablet_service.cc:1511] Processing CreateTablet for tablet 642bb41f622740c8a71312a693908c7f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:50:14.143049  1213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642bb41f622740c8a71312a693908c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.141856  1212 tablet_service.cc:1511] Processing CreateTablet for tablet 5c01fa23f0d64a108fe80f75c150093f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:50:14.146956  1137 tablet_service.cc:1511] Processing CreateTablet for tablet 3a15135475a0481bbe582a59a0cdc970 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:50:14.148365  1212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c01fa23f0d64a108fe80f75c150093f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.151357  1211 tablet_service.cc:1511] Processing CreateTablet for tablet 3a15135475a0481bbe582a59a0cdc970 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:50:14.153827  1211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15135475a0481bbe582a59a0cdc970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.122265  1141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42adb749413c4474a1ce87707146f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.159822  1136 tablet_service.cc:1511] Processing CreateTablet for tablet e13914cc890f49a9aef675fa26059975 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:50:14.161571  1136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13914cc890f49a9aef675fa26059975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.163311  1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642bb41f622740c8a71312a693908c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.164147  1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c01fa23f0d64a108fe80f75c150093f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.163555  1210 tablet_service.cc:1511] Processing CreateTablet for tablet e13914cc890f49a9aef675fa26059975 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:50:14.164975  1137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15135475a0481bbe582a59a0cdc970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.165817  1210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13914cc890f49a9aef675fa26059975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.170562  1135 tablet_service.cc:1511] Processing CreateTablet for tablet e1841dab6ad04cfc82540d9e9bac7e77 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:50:14.172772  1135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1841dab6ad04cfc82540d9e9bac7e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.184504  1209 tablet_service.cc:1511] Processing CreateTablet for tablet e1841dab6ad04cfc82540d9e9bac7e77 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:50:14.186546  1209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1841dab6ad04cfc82540d9e9bac7e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.194199  1134 tablet_service.cc:1511] Processing CreateTablet for tablet c641948ed29645df864b8519726e3a9b (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:50:14.195585  1067 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3254117a704799bdbf446c04c070f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:50:14.197100  1067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3254117a704799bdbf446c04c070f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.197249  1063 tablet_service.cc:1511] Processing CreateTablet for tablet 5c01fa23f0d64a108fe80f75c150093f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:50:14.199009  1064 tablet_service.cc:1511] Processing CreateTablet for tablet 642bb41f622740c8a71312a693908c7f (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:50:14.199476  1065 tablet_service.cc:1511] Processing CreateTablet for tablet c61ed07ed88a45efad0fc152edd82509 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:50:14.200922  1134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c641948ed29645df864b8519726e3a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.200829  1062 tablet_service.cc:1511] Processing CreateTablet for tablet 3a15135475a0481bbe582a59a0cdc970 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:50:14.201550  1066 tablet_service.cc:1511] Processing CreateTablet for tablet 42adb749413c4474a1ce87707146f923 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:50:14.203176  1061 tablet_service.cc:1511] Processing CreateTablet for tablet e13914cc890f49a9aef675fa26059975 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:50:14.205416  1060 tablet_service.cc:1511] Processing CreateTablet for tablet e1841dab6ad04cfc82540d9e9bac7e77 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:50:14.208050  1059 tablet_service.cc:1511] Processing CreateTablet for tablet c641948ed29645df864b8519726e3a9b (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:50:14.198750  1063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c01fa23f0d64a108fe80f75c150093f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.220242  1065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61ed07ed88a45efad0fc152edd82509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.223403  1066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42adb749413c4474a1ce87707146f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.231413  1061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13914cc890f49a9aef675fa26059975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.234797  1062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a15135475a0481bbe582a59a0cdc970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.242327  1060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1841dab6ad04cfc82540d9e9bac7e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.247318  1059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c641948ed29645df864b8519726e3a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.250541  1064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642bb41f622740c8a71312a693908c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.308080  1208 tablet_service.cc:1511] Processing CreateTablet for tablet c641948ed29645df864b8519726e3a9b (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:50:14.310226  1208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c641948ed29645df864b8519726e3a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.323139  1058 tablet_service.cc:1511] Processing CreateTablet for tablet cd4d3e2cfebd44a5b371fb14215487f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:50:14.325079  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4d3e2cfebd44a5b371fb14215487f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.331647  1207 tablet_service.cc:1511] Processing CreateTablet for tablet cd4d3e2cfebd44a5b371fb14215487f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:50:14.333688  1207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4d3e2cfebd44a5b371fb14215487f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.348120  1138 tablet_service.cc:1511] Processing CreateTablet for tablet cd4d3e2cfebd44a5b371fb14215487f7 (DEFAULT_TABLE table=test-workload [id=2c198b991211488e9ccb233ca4bd0bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:50:14.362762  1266 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.382663  1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4d3e2cfebd44a5b371fb14215487f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:14.402920  1267 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.412067  1268 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.421501  1267 tablet_bootstrap.cc:654] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.433744  1268 tablet_bootstrap.cc:654] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.446600  1266 tablet_bootstrap.cc:654] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.450487  1267 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.450593  1268 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.451411  1268 ts_tablet_manager.cc:1403] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.010s
I20260709 14:50:14.451473  1267 ts_tablet_manager.cc:1403] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.020s
I20260709 14:50:14.455042  1268 raft_consensus.cc:359] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.455858  1268 raft_consensus.cc:385] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.456228  1268 raft_consensus.cc:740] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.455935  1267 raft_consensus.cc:359] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.456959  1267 raft_consensus.cc:385] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.457358  1267 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.457119  1268 consensus_queue.cc:260] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.458283  1267 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.466123  1251 heartbeater.cc:499] Master 127.0.107.254:41327 was elected leader, sending a full tablet report...
I20260709 14:50:14.466907  1267 ts_tablet_manager.cc:1434] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.006s
I20260709 14:50:14.467446  1102 heartbeater.cc:499] Master 127.0.107.254:41327 was elected leader, sending a full tablet report...
I20260709 14:50:14.467988  1268 ts_tablet_manager.cc:1434] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.010s
I20260709 14:50:14.468475  1267 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.469033  1268 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.476204  1268 tablet_bootstrap.cc:654] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.477850  1267 tablet_bootstrap.cc:654] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.486426  1272 raft_consensus.cc:493] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.487233  1272 raft_consensus.cc:515] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.489285  1268 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.489878  1268 ts_tablet_manager.cc:1403] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 14:50:14.489285  1266 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.493554  1268 raft_consensus.cc:359] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.494567  1266 ts_tablet_manager.cc:1403] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.132s	user 0.015s	sys 0.010s
I20260709 14:50:14.494699  1268 raft_consensus.cc:385] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.495452  1268 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.496589  1268 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.499102  1268 ts_tablet_manager.cc:1434] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 14:50:14.500669  1268 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.505409  1273 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.507843  1272 leader_election.cc:290] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.506306  1273 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
W20260709 14:50:14.520442  1023 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:14.521291  1268 tablet_bootstrap.cc:654] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.545781  1273 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:14.550484  1266 raft_consensus.cc:359] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:14.551476  1266 raft_consensus.cc:385] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.551801  1266 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.553519  1267 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.554082  1267 ts_tablet_manager.cc:1403] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.051s
I20260709 14:50:14.558329  1267 raft_consensus.cc:359] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.559726  1267 raft_consensus.cc:385] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.559907  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:14.560292  1267 raft_consensus.cc:740] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.559795  1266 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:14.562268  1267 consensus_queue.cc:260] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
W20260709 14:50:14.562815  1041 leader_election.cc:343] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.565735  1267 ts_tablet_manager.cc:1434] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 14:50:14.567453  1267 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.573311  1268 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.574095  1268 ts_tablet_manager.cc:1403] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.019s
I20260709 14:50:14.576251  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:14.578558  1267 tablet_bootstrap.cc:654] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:14.586489  1190 leader_election.cc:343] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.593050  1275 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.593853  1275 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.599805  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
W20260709 14:50:14.602473  1041 leader_election.cc:343] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.602207  1268 raft_consensus.cc:359] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:14.603411  1268 raft_consensus.cc:385] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.603855  1268 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.604954  1268 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:14.606617  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.607344  1226 raft_consensus.cc:2468] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
I20260709 14:50:14.609853  1268 ts_tablet_manager.cc:1434] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.003s
I20260709 14:50:14.611021  1268 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.611531  1040 leader_election.cc:304] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:14.613256  1272 raft_consensus.cc:2804] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:14.613659  1272 raft_consensus.cc:493] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:14.614007  1272 raft_consensus.cc:3060] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.618010  1268 tablet_bootstrap.cc:654] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.623126  1272 raft_consensus.cc:515] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.626145  1275 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.626763  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:14.627878  1077 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:14.630180  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.630892  1226 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
I20260709 14:50:14.631498  1268 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.631955  1266 ts_tablet_manager.cc:1434] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.137s	user 0.012s	sys 0.002s
I20260709 14:50:14.632059  1268 ts_tablet_manager.cc:1403] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 14:50:14.632083  1189 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:14.633116  1177 heartbeater.cc:499] Master 127.0.107.254:41327 was elected leader, sending a full tablet report...
I20260709 14:50:14.633369  1274 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:14.633855  1274 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:14.634250  1274 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.636075  1268 raft_consensus.cc:359] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.636927  1268 raft_consensus.cc:385] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.637269  1268 raft_consensus.cc:740] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.638144  1268 consensus_queue.cc:260] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.639083  1040 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:14.639775  1266 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.640750  1268 ts_tablet_manager.cc:1434] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 14:50:14.642633  1268 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.645794  1287 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:14.647373  1287 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:14.648401  1287 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.653324  1274 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.654968  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:14.655843  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:14.656522  1226 raft_consensus.cc:3060] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:14.656625  1041 leader_election.cc:343] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.658885  1272 leader_election.cc:290] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.651300  1266 tablet_bootstrap.cc:654] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.661623  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:14.663031  1267 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.663271  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:14.663794  1267 ts_tablet_manager.cc:1403] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.096s	user 0.031s	sys 0.038s
W20260709 14:50:14.665151  1190 leader_election.cc:343] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.650729  1268 tablet_bootstrap.cc:654] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.667093  1226 raft_consensus.cc:2468] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
I20260709 14:50:14.667773  1267 raft_consensus.cc:359] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.668725  1267 raft_consensus.cc:385] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.668820  1040 leader_election.cc:304] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:14.669257  1274 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:14.670462  1272 raft_consensus.cc:2804] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:14.669198  1267 raft_consensus.cc:740] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.673024  1267 consensus_queue.cc:260] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.681738  1267 ts_tablet_manager.cc:1434] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 14:50:14.671691  1272 raft_consensus.cc:697] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:14.683002  1272 consensus_queue.cc:237] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.687904  1268 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.688050  1267 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.688628  1268 ts_tablet_manager.cc:1403] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.010s
I20260709 14:50:14.691881  1268 raft_consensus.cc:359] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.692555  1268 raft_consensus.cc:385] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.692852  1268 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.693585  1268 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.696050  1268 ts_tablet_manager.cc:1434] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 14:50:14.696103  1267 tablet_bootstrap.cc:654] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.697173  1268 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.697758   973 catalog_manager.cc:5697] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:14.678664  1287 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.699739  1266 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.700325  1266 ts_tablet_manager.cc:1403] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.008s
I20260709 14:50:14.700486  1077 raft_consensus.cc:2393] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in current term 1: Already voted for candidate 9fef998d350141c895e2167984302880 in this term.
I20260709 14:50:14.704803  1266 raft_consensus.cc:359] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.705968  1266 raft_consensus.cc:385] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.706038  1189 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880
I20260709 14:50:14.706532  1266 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.708029  1274 raft_consensus.cc:2749] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:14.708005  1266 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.712059  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:14.712598  1266 ts_tablet_manager.cc:1434] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 14:50:14.713794  1266 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
W20260709 14:50:14.718526  1041 leader_election.cc:343] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.722216  1268 tablet_bootstrap.cc:654] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.726915  1287 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.728396  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:14.729417  1226 raft_consensus.cc:2393] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9fef998d350141c895e2167984302880 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:14.731168  1040 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:14.732143  1287 raft_consensus.cc:2749] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:14.732156  1266 tablet_bootstrap.cc:654] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.736261  1267 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.736848  1267 ts_tablet_manager.cc:1403] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.032s
I20260709 14:50:14.740167  1268 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.740650  1268 ts_tablet_manager.cc:1403] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.007s
I20260709 14:50:14.740501  1267 raft_consensus.cc:359] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.741315  1267 raft_consensus.cc:385] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.741525  1266 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.741662  1267 raft_consensus.cc:740] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.742029  1266 ts_tablet_manager.cc:1403] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 14:50:14.742517  1267 consensus_queue.cc:260] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.743364  1268 raft_consensus.cc:359] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.744051  1268 raft_consensus.cc:385] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.744345  1268 raft_consensus.cc:740] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.744951  1267 ts_tablet_manager.cc:1434] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:50:14.745129  1266 raft_consensus.cc:359] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.745810  1266 raft_consensus.cc:385] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.745939  1267 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.745555  1268 consensus_queue.cc:260] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.746173  1266 raft_consensus.cc:740] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.747135  1266 consensus_queue.cc:260] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.748263  1268 ts_tablet_manager.cc:1434] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:50:14.749295  1268 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.752269  1266 ts_tablet_manager.cc:1434] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.007s
I20260709 14:50:14.753533  1266 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.753722  1267 tablet_bootstrap.cc:654] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.756438  1268 tablet_bootstrap.cc:654] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.766633  1267 tablet_bootstrap.cc:492] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.766796  1266 tablet_bootstrap.cc:654] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.767277  1267 ts_tablet_manager.cc:1403] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 14:50:14.773598  1267 raft_consensus.cc:359] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.774655  1268 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.774720  1267 raft_consensus.cc:385] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.775422  1267 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.775609  1268 ts_tablet_manager.cc:1403] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.004s
I20260709 14:50:14.776806  1267 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.780364  1268 raft_consensus.cc:359] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.781539  1268 raft_consensus.cc:385] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.781958  1268 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.785530  1267 ts_tablet_manager.cc:1434] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 14:50:14.786590  1268 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.788698  1267 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.789806  1266 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.790378  1266 ts_tablet_manager.cc:1403] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.012s
I20260709 14:50:14.791512  1268 ts_tablet_manager.cc:1434] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 14:50:14.792683  1268 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.796746  1267 tablet_bootstrap.cc:654] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.800973  1268 tablet_bootstrap.cc:654] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.803108  1266 raft_consensus.cc:359] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.804462  1266 raft_consensus.cc:385] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.804909  1266 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.806119  1266 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.809432  1266 ts_tablet_manager.cc:1434] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 14:50:14.810319  1267 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.810688  1266 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.810993  1267 ts_tablet_manager.cc:1403] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.011s
I20260709 14:50:14.814532  1267 raft_consensus.cc:359] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.815546  1267 raft_consensus.cc:385] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.815891  1267 raft_consensus.cc:740] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.817118  1267 consensus_queue.cc:260] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.818861  1266 tablet_bootstrap.cc:654] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.820856  1268 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.821419  1268 ts_tablet_manager.cc:1403] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.011s
I20260709 14:50:14.823879  1267 ts_tablet_manager.cc:1434] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 14:50:14.824700  1268 raft_consensus.cc:359] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.825443  1267 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:14.825735  1268 raft_consensus.cc:385] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.826051  1268 raft_consensus.cc:740] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.826812  1268 consensus_queue.cc:260] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.827488  1287 raft_consensus.cc:493] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.828002  1287 raft_consensus.cc:515] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.829198  1268 ts_tablet_manager.cc:1434] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 14:50:14.830511  1268 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.830408  1287 leader_election.cc:290] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.832173  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.835438  1266 tablet_bootstrap.cc:492] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.836053  1266 ts_tablet_manager.cc:1403] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.011s
W20260709 14:50:14.836089  1040 leader_election.cc:343] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.834782  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:14.837095  1152 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
I20260709 14:50:14.838197  1267 tablet_bootstrap.cc:654] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.838380  1268 tablet_bootstrap.cc:654] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.838753  1041 leader_election.cc:304] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:14.839766  1266 raft_consensus.cc:359] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.840667  1266 raft_consensus.cc:385] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.841004  1266 raft_consensus.cc:740] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.842504  1287 raft_consensus.cc:2804] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:14.842016  1266 consensus_queue.cc:260] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.843140  1287 raft_consensus.cc:493] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:14.843743  1287 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.847443  1284 raft_consensus.cc:493] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.848315  1284 raft_consensus.cc:515] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.849241  1266 ts_tablet_manager.cc:1434] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 14:50:14.850903  1266 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.851727  1268 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.852301  1268 ts_tablet_manager.cc:1403] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.006s
I20260709 14:50:14.860134  1268 raft_consensus.cc:359] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.864941  1268 raft_consensus.cc:385] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.866037  1268 raft_consensus.cc:740] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.867479  1268 consensus_queue.cc:260] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.860798  1266 tablet_bootstrap.cc:654] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.869843  1284 leader_election.cc:290] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.870692  1284 raft_consensus.cc:493] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.873790  1268 ts_tablet_manager.cc:1434] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 14:50:14.876711  1305 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.877506  1305 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.880021  1268 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: Bootstrap starting.
I20260709 14:50:14.884179  1305 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.892124  1268 tablet_bootstrap.cc:654] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:14.871353  1284 raft_consensus.cc:515] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.907780  1268 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: No bootstrap required, opened a new log
I20260709 14:50:14.908542  1268 ts_tablet_manager.cc:1403] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.008s
I20260709 14:50:14.909143  1284 leader_election.cc:290] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.912532  1268 raft_consensus.cc:359] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.913635  1268 raft_consensus.cc:385] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.914021  1268 raft_consensus.cc:740] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.915391  1268 consensus_queue.cc:260] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.923089  1268 ts_tablet_manager.cc:1434] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 14:50:14.934033  1287 raft_consensus.cc:515] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.941278  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:14.942341  1152 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.955245  1152 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
I20260709 14:50:14.957180  1041 leader_election.cc:304] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:14.958756  1274 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.958920  1272 raft_consensus.cc:2804] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:14.959426  1274 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.962261  1287 leader_election.cc:290] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:14.959712  1272 raft_consensus.cc:697] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:14.963907  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:14.964188  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:14.964926  1077 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:14.964670  1272 consensus_queue.cc:237] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.966190  1189 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:14.968364  1274 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:14.968884  1273 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:14.969305  1273 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:14.969791  1273 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:14.974781  1040 leader_election.cc:343] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:14.976007  1151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:14.977344  1151 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:14.983603  1266 tablet_bootstrap.cc:492] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:14.984297  1266 ts_tablet_manager.cc:1403] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.134s	user 0.043s	sys 0.069s
I20260709 14:50:14.986565  1284 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:14.987233  1284 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:14.988242  1266 raft_consensus.cc:359] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.989012  1266 raft_consensus.cc:385] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.989354  1266 raft_consensus.cc:740] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.989514  1273 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.990341  1266 consensus_queue.cc:260] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.991209  1267 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:14.991787  1267 ts_tablet_manager.cc:1403] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.167s	user 0.057s	sys 0.079s
I20260709 14:50:14.993223  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:14.993923  1273 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:14.993923  1077 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:14.993273  1266 ts_tablet_manager.cc:1434] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:50:14.997581  1266 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:14.997236  1267 raft_consensus.cc:359] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:14.997990  1267 raft_consensus.cc:385] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:14.998036  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.998399  1267 raft_consensus.cc:740] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:14.999437  1225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.999279  1267 consensus_queue.cc:260] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.000311  1225 raft_consensus.cc:2393] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:15.001569  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.002477  1224 raft_consensus.cc:2393] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 9fef998d350141c895e2167984302880 in this term.
I20260709 14:50:14.998216  1272 raft_consensus.cc:493] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.003747  1272 raft_consensus.cc:515] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.005550  1267 ts_tablet_manager.cc:1434] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 14:50:15.007025  1267 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:15.008312  1266 tablet_bootstrap.cc:654] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:15.010072  1115 leader_election.cc:343] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.010206  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:14.995113  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.006618  1272 leader_election.cc:290] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.012980  1076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.012902  1074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642bb41f622740c8a71312a693908c7f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.013757  1076 raft_consensus.cc:2468] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 0.
I20260709 14:50:15.011780  1152 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.014932  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.015880  1267 tablet_bootstrap.cc:654] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:15.016723  1284 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491), 9fef998d350141c895e2167984302880 (127.0.107.193:44667)
I20260709 14:50:15.023298  1152 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:15.026010  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.026811  1226 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
I20260709 14:50:15.028362  1040 leader_election.cc:304] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.029284  1272 raft_consensus.cc:2804] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.029685  1272 raft_consensus.cc:493] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.030056  1272 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.030769  1266 tablet_bootstrap.cc:492] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:15.031543  1266 ts_tablet_manager.cc:1403] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.009s
I20260709 14:50:15.035996  1266 raft_consensus.cc:359] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.036690  1151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.037235  1266 raft_consensus.cc:385] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.037431  1151 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
I20260709 14:50:15.037774  1266 raft_consensus.cc:740] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.039175  1190 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:15.038357  1272 raft_consensus.cc:515] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.039011  1266 consensus_queue.cc:260] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.040246  1273 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.040802  1273 raft_consensus.cc:697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:15.041710  1273 consensus_queue.cc:237] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.043308  1266 ts_tablet_manager.cc:1434] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 14:50:15.043192  1274 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.044137  1274 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.045133  1266 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:15.043541  1287 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.047134  1287 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.048293  1076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.049023  1076 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:15.049525  1287 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.052122  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.052590  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.052841  1226 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 0.
W20260709 14:50:15.055755  1115 leader_election.cc:343] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.056962  1115 leader_election.cc:304] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880
I20260709 14:50:15.053474  1075 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 0.
I20260709 14:50:15.063231  1284 raft_consensus.cc:3060] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.064389  1272 leader_election.cc:290] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.065523  1274 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.078612  1266 tablet_bootstrap.cc:654] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:15.075703  1189 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.078408  1077 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:15.078769   974 catalog_manager.cc:5697] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:15.070775  1040 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.065346  1115 leader_election.cc:304] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:15.084648  1305 raft_consensus.cc:2804] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.085098  1305 raft_consensus.cc:493] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.085455  1305 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.088568  1272 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.089174  1272 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.089623  1272 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.090188  1284 raft_consensus.cc:2749] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.096859  1115 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880
I20260709 14:50:15.098145  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:15.098869  1226 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.099287  1274 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.099692  1274 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.100080  1274 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.107008  1284 raft_consensus.cc:2749] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.110628  1305 raft_consensus.cc:515] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.113023  1267 tablet_bootstrap.cc:492] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:15.113425  1305 leader_election.cc:290] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.113646  1267 ts_tablet_manager.cc:1403] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.107s	user 0.038s	sys 0.030s
I20260709 14:50:15.114363  1115 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:15.117066  1151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.118199  1305 raft_consensus.cc:2804] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.118737  1305 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.119164  1305 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.119309  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.120065  1152 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.121421  1267 raft_consensus.cc:359] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.122275  1267 raft_consensus.cc:385] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.122641  1267 raft_consensus.cc:740] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.122540  1226 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
W20260709 14:50:15.123337  1190 leader_election.cc:343] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.123842  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:15.124702  1077 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.122996  1272 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.125371  1287 consensus_queue.cc:1048] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.123514  1150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.127310  1274 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
W20260709 14:50:15.127982  1041 leader_election.cc:343] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.122458  1266 tablet_bootstrap.cc:492] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:15.129949  1150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.130208  1274 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.130399  1266 ts_tablet_manager.cc:1403] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.017s
I20260709 14:50:15.131862  1267 consensus_queue.cc:260] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.118314   973 catalog_manager.cc:5697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:15.134382  1267 ts_tablet_manager.cc:1434] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
W20260709 14:50:15.134797  1041 leader_election.cc:343] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.136564  1040 leader_election.cc:304] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.136979  1266 raft_consensus.cc:359] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.137590  1287 raft_consensus.cc:2804] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.137830  1266 raft_consensus.cc:385] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.138141  1287 raft_consensus.cc:697] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:15.139098  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:15.140039  1075 raft_consensus.cc:2393] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in current term 1: Already voted for candidate 9fef998d350141c895e2167984302880 in this term.
I20260709 14:50:15.140131  1077 raft_consensus.cc:2468] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 1.
I20260709 14:50:15.140147  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:15.141633  1266 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.142021  1224 raft_consensus.cc:2393] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9fef998d350141c895e2167984302880 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:15.139117  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:15.142442  1266 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.143266  1226 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.144225  1267 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:15.144981  1266 ts_tablet_manager.cc:1434] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 14:50:15.146026  1266 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:15.147300  1150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.139132  1287 consensus_queue.cc:237] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.131285  1272 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.155843  1040 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:15.157624  1275 raft_consensus.cc:2749] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.158197  1152 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
W20260709 14:50:15.159163  1190 leader_election.cc:343] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.157464  1226 raft_consensus.cc:2468] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 1.
I20260709 14:50:15.160689  1190 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880
I20260709 14:50:15.160043  1115 leader_election.cc:304] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.162755  1274 raft_consensus.cc:2749] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.164654  1267 tablet_bootstrap.cc:654] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:15.164268  1284 raft_consensus.cc:2804] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.166978  1305 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:15.169517  1272 consensus_queue.cc:1048] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.170212  1274 raft_consensus.cc:493] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.172353  1266 tablet_bootstrap.cc:654] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:15.174132  1274 raft_consensus.cc:515] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.185845   972 catalog_manager.cc:5697] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:15.196003  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
W20260709 14:50:15.198256  1115 leader_election.cc:343] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.198622  1305 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 election: Requested vote from peers 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491), 9fef998d350141c895e2167984302880 (127.0.107.193:44667)
I20260709 14:50:15.201748  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:15.202594  1077 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.212507  1077 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 1.
I20260709 14:50:15.213997  1115 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:15.215137  1305 raft_consensus.cc:2804] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.215873  1305 raft_consensus.cc:697] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:15.217128  1305 consensus_queue.cc:237] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:15.180841  1284 raft_consensus.cc:697] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:15.220711  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.221674  1075 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:15.223225  1189 leader_election.cc:304] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.224573  1313 raft_consensus.cc:2804] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.225019  1313 raft_consensus.cc:493] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.225401  1313 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.233723  1273 raft_consensus.cc:493] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.234470  1273 raft_consensus.cc:515] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.235100  1284 consensus_queue.cc:237] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.242965  1267 tablet_bootstrap.cc:492] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:15.243747  1267 ts_tablet_manager.cc:1403] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.100s	user 0.045s	sys 0.014s
I20260709 14:50:15.244978  1273 leader_election.cc:290] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.245927  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.246671  1077 raft_consensus.cc:2468] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:15.246488  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.248116  1189 leader_election.cc:304] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:15.248837  1313 raft_consensus.cc:515] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
W20260709 14:50:15.249091  1190 leader_election.cc:343] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.250329  1273 raft_consensus.cc:2804] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:15.250720  1273 raft_consensus.cc:493] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:15.250586  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.251246  1273 raft_consensus.cc:3060] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.251601  1149 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 0.
I20260709 14:50:15.254762  1313 leader_election.cc:290] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.255990  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.256219  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:15.256650  1149 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.256955  1077 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.265259  1149 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:15.266922  1266 tablet_bootstrap.cc:492] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:15.266719  1190 leader_election.cc:304] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:15.267470  1266 ts_tablet_manager.cc:1403] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.122s	user 0.025s	sys 0.015s
I20260709 14:50:15.267787  1313 raft_consensus.cc:2804] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.269717  1267 raft_consensus.cc:359] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:15.270515  1267 raft_consensus.cc:385] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.270859  1267 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.270434  1266 raft_consensus.cc:359] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.271346  1266 raft_consensus.cc:385] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.271637  1266 raft_consensus.cc:740] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.248147  1274 leader_election.cc:290] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.272569  1266 consensus_queue.cc:260] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.275576  1313 raft_consensus.cc:697] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:15.275978  1266 ts_tablet_manager.cc:1434] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:50:15.276671  1313 consensus_queue.cc:237] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.277541  1266 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: Bootstrap starting.
I20260709 14:50:15.271639  1267 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:15.281059  1267 ts_tablet_manager.cc:1434] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 14:50:15.282348  1267 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Bootstrap starting.
I20260709 14:50:15.284750  1273 raft_consensus.cc:515] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.291321  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:15.292109  1075 raft_consensus.cc:3060] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.293263  1077 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:15.298457  1267 tablet_bootstrap.cc:654] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:15.299101  1273 leader_election.cc:290] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.300616  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:15.301450  1266 tablet_bootstrap.cc:654] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:15.302186  1190 leader_election.cc:343] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:15.313915   974 catalog_manager.cc:5697] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:15.314954   972 catalog_manager.cc:5697] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:15.316694  1266 tablet_bootstrap.cc:492] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: No bootstrap required, opened a new log
I20260709 14:50:15.317358  1266 ts_tablet_manager.cc:1403] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.000s
I20260709 14:50:15.318321  1267 tablet_bootstrap.cc:492] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: No bootstrap required, opened a new log
I20260709 14:50:15.318694  1075 raft_consensus.cc:2468] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:15.318908  1267 ts_tablet_manager.cc:1403] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.000s
I20260709 14:50:15.320195  1189 leader_election.cc:304] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:15.320533  1266 raft_consensus.cc:359] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.321278  1266 raft_consensus.cc:385] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.321514  1313 raft_consensus.cc:2804] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:15.321575  1266 raft_consensus.cc:740] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.322140  1313 raft_consensus.cc:697] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:15.322363  1267 raft_consensus.cc:359] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.322597  1266 consensus_queue.cc:260] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.323289  1267 raft_consensus.cc:385] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:15.323120  1313 consensus_queue.cc:237] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.323737  1267 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Initialized, Role: FOLLOWER
I20260709 14:50:15.325169  1267 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.326015  1266 ts_tablet_manager.cc:1434] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:50:15.327872  1267 ts_tablet_manager.cc:1434] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:50:15.329667   972 catalog_manager.cc:5697] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:15.342152   974 catalog_manager.cc:5697] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:15.416468  1313 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.417088  1313 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:15.419060  1313 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:15.420089  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.420291  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:15.420854  1075 raft_consensus.cc:2393] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in current term 1: Already voted for candidate 25e30a3a1cc646798a918dc1d90bb813 in this term.
I20260709 14:50:15.421937  1189 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880
I20260709 14:50:15.422641  1313 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.426985  1284 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.427639  1284 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.429503  1313 raft_consensus.cc:2749] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.430034  1284 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.430900  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.431205  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.431632  1075 raft_consensus.cc:2393] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 9fef998d350141c895e2167984302880 in this term.
I20260709 14:50:15.431958  1224 raft_consensus.cc:2393] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:15.432889  1115 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880
I20260709 14:50:15.433566  1284 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.439276  1284 raft_consensus.cc:2749] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.485764  1284 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.486486  1284 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.488590  1284 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.489511  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.489881  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.490350  1075 raft_consensus.cc:2393] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 9fef998d350141c895e2167984302880 in this term.
I20260709 14:50:15.490691  1224 raft_consensus.cc:2393] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:15.491932  1115 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880
I20260709 14:50:15.492729  1284 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.498981  1284 raft_consensus.cc:2749] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.503937  1272 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.515359  1224 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.526243  1275 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:50:15.529866  1023 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:15.553032  1275 consensus_queue.cc:1048] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.565032  1287 consensus_queue.cc:1048] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:15.575450  1313 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.588670  1273 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.671578  1273 consensus_queue.cc:1048] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.684237  1327 consensus_queue.cc:1048] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:15.711285  1284 consensus_queue.cc:1048] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.724700  1318 consensus_queue.cc:1048] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:15.734503  1318 raft_consensus.cc:493] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:15.735075  1318 raft_consensus.cc:515] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:15.737124  1318 leader_election.cc:290] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:15.741539  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:15.739379  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4d3e2cfebd44a5b371fb14215487f7" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:15.742962  1075 raft_consensus.cc:2393] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e30a3a1cc646798a918dc1d90bb813 in current term 1: Already voted for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in this term.
I20260709 14:50:15.744391  1115 leader_election.cc:304] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880
I20260709 14:50:15.745329  1318 raft_consensus.cc:3060] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:15.751868  1318 raft_consensus.cc:2749] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:15.752885  1318 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.768642  1318 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:15.822223  1327 consensus_queue.cc:1048] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:15.835657  1313 consensus_queue.cc:1048] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:16.400269  1274 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:16.400825  1274 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.402956  1274 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:16.404076  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:16.403957  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880" is_pre_election: true
I20260709 14:50:16.404947  1075 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:16.404886  1149 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 1.
I20260709 14:50:16.406356  1189 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:16.407195  1274 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:50:16.407946  1274 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:16.408315  1274 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.413758  1274 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.415709  1274 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 2 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:16.417002  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:16.417153  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:16.417618  1075 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.417690  1149 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.424029  1149 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 2.
I20260709 14:50:16.424772  1075 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 2.
I20260709 14:50:16.425488  1190 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:16.426344  1274 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:16.426800  1274 raft_consensus.cc:697] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:16.427717  1274 consensus_queue.cc:237] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.436008   975 catalog_manager.cc:5697] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 0 to 2, leader changed from <none> to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
W20260709 14:50:16.536126  1023 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:16.665140  1272 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:16.665822  1272 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.667852  1272 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:16.669132  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813" is_pre_election: true
I20260709 14:50:16.669365  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" is_pre_election: true
I20260709 14:50:16.669854  1149 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
I20260709 14:50:16.670042  1224 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 1.
I20260709 14:50:16.671233  1040 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:16.672163  1272 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:50:16.672675  1272 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:16.672969  1272 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.678754  1272 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.680876  1272 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:16.681721  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:16.682456  1149 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.682452  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:16.683276  1224 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:16.689870  1149 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:16.689870  1224 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:16.691249  1040 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:16.692075  1272 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:16.693005  1272 raft_consensus.cc:697] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:16.693868  1272 consensus_queue.cc:237] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.702207   975 catalog_manager.cc:5697] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:16.888810  1149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:38476
I20260709 14:50:16.889544  1149 raft_consensus.cc:606] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:16.911706  1224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:37542
I20260709 14:50:16.912395  1224 raft_consensus.cc:606] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:16.922199  1149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:38476
I20260709 14:50:16.922989  1149 raft_consensus.cc:606] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:16.937897  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:52882
I20260709 14:50:16.938549  1075 raft_consensus.cc:606] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:16.944454  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e13914cc890f49a9aef675fa26059975"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:52882
I20260709 14:50:16.945111  1075 raft_consensus.cc:606] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:16.950959  1224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:37542
I20260709 14:50:16.951566  1224 raft_consensus.cc:606] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:16.955847  1075 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:16.957391  1224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c641948ed29645df864b8519726e3a9b"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:37542
I20260709 14:50:16.957355  1313 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:16.958179  1224 raft_consensus.cc:606] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:16.964736  1149 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:16.966339  1274 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:16.985909  1313 raft_consensus.cc:993] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:16.986474  1274 raft_consensus.cc:1081] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:16.989408  1149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38472
I20260709 14:50:16.990187  1149 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:16.991659  1149 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:16.997413  1149 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:16.999648  1149 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 3 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.000823  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.000633  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:17.001570  1224 raft_consensus.cc:3055] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:17.001616  1075 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:17.002024  1224 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:17.002966  1224 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.004570  1224 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:17.010902  1075 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 3.
I20260709 14:50:17.012487  1115 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:17.013342  1318 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:17.012809  1224 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 3.
I20260709 14:50:17.015132  1318 raft_consensus.cc:697] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.016264  1318 consensus_queue.cc:237] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.025377   972 catalog_manager.cc:5697] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 2 to 3, leader changed from 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.059506  1313 raft_consensus.cc:993] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.059981  1274 raft_consensus.cc:1081] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.061522  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52850
I20260709 14:50:17.062063  1075 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.062371  1075 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.068048  1075 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.070103  1075 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.071187  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:17.071432  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.071879  1149 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.071997  1224 raft_consensus.cc:3055] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:17.072368  1224 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:17.073076  1224 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.074088  1224 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.077694  1149 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.079061  1041 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:17.079970  1272 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:17.080066  1224 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.080534  1272 raft_consensus.cc:697] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:17.081542  1272 consensus_queue.cc:237] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.089573   972 catalog_manager.cc:5697] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 1 to 2, leader changed from 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.091552  1351 raft_consensus.cc:993] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.092142  1351 raft_consensus.cc:1081] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.094108  1149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38462
I20260709 14:50:17.094758  1149 raft_consensus.cc:493] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.095278  1149 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.102109  1149 raft_consensus.cc:515] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.104090  1149 leader_election.cc:290] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.105265  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:17.105803  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.105965  1075 raft_consensus.cc:3055] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:17.106428  1075 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:17.106421  1224 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.107436  1075 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.108539  1075 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.112918  1224 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:17.114313  1115 leader_election.cc:304] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:17.114691  1075 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:17.115365  1318 raft_consensus.cc:2804] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:17.115893  1318 raft_consensus.cc:697] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.116847  1318 consensus_queue.cc:237] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.126322   972 catalog_manager.cc:5697] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 1 to 2, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.219080  1363 raft_consensus.cc:993] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.219796  1318 raft_consensus.cc:1081] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.222239  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52866
I20260709 14:50:17.222999  1075 raft_consensus.cc:493] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.223408  1075 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.231037  1075 raft_consensus.cc:515] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.232980  1075 leader_election.cc:290] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.234145  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:17.234153  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c01fa23f0d64a108fe80f75c150093f" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.235069  1224 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.235069  1149 raft_consensus.cc:3055] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:17.235590  1149 raft_consensus.cc:740] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.236527  1149 consensus_queue.cc:260] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.237905  1149 raft_consensus.cc:3060] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.243573  1224 raft_consensus.cc:2468] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.244974  1040 leader_election.cc:304] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:17.245707  1149 raft_consensus.cc:2468] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.245880  1351 raft_consensus.cc:2804] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:17.245927  1152 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:17.246516  1351 raft_consensus.cc:697] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:17.247486  1351 consensus_queue.cc:237] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.248167  1272 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:17.258885  1272 raft_consensus.cc:993] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.258049   972 catalog_manager.cc:5697] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 1 to 2, leader changed from 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.265533  1351 raft_consensus.cc:1081] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.269963  1152 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e13914cc890f49a9aef675fa26059975"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38462
I20260709 14:50:17.270687  1152 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.270946  1224 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:17.271179  1152 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:17.272657  1272 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:17.279615  1152 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.282457  1152 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 3 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.284576  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.285497  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:17.286141  1075 raft_consensus.cc:3055] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:17.286442  1075 raft_consensus.cc:740] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:17.287653  1075 consensus_queue.cc:260] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.288769  1075 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:17.299185  1313 raft_consensus.cc:993] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.299960  1363 raft_consensus.cc:993] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.300038  1313 raft_consensus.cc:1081] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:17.299734  1075 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 3.
I20260709 14:50:17.302163  1370 raft_consensus.cc:1081] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:17.302173  1115 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:17.303930  1370 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:17.304749  1370 raft_consensus.cc:697] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.305292  1152 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c641948ed29645df864b8519726e3a9b"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38472
I20260709 14:50:17.306042  1152 raft_consensus.cc:493] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.305783  1370 consensus_queue.cc:237] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.306490  1152 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.305672  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52866
I20260709 14:50:17.307832  1075 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:17.308357  1075 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.315147  1152 raft_consensus.cc:515] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.315600   972 catalog_manager.cc:5697] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 2 to 3, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.317605  1152 leader_election.cc:290] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:17.319211  1077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:17.319474  1075 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:17.319640  1224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c641948ed29645df864b8519726e3a9b" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.320676  1224 raft_consensus.cc:3055] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:17.320990  1224 raft_consensus.cc:740] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:17.321869  1224 consensus_queue.cc:260] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.322295  1075 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Requested vote from peers 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:17.322773  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:17.320142  1077 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.323405  1226 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.324406  1224 raft_consensus.cc:3060] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.328069  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:17.328833  1152 raft_consensus.cc:3055] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:17.329191  1152 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.330353  1152 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:17.331789  1152 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:17.331845  1226 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.332927  1224 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:17.333425  1040 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:17.333784  1077 raft_consensus.cc:2468] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:17.334484  1115 leader_election.cc:304] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:17.334575  1272 raft_consensus.cc:2804] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:17.335554  1272 raft_consensus.cc:697] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:17.336158  1370 raft_consensus.cc:2804] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:17.336720  1370 raft_consensus.cc:697] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:17.336539  1272 consensus_queue.cc:237] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:17.337664  1370 consensus_queue.cc:237] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:17.340771  1152 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 2.
I20260709 14:50:17.348165   972 catalog_manager.cc:5697] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 1 to 2, leader changed from 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.351948   975 catalog_manager.cc:5697] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 1 to 2, leader changed from 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:17.460022  1226 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:17.461660  1370 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.470973  1077 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:17.472935  1363 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.641990  1226 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.643507  1272 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.653431  1152 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.655797  1351 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.703372  1077 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.704957  1363 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.715385  1226 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.717720  1318 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.764441  1152 raft_consensus.cc:1275] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.765913  1351 consensus_queue.cc:1048] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.780112  1226 raft_consensus.cc:1275] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.782863  1351 consensus_queue.cc:1048] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.821623  1152 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.823076  1351 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.832005  1226 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.834062  1272 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.890272  1226 raft_consensus.cc:1275] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.891855  1377 consensus_queue.cc:1048] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:17.899457  1077 raft_consensus.cc:1275] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:17.903740  1363 consensus_queue.cc:1048] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:17.922487  1077 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:17.922705  1226 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:17.924016  1377 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:17.936054  1226 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:17.937569  1377 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:50:19.501538  1098 debug-util.cc:398] Leaking SignalData structure 0x7b0800156fa0 after lost signal to thread 434
W20260709 14:50:19.502938  1098 debug-util.cc:398] Leaking SignalData structure 0x7b0800146680 after lost signal to thread 1009
W20260709 14:50:19.504187  1098 debug-util.cc:398] Leaking SignalData structure 0x7b0800094b60 after lost signal to thread 1101
W20260709 14:50:19.505270  1098 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3c00 after lost signal to thread 1176
W20260709 14:50:19.506357  1098 debug-util.cc:398] Leaking SignalData structure 0x7b08000e44e0 after lost signal to thread 1250
I20260709 14:50:20.103214   431 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 14:50:20.104538   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:20.132056  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:20.133080  1075 raft_consensus.cc:606] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Received request to transfer leadership to TS 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:20.136739  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:20.137339  1075 raft_consensus.cc:606] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Received request to transfer leadership to TS 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:20.139199   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:50:20.139834   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:50:20.196035  1392 raft_consensus.cc:993] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: : Instructing follower 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:20.196705  1351 raft_consensus.cc:1081] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Signalling peer 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:20.198606  1225 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:37512
I20260709 14:50:20.199252  1225 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:20.199623  1225 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.206956  1225 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:20.209888  1225 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 3 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:20.210955  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:20.211628  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:20.211941  1075 raft_consensus.cc:3055] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:20.212231  1149 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.212301  1075 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:20.213390  1075 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:20.214898  1075 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.221982  1149 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 3.
I20260709 14:50:20.224225  1075 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 3.
I20260709 14:50:20.224043  1190 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:20.225441  1358 raft_consensus.cc:2804] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:20.227057  1358 raft_consensus.cc:697] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:20.228343  1358 consensus_queue.cc:237] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:20.240583   972 catalog_manager.cc:5697] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 2 to 3, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:20.590538  1392 raft_consensus.cc:993] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: : Instructing follower 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:20.591140  1392 raft_consensus.cc:1081] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Signalling peer 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:20.593457  1225 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:37512
I20260709 14:50:20.594081  1225 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:20.594460  1225 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.600873  1225 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:20.603856  1225 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 3 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:20.604017  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:20.604667  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:20.604725  1075 raft_consensus.cc:3055] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:20.605366  1075 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:20.605373  1149 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.606401  1075 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:20.607537  1075 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:20.612954  1075 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 3.
I20260709 14:50:20.613868  1149 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 3.
I20260709 14:50:20.614326  1189 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:20.615598  1358 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:20.616058  1358 raft_consensus.cc:697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:20.617087  1358 consensus_queue.cc:237] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:20.626708   972 catalog_manager.cc:5697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 2 to 3, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:20.743791  1149 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:20.745709  1358 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:20.760584  1075 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:20.764482  1393 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.140782   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:21.152518  1225 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:37542
I20260709 14:50:21.153164  1225 raft_consensus.cc:606] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:21.162474  1225 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:37542
I20260709 14:50:21.163126  1225 raft_consensus.cc:606] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:21.175918  1149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:38476
I20260709 14:50:21.176505  1149 raft_consensus.cc:606] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:21.182781  1149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e13914cc890f49a9aef675fa26059975"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:38476
I20260709 14:50:21.183406  1149 raft_consensus.cc:606] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:21.188803  1149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
mode: GRACEFUL
new_leader_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:38476
I20260709 14:50:21.189545  1149 raft_consensus.cc:606] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Received request to transfer leadership to TS 9fef998d350141c895e2167984302880
I20260709 14:50:21.204598  1075 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:21.206713  1358 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:21.214788  1393 raft_consensus.cc:993] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.220484  1393 raft_consensus.cc:1081] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.225718  1149 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:21.227478  1358 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.228277  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52850
I20260709 14:50:21.228981  1075 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:21.229410  1075 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.239868  1075 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.241750  1075 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:21.242856  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:21.244159  1225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:21.244933  1225 raft_consensus.cc:3055] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:21.245288  1225 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:21.246647  1225 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.248070  1225 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.254467  1225 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.256143  1040 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:21.257052  1351 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:21.259524  1351 raft_consensus.cc:697] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:21.260433  1351 consensus_queue.cc:237] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.269871   975 catalog_manager.cc:5697] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 3 to 4, leader changed from 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:21.270910  1358 raft_consensus.cc:993] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.271492  1358 raft_consensus.cc:1081] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.273478  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52850
I20260709 14:50:21.274458  1075 raft_consensus.cc:493] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:21.275045  1075 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.281500  1075 raft_consensus.cc:515] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:21.284999  1225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:21.285458  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3254117a704799bdbf446c04c070f7" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:21.285727  1225 raft_consensus.cc:3055] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:21.286021  1149 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.286072  1225 raft_consensus.cc:740] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:21.287046  1225 consensus_queue.cc:260] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:21.288259  1225 raft_consensus.cc:3060] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.294428  1149 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.296389  1041 leader_election.cc:304] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:21.284428  1075 leader_election.cc:290] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Requested vote from peers 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:21.297503  1351 raft_consensus.cc:2804] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:21.298055  1351 raft_consensus.cc:697] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:21.296659  1225 raft_consensus.cc:2468] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.298966  1351 consensus_queue.cc:237] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } }
I20260709 14:50:21.308727   975 catalog_manager.cc:5697] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 3 to 4, leader changed from 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:21.397581  1412 raft_consensus.cc:993] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.398298  1389 raft_consensus.cc:1081] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.400477  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52866
I20260709 14:50:21.401103  1075 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:21.401501  1075 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.407765  1075 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.410248  1075 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:21.411159  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:21.411444  1225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:21.411911  1149 raft_consensus.cc:3055] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:21.412181  1225 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.412323  1149 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:21.413475  1149 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.415040  1149 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.419858  1225 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.421211  1040 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:21.422200  1408 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:21.422706  1408 raft_consensus.cc:697] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:21.423902  1149 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.423691  1408 consensus_queue.cc:237] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.433876   975 catalog_manager.cc:5697] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 3 to 4, leader changed from 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:21.450641  1389 raft_consensus.cc:993] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.451306  1412 raft_consensus.cc:1081] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.453280  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e13914cc890f49a9aef675fa26059975"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52866
I20260709 14:50:21.453933  1075 raft_consensus.cc:493] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:21.454254  1075 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.460562  1075 raft_consensus.cc:515] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.462707  1389 raft_consensus.cc:993] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: : Instructing follower 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.463371  1389 raft_consensus.cc:1081] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Signalling peer 9fef998d350141c895e2167984302880 to start an election
I20260709 14:50:21.462721  1075 leader_election.cc:290] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:21.465029  1225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:21.465907  1225 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.466609  1075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "9fef998d350141c895e2167984302880"
 from {username='slave'} at 127.0.0.1:52866
I20260709 14:50:21.467196  1075 raft_consensus.cc:493] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:21.467572  1075 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:21.465027  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13914cc890f49a9aef675fa26059975" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:21.469929  1149 raft_consensus.cc:3055] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:21.470270  1149 raft_consensus.cc:740] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:21.471199  1149 consensus_queue.cc:260] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.472561  1149 raft_consensus.cc:3060] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.475750  1225 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.475775  1075 raft_consensus.cc:515] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.477644  1040 leader_election.cc:304] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea883dd389c4cbbb2c70a9599cd5c1a, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:21.478981  1408 raft_consensus.cc:2804] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:21.479229  1075 leader_election.cc:290] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 3 election: Requested vote from peers 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:21.479122  1152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:21.479578  1408 raft_consensus.cc:697] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:21.479887  1152 raft_consensus.cc:3055] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:21.480268  1152 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:21.480877  1149 raft_consensus.cc:2468] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 4.
I20260709 14:50:21.481374  1152 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.482733  1152 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:21.480782  1408 consensus_queue.cc:237] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.484552  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "9fef998d350141c895e2167984302880" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:21.485277  1226 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:21.492789  1152 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 3.
I20260709 14:50:21.494656  1041 leader_election.cc:304] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 9fef998d350141c895e2167984302880; no voters: 
I20260709 14:50:21.495932  1408 raft_consensus.cc:2804] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:21.496738  1408 raft_consensus.cc:697] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 LEADER]: Becoming Leader. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:21.497743  1226 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fef998d350141c895e2167984302880 in term 3.
I20260709 14:50:21.497767  1408 consensus_queue.cc:237] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:21.500967   975 catalog_manager.cc:5697] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 3 to 4, leader changed from 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:21.516666   975 catalog_manager.cc:5697] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 reported cstate change: term changed from 2 to 3, leader changed from 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194) to 9fef998d350141c895e2167984302880 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9fef998d350141c895e2167984302880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:21.673508  1152 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:21.680131  1152 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:21.682046  1351 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:21.684046  1226 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:21.685765  1351 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:21.794709  1152 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:21.797044  1351 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:21.814930  1152 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:50:21.816780  1416 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.818387  1226 raft_consensus.cc:1275] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:21.820271  1416 consensus_queue.cc:1048] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:21.838493  1225 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:50:21.842978  1351 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.920738  1149 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:21.922173  1351 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:50:21.936200  1225 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:50:21.938133  1416 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.945508  1226 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:50:21.947507  1416 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:21.970006  1149 raft_consensus.cc:1275] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Refusing update from remote peer 9fef998d350141c895e2167984302880: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:50:21.971803  1408 consensus_queue.cc:1048] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:50:24.198467   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:24.206808  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:24.207453  1075 raft_consensus.cc:606] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:24.210032  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42adb749413c4474a1ce87707146f923"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:24.210646  1075 raft_consensus.cc:606] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Received request to transfer leadership to TS 25e30a3a1cc646798a918dc1d90bb813
I20260709 14:50:24.213241  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:24.213640  1075 raft_consensus.cc:606] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Received request to transfer leadership to TS 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:24.215986  1075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "9fef998d350141c895e2167984302880"
mode: GRACEFUL
new_leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:44378
I20260709 14:50:24.216470  1075 raft_consensus.cc:606] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Received request to transfer leadership to TS 3ea883dd389c4cbbb2c70a9599cd5c1a
I20260709 14:50:24.217561   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 14:50:24.218084   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:50:24.333927  1425 raft_consensus.cc:993] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:24.334583  1408 raft_consensus.cc:1081] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:24.336467  1149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42adb749413c4474a1ce87707146f923"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38462
I20260709 14:50:24.337081  1149 raft_consensus.cc:493] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:24.337533  1149 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:24.343052  1149 raft_consensus.cc:515] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.344791  1149 leader_election.cc:290] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:24.346015  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:24.346274  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42adb749413c4474a1ce87707146f923" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:24.346686  1075 raft_consensus.cc:3055] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:24.346958  1226 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:24.347209  1075 raft_consensus.cc:740] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:24.348253  1075 consensus_queue.cc:260] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.349427  1075 raft_consensus.cc:3060] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:24.354641  1226 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:24.356031  1075 raft_consensus.cc:2468] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 2.
I20260709 14:50:24.356212  1115 leader_election.cc:304] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:24.357095  1389 raft_consensus.cc:2804] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:24.358160  1389 raft_consensus.cc:697] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:24.359061  1389 consensus_queue.cc:237] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.366285   972 catalog_manager.cc:5697] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 1 to 2, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:24.422000  1425 raft_consensus.cc:993] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: : Instructing follower 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:24.422475  1408 raft_consensus.cc:1081] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 LEADER]: Signalling peer 25e30a3a1cc646798a918dc1d90bb813 to start an election
I20260709 14:50:24.423979  1149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970"
dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
 from {username='slave'} at 127.0.0.1:38462
I20260709 14:50:24.424513  1149 raft_consensus.cc:493] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:24.424882  1149 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:24.429999  1149 raft_consensus.cc:515] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.431653  1149 leader_election.cc:290] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 4 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195:42491)
I20260709 14:50:24.432770  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:24.433017  1226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a15135475a0481bbe582a59a0cdc970" candidate_uuid: "25e30a3a1cc646798a918dc1d90bb813" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
I20260709 14:50:24.433341  1075 raft_consensus.cc:3055] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:50:24.433547  1226 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:24.433667  1075 raft_consensus.cc:740] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:24.434481  1075 consensus_queue.cc:260] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.435920  1075 raft_consensus.cc:3060] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:50:24.439307  1226 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 4.
I20260709 14:50:24.440421  1115 leader_election.cc:304] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:24.441129  1389 raft_consensus.cc:2804] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:50:24.441124  1075 raft_consensus.cc:2468] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e30a3a1cc646798a918dc1d90bb813 in term 4.
I20260709 14:50:24.441911  1389 raft_consensus.cc:697] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 LEADER]: Becoming Leader. State: Replica: 25e30a3a1cc646798a918dc1d90bb813, State: Running, Role: LEADER
I20260709 14:50:24.442740  1389 consensus_queue.cc:237] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.451231  1408 raft_consensus.cc:993] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: : Instructing follower 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:24.451814  1408 raft_consensus.cc:1081] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Signalling peer 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:24.451814   972 catalog_manager.cc:5697] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 reported cstate change: term changed from 3 to 4, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "25e30a3a1cc646798a918dc1d90bb813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:24.453537  1226 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:37512
I20260709 14:50:24.454116  1226 raft_consensus.cc:493] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:24.454473  1226 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.459762  1226 raft_consensus.cc:515] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.461800  1226 leader_election.cc:290] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 5 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:24.462528  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:24.462755  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1841dab6ad04cfc82540d9e9bac7e77" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:24.463192  1075 raft_consensus.cc:3055] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:50:24.463349  1149 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.463500  1075 raft_consensus.cc:740] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:24.464054  1075 consensus_queue.cc:260] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.464876  1075 raft_consensus.cc:3060] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.469584  1149 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 5.
I20260709 14:50:24.470920  1190 leader_election.cc:304] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:24.471184  1075 raft_consensus.cc:2468] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 5.
I20260709 14:50:24.471773  1427 raft_consensus.cc:2804] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:50:24.473034  1427 raft_consensus.cc:697] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:24.473827  1427 consensus_queue.cc:237] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.482802   972 catalog_manager.cc:5697] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 4 to 5, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 5 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:24.693862  1408 raft_consensus.cc:993] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: : Instructing follower 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:24.694389  1435 raft_consensus.cc:1081] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Signalling peer 3ea883dd389c4cbbb2c70a9599cd5c1a to start an election
I20260709 14:50:24.696048  1226 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509"
dest_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a"
 from {username='slave'} at 127.0.0.1:37512
I20260709 14:50:24.696539  1226 raft_consensus.cc:493] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:24.696786  1226 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.701763  1226 raft_consensus.cc:515] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.703979  1226 leader_election.cc:290] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 5 election: Requested vote from peers 9fef998d350141c895e2167984302880 (127.0.107.193:44667), 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325)
I20260709 14:50:24.704968  1075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9fef998d350141c895e2167984302880"
I20260709 14:50:24.705196  1149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61ed07ed88a45efad0fc152edd82509" candidate_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "25e30a3a1cc646798a918dc1d90bb813"
I20260709 14:50:24.705654  1075 raft_consensus.cc:3055] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:50:24.705755  1149 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.705960  1075 raft_consensus.cc:740] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9fef998d350141c895e2167984302880, State: Running, Role: LEADER
I20260709 14:50:24.706775  1075 consensus_queue.cc:260] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.707823  1075 raft_consensus.cc:3060] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:50:24.710853  1149 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 5.
I20260709 14:50:24.712013  1190 leader_election.cc:304] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e30a3a1cc646798a918dc1d90bb813, 3ea883dd389c4cbbb2c70a9599cd5c1a; no voters: 
I20260709 14:50:24.712604  1075 raft_consensus.cc:2468] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea883dd389c4cbbb2c70a9599cd5c1a in term 5.
I20260709 14:50:24.712779  1427 raft_consensus.cc:2804] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:50:24.713325  1427 raft_consensus.cc:697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 LEADER]: Becoming Leader. State: Replica: 3ea883dd389c4cbbb2c70a9599cd5c1a, State: Running, Role: LEADER
I20260709 14:50:24.714222  1427 consensus_queue.cc:237] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } }
I20260709 14:50:24.721930   972 catalog_manager.cc:5697] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a reported cstate change: term changed from 4 to 5, leader changed from 9fef998d350141c895e2167984302880 (127.0.107.193) to 3ea883dd389c4cbbb2c70a9599cd5c1a (127.0.107.195). New cstate: current_term: 5 leader_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:24.774422  1075 raft_consensus.cc:1275] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:24.775933  1389 consensus_queue.cc:1048] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:24.788064  1226 raft_consensus.cc:1275] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:24.789481  1428 consensus_queue.cc:1048] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:24.899513  1226 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:24.901003  1428 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea883dd389c4cbbb2c70a9599cd5c1a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:24.901523  1075 raft_consensus.cc:1275] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Refusing update from remote peer 25e30a3a1cc646798a918dc1d90bb813: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:50:24.903321  1428 consensus_queue.cc:1048] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:50:25.025283  1149 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:50:25.026736  1427 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:25.029788  1075 raft_consensus.cc:1275] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:50:25.031337  1427 consensus_queue.cc:1048] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:50:25.123832  1075 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:25.125485  1427 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef998d350141c895e2167984302880" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:50:25.136258  1149 raft_consensus.cc:1275] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Refusing update from remote peer 3ea883dd389c4cbbb2c70a9599cd5c1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:50:25.137990  1448 consensus_queue.cc:1048] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e30a3a1cc646798a918dc1d90bb813" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:50:25.219151   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:25.228148   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 14:50:25.271608   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:25.272331   431 tablet_replica.cc:333] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.273132   431 raft_consensus.cc:2243] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.273720   431 raft_consensus.cc:2272] T c641948ed29645df864b8519726e3a9b P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.276117   431 tablet_replica.cc:333] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.276721   431 raft_consensus.cc:2243] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.277177   431 raft_consensus.cc:2272] T 3a15135475a0481bbe582a59a0cdc970 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.279832   431 tablet_replica.cc:333] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.280501   431 raft_consensus.cc:2243] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.281315   431 raft_consensus.cc:2272] T e13914cc890f49a9aef675fa26059975 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.283210   431 tablet_replica.cc:333] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.283730   431 raft_consensus.cc:2243] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.284152   431 raft_consensus.cc:2272] T 42adb749413c4474a1ce87707146f923 P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.286455   431 tablet_replica.cc:333] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.287120   431 raft_consensus.cc:2243] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.287585   431 raft_consensus.cc:2272] T c61ed07ed88a45efad0fc152edd82509 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.290102   431 tablet_replica.cc:333] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.290691   431 raft_consensus.cc:2243] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.291234   431 raft_consensus.cc:2272] T e1841dab6ad04cfc82540d9e9bac7e77 P 9fef998d350141c895e2167984302880 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.293912   431 tablet_replica.cc:333] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.294548   431 raft_consensus.cc:2243] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.295497   431 raft_consensus.cc:2272] T 5c01fa23f0d64a108fe80f75c150093f P 9fef998d350141c895e2167984302880 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.297719   431 tablet_replica.cc:333] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.298296   431 raft_consensus.cc:2243] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.299201   431 raft_consensus.cc:2272] T 7b3254117a704799bdbf446c04c070f7 P 9fef998d350141c895e2167984302880 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.301095   431 tablet_replica.cc:333] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.301682   431 raft_consensus.cc:2243] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.302095   431 raft_consensus.cc:2272] T cd4d3e2cfebd44a5b371fb14215487f7 P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.304093   431 tablet_replica.cc:333] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880: stopping tablet replica
I20260709 14:50:25.304561   431 raft_consensus.cc:2243] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.305313   431 raft_consensus.cc:2272] T 642bb41f622740c8a71312a693908c7f P 9fef998d350141c895e2167984302880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.346319   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:50:25.353677  1115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44667: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260709 14:50:25.357758  1115 consensus_peers.cc:597] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 -> Peer 9fef998d350141c895e2167984302880 (127.0.107.193:44667): Couldn't send request to peer 9fef998d350141c895e2167984302880. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:25.372891   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 14:50:25.407112   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:25.407987   431 tablet_replica.cc:333] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.408732   431 raft_consensus.cc:2243] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.409618   431 raft_consensus.cc:2272] T c641948ed29645df864b8519726e3a9b P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.411854   431 tablet_replica.cc:333] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.412470   431 raft_consensus.cc:2243] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.412925   431 raft_consensus.cc:2272] T e1841dab6ad04cfc82540d9e9bac7e77 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.415580   431 tablet_replica.cc:333] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.416170   431 raft_consensus.cc:2243] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.417167   431 raft_consensus.cc:2272] T 3a15135475a0481bbe582a59a0cdc970 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.419806   431 tablet_replica.cc:333] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.420361   431 raft_consensus.cc:2243] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.420818   431 raft_consensus.cc:2272] T e13914cc890f49a9aef675fa26059975 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.423389   431 tablet_replica.cc:333] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.424101   431 raft_consensus.cc:2243] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.425074   431 raft_consensus.cc:2272] T 42adb749413c4474a1ce87707146f923 P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.427544   431 tablet_replica.cc:333] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.428401   431 raft_consensus.cc:2243] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.429324   431 raft_consensus.cc:2272] T c61ed07ed88a45efad0fc152edd82509 P 25e30a3a1cc646798a918dc1d90bb813 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.431921   431 tablet_replica.cc:333] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.432564   431 raft_consensus.cc:2243] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.433084   431 raft_consensus.cc:2272] T 7b3254117a704799bdbf446c04c070f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.435031   431 tablet_replica.cc:333] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.435580   431 raft_consensus.cc:2243] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.436025   431 raft_consensus.cc:2272] T 5c01fa23f0d64a108fe80f75c150093f P 25e30a3a1cc646798a918dc1d90bb813 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.437932   431 tablet_replica.cc:333] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.438419   431 raft_consensus.cc:2243] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.438932   431 raft_consensus.cc:2272] T cd4d3e2cfebd44a5b371fb14215487f7 P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.440865   431 tablet_replica.cc:333] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813: stopping tablet replica
I20260709 14:50:25.441381   431 raft_consensus.cc:2243] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.441820   431 raft_consensus.cc:2272] T 642bb41f622740c8a71312a693908c7f P 25e30a3a1cc646798a918dc1d90bb813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.474282   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 14:50:25.491964  1189 consensus_peers.cc:597] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a -> Peer 9fef998d350141c895e2167984302880 (127.0.107.193:44667): Couldn't send request to peer 9fef998d350141c895e2167984302880. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:25.499199  1190 consensus_peers.cc:597] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a -> Peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Couldn't send request to peer 25e30a3a1cc646798a918dc1d90bb813. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:25.501386  1189 consensus_peers.cc:597] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a -> Peer 9fef998d350141c895e2167984302880 (127.0.107.193:44667): Couldn't send request to peer 9fef998d350141c895e2167984302880. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:25.502420  1190 consensus_peers.cc:597] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a -> Peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Couldn't send request to peer 25e30a3a1cc646798a918dc1d90bb813. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:25.506036   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 14:50:25.528824  1190 consensus_peers.cc:597] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a -> Peer 25e30a3a1cc646798a918dc1d90bb813 (127.0.107.194:40325): Couldn't send request to peer 25e30a3a1cc646798a918dc1d90bb813. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:25.539981   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:25.540779   431 tablet_replica.cc:333] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.541411   431 raft_consensus.cc:2243] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.541924   431 raft_consensus.cc:2272] T 3a15135475a0481bbe582a59a0cdc970 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.544741   431 tablet_replica.cc:333] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.545387   431 raft_consensus.cc:2243] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.545856   431 raft_consensus.cc:2272] T 7b3254117a704799bdbf446c04c070f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.547839   431 tablet_replica.cc:333] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.548393   431 raft_consensus.cc:2243] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.548843   431 raft_consensus.cc:2272] T 5c01fa23f0d64a108fe80f75c150093f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.550752   431 tablet_replica.cc:333] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.551510   431 raft_consensus.cc:2243] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.552002   431 raft_consensus.cc:2272] T e13914cc890f49a9aef675fa26059975 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.554121   431 tablet_replica.cc:333] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.554737   431 raft_consensus.cc:2243] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.555701   431 raft_consensus.cc:2272] T c61ed07ed88a45efad0fc152edd82509 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.558091   431 tablet_replica.cc:333] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.558784   431 raft_consensus.cc:2243] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.559358   431 raft_consensus.cc:2272] T 42adb749413c4474a1ce87707146f923 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.561964   431 tablet_replica.cc:333] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.562616   431 raft_consensus.cc:2243] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.563217   431 raft_consensus.cc:2272] T c641948ed29645df864b8519726e3a9b P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.565312   431 tablet_replica.cc:333] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.565900   431 raft_consensus.cc:2243] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.566792   431 raft_consensus.cc:2272] T cd4d3e2cfebd44a5b371fb14215487f7 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.569067   431 tablet_replica.cc:333] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.569597   431 raft_consensus.cc:2243] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:25.570086   431 raft_consensus.cc:2272] T 642bb41f622740c8a71312a693908c7f P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.572067   431 tablet_replica.cc:333] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a: stopping tablet replica
I20260709 14:50:25.572556   431 raft_consensus.cc:2243] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.573354   431 raft_consensus.cc:2272] T e1841dab6ad04cfc82540d9e9bac7e77 P 3ea883dd389c4cbbb2c70a9599cd5c1a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.597281   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:50:25.621423   431 master.cc:562] Master@127.0.107.254:41327 shutting down...
I20260709 14:50:25.648524   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:25.649274   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:25.649647   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1ddb3440e4e647da8a0d1a6656140959: stopping tablet replica
I20260709 14:50:25.671363   431 master.cc:584] Master@127.0.107.254:41327 shutdown complete.
25e30a3a1cc646798a918dc1d90bb813  4
3ea883dd389c4cbbb2c70a9599cd5c1a  1
9fef998d350141c895e2167984302880  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (12434 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 14:50:25.724009   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38079
I20260709 14:50:25.725279   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:25.732895  1459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:25.732889  1458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:25.733515   431 server_base.cc:1061] running on GCE node
W20260709 14:50:25.732882  1461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:25.734795   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:25.735103   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:25.735248   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608625735236 us; error 0 us; skew 500 ppm
I20260709 14:50:25.735778   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:25.738906   431 webserver.cc:533] Webserver started at http://127.0.107.254:34119/ using document root <none> and password file <none>
I20260709 14:50:25.739493   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:25.739681   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:25.739921   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:25.741271   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "36846e51a4be4e68ba398d50874f8d9e"
format_stamp: "Formatted at 2026-07-09 14:50:25 on dist-test-slave-wg8n"
I20260709 14:50:25.746584   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 14:50:25.751008  1466 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:25.751873   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:50:25.752170   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "36846e51a4be4e68ba398d50874f8d9e"
format_stamp: "Formatted at 2026-07-09 14:50:25 on dist-test-slave-wg8n"
I20260709 14:50:25.752465   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:25.794224   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:25.795739   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:25.844116   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38079
I20260709 14:50:25.844223  1517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38079 every 8 connection(s)
I20260709 14:50:25.848807  1518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:25.861202  1518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: Bootstrap starting.
I20260709 14:50:25.866766  1518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:25.872674  1518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: No bootstrap required, opened a new log
I20260709 14:50:25.875808  1518 raft_consensus.cc:359] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER }
I20260709 14:50:25.876263  1518 raft_consensus.cc:385] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:25.876449  1518 raft_consensus.cc:740] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36846e51a4be4e68ba398d50874f8d9e, State: Initialized, Role: FOLLOWER
I20260709 14:50:25.877223  1518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER }
I20260709 14:50:25.877763  1518 raft_consensus.cc:399] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:50:25.877974  1518 raft_consensus.cc:493] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:50:25.878201  1518 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:25.975427  1518 raft_consensus.cc:515] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER }
I20260709 14:50:25.976171  1518 leader_election.cc:304] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36846e51a4be4e68ba398d50874f8d9e; no voters: 
I20260709 14:50:25.979456  1518 leader_election.cc:290] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:50:25.979820  1521 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:25.981642  1521 raft_consensus.cc:697] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 1 LEADER]: Becoming Leader. State: Replica: 36846e51a4be4e68ba398d50874f8d9e, State: Running, Role: LEADER
I20260709 14:50:25.982355  1521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER }
I20260709 14:50:25.983044  1518 sys_catalog.cc:565] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:50:25.987905  1522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36846e51a4be4e68ba398d50874f8d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER } }
I20260709 14:50:25.988332  1523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36846e51a4be4e68ba398d50874f8d9e. Latest consensus state: current_term: 1 leader_uuid: "36846e51a4be4e68ba398d50874f8d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36846e51a4be4e68ba398d50874f8d9e" member_type: VOTER } }
I20260709 14:50:25.988560  1522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:25.988960  1523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:25.990659  1528 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:50:25.997205  1528 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:50:26.001102   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:50:26.008699  1528 catalog_manager.cc:1383] Generated new cluster ID: 209a554551894727b24d125f8b743faf
I20260709 14:50:26.009115  1528 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:50:26.027354  1528 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:50:26.028919  1528 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:50:26.043197  1528 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: Generated new TSK 0
I20260709 14:50:26.043931  1528 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:50:26.072467   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:26.080770  1539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:26.082048  1540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:26.085116   431 server_base.cc:1061] running on GCE node
W20260709 14:50:26.085137  1542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:26.086302   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:26.086498   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:26.086709   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608626086671 us; error 0 us; skew 500 ppm
I20260709 14:50:26.087363   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:26.089999   431 webserver.cc:533] Webserver started at http://127.0.107.193:33155/ using document root <none> and password file <none>
I20260709 14:50:26.090528   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:26.090749   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:26.091116   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:26.092351   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "17e506fd82284f2592f156640c0edafe"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.097918   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 14:50:26.101948  1547 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.102911   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 14:50:26.103277   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "17e506fd82284f2592f156640c0edafe"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.103611   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:26.114912   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:26.116456   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:26.118188   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:26.121037   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:26.121302   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.121578   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:26.121788   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.171774   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43817
I20260709 14:50:26.171893  1609 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43817 every 8 connection(s)
I20260709 14:50:26.177469   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:26.186887  1614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:26.189066  1615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:26.192083  1610 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38079
I20260709 14:50:26.192565  1610 heartbeater.cc:461] Registering TS with master...
W20260709 14:50:26.192683  1617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:26.193562   431 server_base.cc:1061] running on GCE node
I20260709 14:50:26.193717  1610 heartbeater.cc:507] Master 127.0.107.254:38079 requested a full tablet report, sending...
I20260709 14:50:26.195413   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:26.195662   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:26.195880   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608626195858 us; error 0 us; skew 500 ppm
I20260709 14:50:26.196439  1483 ts_manager.cc:194] Registered new tserver with Master: 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817)
I20260709 14:50:26.196741   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:26.199230  1483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39638
I20260709 14:50:26.200125   431 webserver.cc:533] Webserver started at http://127.0.107.194:37911/ using document root <none> and password file <none>
I20260709 14:50:26.200817   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:26.201032   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:26.201403   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:26.203115   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "45d5272855ec440cbd07d4d3995c68d3"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.208047   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 14:50:26.212044  1622 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.212926   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:50:26.213274   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "45d5272855ec440cbd07d4d3995c68d3"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.213582   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:26.223404   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:26.224835   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:26.226532   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:26.229249   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:26.229516   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.229791   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:26.229975   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.278502   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44999
I20260709 14:50:26.278656  1684 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44999 every 8 connection(s)
I20260709 14:50:26.283713   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:26.292925  1688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:26.295022  1689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:26.296623  1691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:26.299244  1685 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38079
I20260709 14:50:26.299754  1685 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:26.299926   431 server_base.cc:1061] running on GCE node
I20260709 14:50:26.300890  1685 heartbeater.cc:507] Master 127.0.107.254:38079 requested a full tablet report, sending...
I20260709 14:50:26.301157   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:26.301425   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:26.301631   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608626301610 us; error 0 us; skew 500 ppm
I20260709 14:50:26.302366   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:26.303628  1483 ts_manager.cc:194] Registered new tserver with Master: 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:26.305491  1483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39654
I20260709 14:50:26.305544   431 webserver.cc:533] Webserver started at http://127.0.107.195:43965/ using document root <none> and password file <none>
I20260709 14:50:26.306267   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:26.306517   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:26.306870   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:26.308140   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "36d78753ff96438e9cc2808f05853294"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.313522   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:50:26.317910  1696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.319068   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:50:26.319391   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "36d78753ff96438e9cc2808f05853294"
format_stamp: "Formatted at 2026-07-09 14:50:26 on dist-test-slave-wg8n"
I20260709 14:50:26.319780   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:26.334971   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:26.336498   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:26.338356   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:26.341181   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:26.341488   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.341770   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:26.341964   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:26.395510   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:38515
I20260709 14:50:26.395679  1758 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:38515 every 8 connection(s)
I20260709 14:50:26.412775  1759 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38079
I20260709 14:50:26.413236  1759 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:26.414074  1759 heartbeater.cc:507] Master 127.0.107.254:38079 requested a full tablet report, sending...
I20260709 14:50:26.416442  1483 ts_manager.cc:194] Registered new tserver with Master: 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:26.417527   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017552138s
I20260709 14:50:26.417996  1483 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39666
I20260709 14:50:26.450055  1483 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39676:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:50:26.457211  1483 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:50:26.675040  1724 tablet_service.cc:1511] Processing CreateTablet for tablet c784884b0ef3459b95c5a9028bed9e61 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 14:50:26.675357  1723 tablet_service.cc:1511] Processing CreateTablet for tablet 3244754255c4494a97a20d6ba372923c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:26.676745  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c784884b0ef3459b95c5a9028bed9e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.677386  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3244754255c4494a97a20d6ba372923c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.678282  1650 tablet_service.cc:1511] Processing CreateTablet for tablet c784884b0ef3459b95c5a9028bed9e61 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 14:50:26.679725  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c784884b0ef3459b95c5a9028bed9e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.684300  1649 tablet_service.cc:1511] Processing CreateTablet for tablet 3244754255c4494a97a20d6ba372923c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:26.686146  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3244754255c4494a97a20d6ba372923c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.685680  1722 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe3cb63a07e4ce6b38156b136569cca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:26.687242  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe3cb63a07e4ce6b38156b136569cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.688408  1648 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe3cb63a07e4ce6b38156b136569cca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:26.689937  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe3cb63a07e4ce6b38156b136569cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.692231  1647 tablet_service.cc:1511] Processing CreateTablet for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:26.692589  1721 tablet_service.cc:1511] Processing CreateTablet for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:26.693809  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.693884  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.697260  1646 tablet_service.cc:1511] Processing CreateTablet for tablet 438a8ba60c454c089cab93412c5c93a1 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:26.698856  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438a8ba60c454c089cab93412c5c93a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.701840  1645 tablet_service.cc:1511] Processing CreateTablet for tablet 550babf7e3344307bf7f14bae6eb5b86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:26.703598  1645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550babf7e3344307bf7f14bae6eb5b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.710162  1718 tablet_service.cc:1511] Processing CreateTablet for tablet 73244150135b4bb5aeeba3dc4b94fab7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:26.711997  1718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73244150135b4bb5aeeba3dc4b94fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.713068  1643 tablet_service.cc:1511] Processing CreateTablet for tablet bc44ef62c96a40118ff58c994395ed5a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:26.714634  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc44ef62c96a40118ff58c994395ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.705247  1720 tablet_service.cc:1511] Processing CreateTablet for tablet 438a8ba60c454c089cab93412c5c93a1 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:26.716053  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438a8ba60c454c089cab93412c5c93a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.718900  1717 tablet_service.cc:1511] Processing CreateTablet for tablet bc44ef62c96a40118ff58c994395ed5a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:26.719533  1642 tablet_service.cc:1511] Processing CreateTablet for tablet b844b9b6088d424d8cd83fa504010c9c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:26.720613  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc44ef62c96a40118ff58c994395ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.721171  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b844b9b6088d424d8cd83fa504010c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.725940  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea7a74d9aa14ef38365e872e3efb562 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:26.728519  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea7a74d9aa14ef38365e872e3efb562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.728773  1716 tablet_service.cc:1511] Processing CreateTablet for tablet b844b9b6088d424d8cd83fa504010c9c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:26.730225  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b844b9b6088d424d8cd83fa504010c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.732286  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea7a74d9aa14ef38365e872e3efb562 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:26.733919  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea7a74d9aa14ef38365e872e3efb562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.707561  1719 tablet_service.cc:1511] Processing CreateTablet for tablet 550babf7e3344307bf7f14bae6eb5b86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:26.736694  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550babf7e3344307bf7f14bae6eb5b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.707466  1644 tablet_service.cc:1511] Processing CreateTablet for tablet 73244150135b4bb5aeeba3dc4b94fab7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:26.739888  1644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73244150135b4bb5aeeba3dc4b94fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.772768  1775 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:26.778685  1647 tablet_service.cc:1511] Processing CreateTablet for tablet c69d6d1223974de892e79da6d6c27d86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:26.780196  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69d6d1223974de892e79da6d6c27d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.785431  1646 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7fe23a70e9483db6bb31a0a6ab1096 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:26.786980  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7fe23a70e9483db6bb31a0a6ab1096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.792572  1775 tablet_bootstrap.cc:654] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:26.815740  1575 tablet_service.cc:1511] Processing CreateTablet for tablet c784884b0ef3459b95c5a9028bed9e61 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 14:50:26.818042  1571 tablet_service.cc:1511] Processing CreateTablet for tablet 438a8ba60c454c089cab93412c5c93a1 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:26.819789  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438a8ba60c454c089cab93412c5c93a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.823040  1570 tablet_service.cc:1511] Processing CreateTablet for tablet 550babf7e3344307bf7f14bae6eb5b86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:26.824867  1570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550babf7e3344307bf7f14bae6eb5b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.825387  1572 tablet_service.cc:1511] Processing CreateTablet for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:26.828646  1574 tablet_service.cc:1511] Processing CreateTablet for tablet 3244754255c4494a97a20d6ba372923c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:26.823312  1573 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe3cb63a07e4ce6b38156b136569cca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:26.831527  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1298fa6acd614e99b7cdb2fc4adbf5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.835515  1573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe3cb63a07e4ce6b38156b136569cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.835592  1569 tablet_service.cc:1511] Processing CreateTablet for tablet 73244150135b4bb5aeeba3dc4b94fab7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:26.839365  1569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73244150135b4bb5aeeba3dc4b94fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.839146  1568 tablet_service.cc:1511] Processing CreateTablet for tablet bc44ef62c96a40118ff58c994395ed5a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:26.844396  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea7a74d9aa14ef38365e872e3efb562 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:26.842197  1567 tablet_service.cc:1511] Processing CreateTablet for tablet b844b9b6088d424d8cd83fa504010c9c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:26.852135  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc44ef62c96a40118ff58c994395ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.860610  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c784884b0ef3459b95c5a9028bed9e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.863435  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea7a74d9aa14ef38365e872e3efb562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.865130  1574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3244754255c4494a97a20d6ba372923c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.866748  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b844b9b6088d424d8cd83fa504010c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.926129  1774 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:26.936230  1646 tablet_service.cc:1511] Processing CreateTablet for tablet b4987548a10d4e82b06659b89058e65c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:26.947633  1647 tablet_service.cc:1511] Processing CreateTablet for tablet 08aaabb4b40d415dacd518d43a09869a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:26.982965  1722 tablet_service.cc:1511] Processing CreateTablet for tablet c69d6d1223974de892e79da6d6c27d86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:26.985797  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69d6d1223974de892e79da6d6c27d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.940467  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4987548a10d4e82b06659b89058e65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:26.994803  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08aaabb4b40d415dacd518d43a09869a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:26.996281  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:27.000982  1774 tablet_bootstrap.cc:654] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.005622  1649 tablet_service.cc:1511] Processing CreateTablet for tablet 2550480abe0c4d12b8c11bf82ec5039b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:27.029009  1571 tablet_service.cc:1511] Processing CreateTablet for tablet c69d6d1223974de892e79da6d6c27d86 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:27.037374  1644 tablet_service.cc:1511] Processing CreateTablet for tablet d0ccd774316a4d69b235f2eddf58ddcc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:27.052034  1775 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:27.052894  1775 ts_tablet_manager.cc:1403] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.281s	user 0.070s	sys 0.135s
I20260709 14:50:27.054632  1642 tablet_service.cc:1511] Processing CreateTablet for tablet 422982f0dc6b4d3ca2fdaf28d0411514 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:27.056027  1723 tablet_service.cc:1511] Processing CreateTablet for tablet b4987548a10d4e82b06659b89058e65c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:27.057391  1648 tablet_service.cc:1511] Processing CreateTablet for tablet 5af1fd4b621c4a69afab73a4d89fa1a6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:27.058164  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4987548a10d4e82b06659b89058e65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.060448  1724 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7fe23a70e9483db6bb31a0a6ab1096 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:27.062007  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7fe23a70e9483db6bb31a0a6ab1096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.063294  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2550480abe0c4d12b8c11bf82ec5039b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.026703  1645 tablet_service.cc:1511] Processing CreateTablet for tablet 9203095ce82145248458d98a71f8f07f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:27.062975  1569 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7fe23a70e9483db6bb31a0a6ab1096 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:27.076889  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69d6d1223974de892e79da6d6c27d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.076400  1719 tablet_service.cc:1511] Processing CreateTablet for tablet 2550480abe0c4d12b8c11bf82ec5039b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:27.077973  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2550480abe0c4d12b8c11bf82ec5039b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.079558  1645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9203095ce82145248458d98a71f8f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.088367  1650 tablet_service.cc:1511] Processing CreateTablet for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:27.090076  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.091801  1721 tablet_service.cc:1511] Processing CreateTablet for tablet 9203095ce82145248458d98a71f8f07f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:27.093118  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af1fd4b621c4a69afab73a4d89fa1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.092937  1720 tablet_service.cc:1511] Processing CreateTablet for tablet 08aaabb4b40d415dacd518d43a09869a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:27.088090  1570 tablet_service.cc:1511] Processing CreateTablet for tablet 08aaabb4b40d415dacd518d43a09869a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:27.094480  1569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7fe23a70e9483db6bb31a0a6ab1096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.108676  1777 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:27.108807  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422982f0dc6b4d3ca2fdaf28d0411514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.100533  1775 raft_consensus.cc:359] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.111750  1775 raft_consensus.cc:385] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.112118  1775 raft_consensus.cc:740] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.113174  1775 consensus_queue.cc:260] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.116963  1644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ccd774316a4d69b235f2eddf58ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.118808  1716 tablet_service.cc:1511] Processing CreateTablet for tablet d0ccd774316a4d69b235f2eddf58ddcc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:27.120514  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ccd774316a4d69b235f2eddf58ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.100907  1718 tablet_service.cc:1511] Processing CreateTablet for tablet 5af1fd4b621c4a69afab73a4d89fa1a6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:27.110754  1717 tablet_service.cc:1511] Processing CreateTablet for tablet 422982f0dc6b4d3ca2fdaf28d0411514 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:27.129807  1641 tablet_service.cc:1511] Processing CreateTablet for tablet c5deb3961d984f8481c1ee3e463f8a3b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:27.131285  1715 tablet_service.cc:1511] Processing CreateTablet for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:27.156136  1575 tablet_service.cc:1511] Processing CreateTablet for tablet 2550480abe0c4d12b8c11bf82ec5039b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:27.158124  1574 tablet_service.cc:1511] Processing CreateTablet for tablet 5af1fd4b621c4a69afab73a4d89fa1a6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:27.159171  1570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08aaabb4b40d415dacd518d43a09869a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.159724  1571 tablet_service.cc:1511] Processing CreateTablet for tablet 422982f0dc6b4d3ca2fdaf28d0411514 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:27.159785  1572 tablet_service.cc:1511] Processing CreateTablet for tablet 9203095ce82145248458d98a71f8f07f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:27.161162  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422982f0dc6b4d3ca2fdaf28d0411514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.094803  1566 tablet_service.cc:1511] Processing CreateTablet for tablet b4987548a10d4e82b06659b89058e65c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:27.163297  1567 tablet_service.cc:1511] Processing CreateTablet for tablet d0ccd774316a4d69b235f2eddf58ddcc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:27.164740  1774 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:27.164835  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ccd774316a4d69b235f2eddf58ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.165402  1774 ts_tablet_manager.cc:1403] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.240s	user 0.014s	sys 0.025s
I20260709 14:50:27.169757  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2550480abe0c4d12b8c11bf82ec5039b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.171221  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08aaabb4b40d415dacd518d43a09869a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.174266  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5deb3961d984f8481c1ee3e463f8a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.187994  1775 ts_tablet_manager.cc:1434] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.135s	user 0.009s	sys 0.016s
I20260709 14:50:27.153030  1643 tablet_service.cc:1511] Processing CreateTablet for tablet a3c555043e8447c4baadb1583c60a7e6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:27.191583  1685 heartbeater.cc:499] Master 127.0.107.254:38079 was elected leader, sending a full tablet report...
I20260709 14:50:27.193200  1647 tablet_service.cc:1511] Processing CreateTablet for tablet ce56315bb31441e1bbe7db96a8ce4410 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:27.195539  1775 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:27.193686  1568 tablet_service.cc:1511] Processing CreateTablet for tablet a3c555043e8447c4baadb1583c60a7e6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:27.197961  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9203095ce82145248458d98a71f8f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.199290  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9203095ce82145248458d98a71f8f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.200381  1573 tablet_service.cc:1511] Processing CreateTablet for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:27.202982  1610 heartbeater.cc:499] Master 127.0.107.254:38079 was elected leader, sending a full tablet report...
I20260709 14:50:27.208832  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422982f0dc6b4d3ca2fdaf28d0411514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.211200  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.223091  1574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af1fd4b621c4a69afab73a4d89fa1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.223793  1779 raft_consensus.cc:493] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:27.224426  1779 raft_consensus.cc:515] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.227219  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4987548a10d4e82b06659b89058e65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.229548  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c555043e8447c4baadb1583c60a7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.233858  1779 leader_election.cc:290] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:27.202992  1777 tablet_bootstrap.cc:654] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.239679  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c555043e8447c4baadb1583c60a7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.243396  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce56315bb31441e1bbe7db96a8ce4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.246659  1718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af1fd4b621c4a69afab73a4d89fa1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.255126  1573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93fe5d42f4a4b38a4d1adc56b5fcabb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.265854  1648 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:27.283018  1650 tablet_service.cc:1511] Processing CreateTablet for tablet fcd1126d930d488dab086eb5d690385a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:27.284610  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.308993  1719 tablet_service.cc:1511] Processing CreateTablet for tablet a3c555043e8447c4baadb1583c60a7e6 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:27.312229  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd1126d930d488dab086eb5d690385a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.317317  1775 tablet_bootstrap.cc:654] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.317320  1571 tablet_service.cc:1511] Processing CreateTablet for tablet c5deb3961d984f8481c1ee3e463f8a3b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:27.320390  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5deb3961d984f8481c1ee3e463f8a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.321192  1723 tablet_service.cc:1511] Processing CreateTablet for tablet c5deb3961d984f8481c1ee3e463f8a3b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:27.321066  1646 tablet_service.cc:1511] Processing CreateTablet for tablet a043e41dae8347b3988abf806a866b74 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:27.310668  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c555043e8447c4baadb1583c60a7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.333700  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a043e41dae8347b3988abf806a866b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.337937  1642 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6621a92ee54b33a369dab7c279fc50 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:27.341758  1643 tablet_service.cc:1511] Processing CreateTablet for tablet 59f4b54cf7034f25bdd1182b79e561d7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:27.343163  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5deb3961d984f8481c1ee3e463f8a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.343410  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f4b54cf7034f25bdd1182b79e561d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.356531  1645 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4ff68bffd45df8bbdafb01119e3aa (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:27.358187  1645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4ff68bffd45df8bbdafb01119e3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.334815  1777 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:27.377696  1721 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:27.377802  1568 tablet_service.cc:1511] Processing CreateTablet for tablet ce56315bb31441e1bbe7db96a8ce4410 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:27.386894  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6621a92ee54b33a369dab7c279fc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.388171  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 72c265ae69744159a2cbd5fea8af0d07 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:27.388813  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce56315bb31441e1bbe7db96a8ce4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.389659  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c265ae69744159a2cbd5fea8af0d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.301189  1774 raft_consensus.cc:359] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.392089  1774 raft_consensus.cc:385] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.392447  1774 raft_consensus.cc:740] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.393046  1649 tablet_service.cc:1511] Processing CreateTablet for tablet 915bcb7025e246cf9db5e084d3115085 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:27.393304  1774 consensus_queue.cc:260] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.394500  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915bcb7025e246cf9db5e084d3115085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.405635  1575 tablet_service.cc:1511] Processing CreateTablet for tablet fcd1126d930d488dab086eb5d690385a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:27.410756  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd1126d930d488dab086eb5d690385a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.367841  1777 ts_tablet_manager.cc:1403] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.259s	user 0.021s	sys 0.012s
I20260709 14:50:27.419461  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.420910  1759 heartbeater.cc:499] Master 127.0.107.254:38079 was elected leader, sending a full tablet report...
I20260709 14:50:27.420567  1724 tablet_service.cc:1511] Processing CreateTablet for tablet ce56315bb31441e1bbe7db96a8ce4410 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:27.424883  1643 tablet_service.cc:1511] Processing CreateTablet for tablet b259d0baa0c4493eba83ff1de96cf93f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:27.426364  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b259d0baa0c4493eba83ff1de96cf93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.431288  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce56315bb31441e1bbe7db96a8ce4410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.436334  1720 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6621a92ee54b33a369dab7c279fc50 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:27.437736  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6621a92ee54b33a369dab7c279fc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.440851  1717 tablet_service.cc:1511] Processing CreateTablet for tablet a043e41dae8347b3988abf806a866b74 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:27.442337  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a043e41dae8347b3988abf806a866b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.433139  1722 tablet_service.cc:1511] Processing CreateTablet for tablet fcd1126d930d488dab086eb5d690385a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:27.458643  1644 tablet_service.cc:1511] Processing CreateTablet for tablet 4d58c741dd1442bf9096d719c5d54058 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:27.460716  1644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d58c741dd1442bf9096d719c5d54058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.482434  1775 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:27.483359  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd1126d930d488dab086eb5d690385a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.492573  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:27.494776  1623 leader_election.cc:343] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:27.494773  1572 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:27.496587  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd3d3a0c81c4e6e83d700d44aa3d2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.498862  1575 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6621a92ee54b33a369dab7c279fc50 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:27.500324  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6621a92ee54b33a369dab7c279fc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.506075  1774 ts_tablet_manager.cc:1434] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.340s	user 0.010s	sys 0.010s
I20260709 14:50:27.507366  1774 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:27.510047  1569 tablet_service.cc:1511] Processing CreateTablet for tablet 72c265ae69744159a2cbd5fea8af0d07 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:27.456590  1777 raft_consensus.cc:359] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.450625  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 59f4b54cf7034f25bdd1182b79e561d7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:27.528334  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:27.547398  1777 raft_consensus.cc:385] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.547866  1777 raft_consensus.cc:740] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.548965  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f4b54cf7034f25bdd1182b79e561d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.547849  1569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c265ae69744159a2cbd5fea8af0d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.552802  1777 consensus_queue.cc:260] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.529498  1567 tablet_service.cc:1511] Processing CreateTablet for tablet 915bcb7025e246cf9db5e084d3115085 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:27.531060  1566 tablet_service.cc:1511] Processing CreateTablet for tablet a043e41dae8347b3988abf806a866b74 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:27.498720  1716 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4ff68bffd45df8bbdafb01119e3aa (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:27.532534  1775 ts_tablet_manager.cc:1403] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.337s	user 0.015s	sys 0.005s
I20260709 14:50:27.530330  1574 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4ff68bffd45df8bbdafb01119e3aa (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:27.567574  1571 tablet_service.cc:1511] Processing CreateTablet for tablet 4d58c741dd1442bf9096d719c5d54058 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:27.569032  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4ff68bffd45df8bbdafb01119e3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.571913  1568 tablet_service.cc:1511] Processing CreateTablet for tablet b259d0baa0c4493eba83ff1de96cf93f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:27.572363  1648 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce2a67caa54429fb9510b8ee027fba5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:27.572811  1573 tablet_service.cc:1511] Processing CreateTablet for tablet b0a4bcec2d91452aa995a62be8b6c26b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:27.573691  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce2a67caa54429fb9510b8ee027fba5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:27.575392  1624 leader_election.cc:343] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:27.575909  1624 leader_election.cc:304] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:27.577215  1779 raft_consensus.cc:2749] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:27.466336  1646 tablet_service.cc:1511] Processing CreateTablet for tablet b0a4bcec2d91452aa995a62be8b6c26b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:27.560920  1647 tablet_service.cc:1511] Processing CreateTablet for tablet 4130f75a588248fba88156660a11b6e8 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:27.563340  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915bcb7025e246cf9db5e084d3115085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.584033  1775 raft_consensus.cc:359] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.585547  1775 raft_consensus.cc:385] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.586154  1775 raft_consensus.cc:740] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.585565  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4130f75a588248fba88156660a11b6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.589617  1718 tablet_service.cc:1511] Processing CreateTablet for tablet 72c265ae69744159a2cbd5fea8af0d07 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:27.593231  1775 consensus_queue.cc:260] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.594530  1575 tablet_service.cc:1511] Processing CreateTablet for tablet f22f5227c90341148049d7ab114e5272 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:27.596364  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a043e41dae8347b3988abf806a866b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.597082  1574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4ff68bffd45df8bbdafb01119e3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.599293  1573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a4bcec2d91452aa995a62be8b6c26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.600158  1645 tablet_service.cc:1511] Processing CreateTablet for tablet 66b38ba9a15445b787fe3e5738e58a0f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:27.600270  1719 tablet_service.cc:1511] Processing CreateTablet for tablet 915bcb7025e246cf9db5e084d3115085 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:27.608603  1721 tablet_service.cc:1511] Processing CreateTablet for tablet 4d58c741dd1442bf9096d719c5d54058 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:27.609918  1570 tablet_service.cc:1511] Processing CreateTablet for tablet 59f4b54cf7034f25bdd1182b79e561d7 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:27.611512  1570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f4b54cf7034f25bdd1182b79e561d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.612418  1777 ts_tablet_manager.cc:1434] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.197s	user 0.007s	sys 0.009s
I20260709 14:50:27.613698  1777 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:27.614270  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b259d0baa0c4493eba83ff1de96cf93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.614995  1650 tablet_service.cc:1511] Processing CreateTablet for tablet f22f5227c90341148049d7ab114e5272 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:27.616535  1775 ts_tablet_manager.cc:1434] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 14:50:27.617835  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22f5227c90341148049d7ab114e5272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.619293  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d58c741dd1442bf9096d719c5d54058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.634598  1718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c265ae69744159a2cbd5fea8af0d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.636132  1649 tablet_service.cc:1511] Processing CreateTablet for tablet b1c0c3916ca4477fb2e82c720b4c1e87 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:27.639698  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915bcb7025e246cf9db5e084d3115085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.643610  1645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b38ba9a15445b787fe3e5738e58a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.645181  1775 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:27.648252  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22f5227c90341148049d7ab114e5272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.648691  1642 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9d4f77ab264824824b9de6802cec54 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 14:50:27.652456  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0c3916ca4477fb2e82c720b4c1e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.653417  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a4bcec2d91452aa995a62be8b6c26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.655865  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 575fecfb4a7e4812b9b8f6fb093936b5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:27.657359  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575fecfb4a7e4812b9b8f6fb093936b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.662364  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9d4f77ab264824824b9de6802cec54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.669701  1644 tablet_service.cc:1511] Processing CreateTablet for tablet 6915b13ffb704edcac0fd629381d253e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:27.694260  1644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6915b13ffb704edcac0fd629381d253e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.671175  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d58c741dd1442bf9096d719c5d54058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.673166  1786 raft_consensus.cc:493] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:27.675658  1774 tablet_bootstrap.cc:654] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.699429  1786 raft_consensus.cc:515] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.681097  1568 tablet_service.cc:1511] Processing CreateTablet for tablet 4130f75a588248fba88156660a11b6e8 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:27.703403  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4130f75a588248fba88156660a11b6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.703924  1784 raft_consensus.cc:493] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:27.704473  1784 raft_consensus.cc:515] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.682294  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce2a67caa54429fb9510b8ee027fba5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:27.703475  1775 tablet_bootstrap.cc:654] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.685988  1569 tablet_service.cc:1511] Processing CreateTablet for tablet 66b38ba9a15445b787fe3e5738e58a0f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:27.725159  1724 tablet_service.cc:1511] Processing CreateTablet for tablet b259d0baa0c4493eba83ff1de96cf93f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:27.705998  1786 leader_election.cc:290] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:27.729547  1569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b38ba9a15445b787fe3e5738e58a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.726799  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b259d0baa0c4493eba83ff1de96cf93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.753520  1572 tablet_service.cc:1511] Processing CreateTablet for tablet b1c0c3916ca4477fb2e82c720b4c1e87 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:27.753729  1717 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce2a67caa54429fb9510b8ee027fba5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:27.755734  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 1e64847f9b324f7faefa87f278434672 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:27.757411  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e64847f9b324f7faefa87f278434672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.760056  1784 leader_election.cc:290] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:27.768258  1777 tablet_bootstrap.cc:654] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:27.770774  1774 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:27.729219  1723 tablet_service.cc:1511] Processing CreateTablet for tablet f22f5227c90341148049d7ab114e5272 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:27.771430  1774 ts_tablet_manager.cc:1403] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.264s	user 0.009s	sys 0.008s
I20260709 14:50:27.781862  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 66b38ba9a15445b787fe3e5738e58a0f (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:27.768919  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce2a67caa54429fb9510b8ee027fba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.728277  1720 tablet_service.cc:1511] Processing CreateTablet for tablet b0a4bcec2d91452aa995a62be8b6c26b (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:27.786713  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce2a67caa54429fb9510b8ee027fba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.791406  1648 tablet_service.cc:1511] Processing CreateTablet for tablet 610d6977f8694bfe8fcccaa5d37d222a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:27.792970  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610d6977f8694bfe8fcccaa5d37d222a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.830132  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22f5227c90341148049d7ab114e5272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.818850  1649 tablet_service.cc:1511] Processing CreateTablet for tablet 630ab4f3cb154a0bb2fa8c41fbe95434 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 14:50:27.855118  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630ab4f3cb154a0bb2fa8c41fbe95434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.859411  1722 tablet_service.cc:1511] Processing CreateTablet for tablet 4130f75a588248fba88156660a11b6e8 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:27.867201  1647 tablet_service.cc:1511] Processing CreateTablet for tablet ab75a712ecfe4deeb97bc690ad53e39c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:27.868970  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75a712ecfe4deeb97bc690ad53e39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.803367  1774 raft_consensus.cc:359] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:27.877019  1774 raft_consensus.cc:385] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.877425  1774 raft_consensus.cc:740] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.878312  1774 consensus_queue.cc:260] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:27.878808  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b38ba9a15445b787fe3e5738e58a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.878552  1643 tablet_service.cc:1511] Processing CreateTablet for tablet b0e50386aa8642078375fbb0a9fb3f82 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:27.880056  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e50386aa8642078375fbb0a9fb3f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.880839  1774 ts_tablet_manager.cc:1434] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.109s	user 0.005s	sys 0.003s
I20260709 14:50:27.882261  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a4bcec2d91452aa995a62be8b6c26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.885133  1716 tablet_service.cc:1511] Processing CreateTablet for tablet b1c0c3916ca4477fb2e82c720b4c1e87 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:27.886654  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0c3916ca4477fb2e82c720b4c1e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.900025  1777 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:27.907493  1774 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:27.918210  1574 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 28) took 1044 ms. Trace:
I20260709 14:50:27.918911  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4130f75a588248fba88156660a11b6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.918767  1574 rpcz_store.cc:276] 0709 14:50:26.873357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.505948 (+632591us) service_pool.cc:224] Handling call
0709 14:50:27.567185 (+ 61237us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.567322 (+   137us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.687377 (+120055us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.781253 (+ 93876us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.918115 (+136862us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.921411  1574 tablet_service.cc:1511] Processing CreateTablet for tablet 6915b13ffb704edcac0fd629381d253e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:27.923231  1573 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 33) took 1005 ms. Trace:
I20260709 14:50:27.923681  1571 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 31) took 1026 ms. Trace:
I20260709 14:50:27.923637  1573 rpcz_store.cc:276] 0709 14:50:26.918179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.571890 (+653711us) service_pool.cc:224] Handling call
0709 14:50:27.573651 (+  1761us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.573750 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.630185 (+ 56435us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.784274 (+154089us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.923146 (+138872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.924125  1571 rpcz_store.cc:276] 0709 14:50:26.897326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.566449 (+669123us) service_pool.cc:224] Handling call
0709 14:50:27.568560 (+  2111us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.568671 (+   111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.725894 (+157223us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.922964 (+197070us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.923602 (+   638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.926050  1573 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9d4f77ab264824824b9de6802cec54 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 14:50:27.930065  1570 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 27) took 1057 ms. Trace:
I20260709 14:50:27.930517  1570 rpcz_store.cc:276] 0709 14:50:26.873003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.499264 (+626261us) service_pool.cc:224] Handling call
0709 14:50:27.610981 (+111717us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.611086 (+   105us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.719747 (+108661us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.849718 (+129971us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.929991 (+ 80273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.931244  1567 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 30) took 1044 ms. Trace:
I20260709 14:50:27.931681  1567 rpcz_store.cc:276] 0709 14:50:26.887028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.528312 (+641284us) service_pool.cc:224] Handling call
0709 14:50:27.562710 (+ 34398us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.562877 (+   167us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.634430 (+ 71553us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.882428 (+247998us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.931166 (+ 48738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.933027  1574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6915b13ffb704edcac0fd629381d253e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.933832  1567 tablet_service.cc:1511] Processing CreateTablet for tablet 1e64847f9b324f7faefa87f278434672 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:27.937309  1570 tablet_service.cc:1511] Processing CreateTablet for tablet 575fecfb4a7e4812b9b8f6fb093936b5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:27.938630  1573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9d4f77ab264824824b9de6802cec54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.944172  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:27.944866  1775 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:27.945441  1775 ts_tablet_manager.cc:1403] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.301s	user 0.028s	sys 0.061s
I20260709 14:50:27.926050  1571 tablet_service.cc:1511] Processing CreateTablet for tablet ab75a712ecfe4deeb97bc690ad53e39c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:27.948818  1719 tablet_service.cc:1511] Processing CreateTablet for tablet 6915b13ffb704edcac0fd629381d253e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:27.950176  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6915b13ffb704edcac0fd629381d253e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.957515  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e64847f9b324f7faefa87f278434672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.957962  1650 tablet_service.cc:1511] Processing CreateTablet for tablet 5242c2facbbb46338963db64018b812e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 14:50:27.959534  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5242c2facbbb46338963db64018b812e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.964293  1570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575fecfb4a7e4812b9b8f6fb093936b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.959405  1642 tablet_service.cc:1511] Processing CreateTablet for tablet c7f577518d474c0782bcd1edaefde746 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:27.965097  1775 raft_consensus.cc:359] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:27.965965  1775 raft_consensus.cc:385] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:27.966037  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f577518d474c0782bcd1edaefde746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.966372  1775 raft_consensus.cc:740] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:27.967447  1575 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 34) took 1019 ms. Trace:
I20260709 14:50:27.967768  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0c3916ca4477fb2e82c720b4c1e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.967898  1575 rpcz_store.cc:276] 0709 14:50:26.948292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.593544 (+645252us) service_pool.cc:224] Handling call
0709 14:50:27.617235 (+ 23691us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.617348 (+   113us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.687377 (+ 70029us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.916899 (+229522us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.967372 (+ 50473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:27.969671  1644 tablet_service.cc:1511] Processing CreateTablet for tablet 806dd19b6e18429c8743b9aceda41aa5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:27.971230  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75a712ecfe4deeb97bc690ad53e39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:27.971062  1575 tablet_service.cc:1511] Processing CreateTablet for tablet 610d6977f8694bfe8fcccaa5d37d222a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:27.972374  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610d6977f8694bfe8fcccaa5d37d222a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:27.975589  1548 leader_election.cc:343] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:27.979298  1777 ts_tablet_manager.cc:1403] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.366s	user 0.031s	sys 0.032s
I20260709 14:50:27.993185  1721 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 31) took 1089 ms. Trace:
I20260709 14:50:27.995141  1721 rpcz_store.cc:276] 0709 14:50:26.903942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.607613 (+703671us) service_pool.cc:224] Handling call
0709 14:50:27.609619 (+  2006us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.609738 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.907488 (+297750us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:27.954758 (+ 47270us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:27.993105 (+ 38347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.001646  1721 tablet_service.cc:1511] Processing CreateTablet for tablet ab75a712ecfe4deeb97bc690ad53e39c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:28.004581  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75a712ecfe4deeb97bc690ad53e39c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:28.006289  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:28.007722  1724 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 32) took 1094 ms. Trace:
I20260709 14:50:28.008268  1724 rpcz_store.cc:276] 0709 14:50:26.913535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.710916 (+797381us) service_pool.cc:224] Handling call
0709 14:50:27.726225 (+ 15309us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.726343 (+   118us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.935108 (+208765us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.007037 (+ 71929us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.007648 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.009425  1723 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 34) took 1028 ms. Trace:
I20260709 14:50:28.009860  1723 rpcz_store.cc:276] 0709 14:50:26.981546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.728174 (+746628us) service_pool.cc:224] Handling call
0709 14:50:27.771435 (+ 43261us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.771548 (+   113us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.950493 (+178945us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.008834 (+ 58341us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.009352 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.014182  1724 tablet_service.cc:1511] Processing CreateTablet for tablet 1e64847f9b324f7faefa87f278434672 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:28.023130  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e64847f9b324f7faefa87f278434672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.033241  1718 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 29) took 1150 ms. Trace:
I20260709 14:50:28.033751  1718 rpcz_store.cc:276] 0709 14:50:26.882731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.588544 (+705813us) service_pool.cc:224] Handling call
0709 14:50:27.590542 (+  1998us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.590651 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.917404 (+326753us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.032604 (+115200us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.033151 (+   547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.034880  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:27.999152  1775 consensus_queue.cc:260] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.036131  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3244754255c4494a97a20d6ba372923c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.037307  1775 ts_tablet_manager.cc:1434] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.001s
I20260709 14:50:28.038446  1775 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.042351  1568 tablet_service.cc:1511] Processing CreateTablet for tablet 630ab4f3cb154a0bb2fa8c41fbe95434 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 14:50:28.043639  1549 leader_election.cc:343] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.056003  1549 leader_election.cc:304] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:28.057555  1786 raft_consensus.cc:2749] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:50:28.044824  1697 leader_election.cc:343] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.063616  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3244754255c4494a97a20d6ba372923c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.064414  1660 raft_consensus.cc:2468] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:28.066294  1697 leader_election.cc:304] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:28.067603  1784 raft_consensus.cc:2804] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.067988  1784 raft_consensus.cc:493] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.068356  1784 raft_consensus.cc:3060] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.068979  1779 raft_consensus.cc:493] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.069258  1774 tablet_bootstrap.cc:654] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.069509  1779 raft_consensus.cc:515] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.071969  1779 leader_election.cc:290] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.073822  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.047735  1646 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38600 (request call id 33) took 1100 ms. Trace:
I20260709 14:50:28.048352  1720 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 33) took 1122 ms. Trace:
I20260709 14:50:28.075095  1646 rpcz_store.cc:276] 0709 14:50:26.947171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.464854 (+517683us) service_pool.cc:224] Handling call
0709 14:50:27.646898 (+182044us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.649676 (+  2778us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.858342 (+208666us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.047106 (+188764us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.047661 (+   555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.075240  1720 rpcz_store.cc:276] 0709 14:50:26.926346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.727082 (+800736us) service_pool.cc:224] Handling call
0709 14:50:27.798294 (+ 71212us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.821265 (+ 22971us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.016308 (+195043us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.047823 (+ 31515us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.048282 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.077416  1718 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9d4f77ab264824824b9de6802cec54 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 14:50:28.077793  1624 leader_election.cc:343] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.049991  1645 tablet_service.cc:1511] Processing CreateTablet for tablet f5b7cdd0e0a24e54b17023ff094ef854 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:28.028508  1644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806dd19b6e18429c8743b9aceda41aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.048242  1777 raft_consensus.cc:359] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.083303  1777 raft_consensus.cc:385] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.083093  1646 tablet_service.cc:1511] Processing CreateTablet for tablet ebb4e4a3db5f46deb623118788ef2fca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:28.083669  1777 raft_consensus.cc:740] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.084688  1777 consensus_queue.cc:260] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.083092  1645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b7cdd0e0a24e54b17023ff094ef854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.088632  1648 tablet_service.cc:1511] Processing CreateTablet for tablet e4fc279a847e48d5a240ecf6cc61c035 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:28.090188  1648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fc279a847e48d5a240ecf6cc61c035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.092983  1777 ts_tablet_manager.cc:1434] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.002s
I20260709 14:50:28.077698  1720 tablet_service.cc:1511] Processing CreateTablet for tablet 610d6977f8694bfe8fcccaa5d37d222a (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:28.099493  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 6015635506e44421bfc21fc36bdb0191 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:28.100528  1718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9d4f77ab264824824b9de6802cec54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.101078  1647 tablet_service.cc:1511] Processing CreateTablet for tablet d08c454fd6fc4270ad8e5ca2542757b0 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:28.102043  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.068186  1723 tablet_service.cc:1511] Processing CreateTablet for tablet 575fecfb4a7e4812b9b8f6fb093936b5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:28.103752  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630ab4f3cb154a0bb2fa8c41fbe95434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.116772  1573 tablet_service.cc:1511] Processing CreateTablet for tablet b0e50386aa8642078375fbb0a9fb3f82 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:28.114924  1572 tablet_service.cc:1511] Processing CreateTablet for tablet c7f577518d474c0782bcd1edaefde746 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:28.118224  1573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e50386aa8642078375fbb0a9fb3f82. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:28.118896  1623 leader_election.cc:343] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.119414  1623 leader_election.cc:304] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:28.121230  1779 raft_consensus.cc:2749] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.107987  1784 raft_consensus.cc:515] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.112159  1566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 36) took 1002 ms. Trace:
I20260709 14:50:28.125474  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610d6977f8694bfe8fcccaa5d37d222a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.124609  1566 rpcz_store.cc:276] 0709 14:50:27.109910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.680380 (+570470us) service_pool.cc:224] Handling call
0709 14:50:27.717274 (+ 36894us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.738213 (+ 20939us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.993841 (+255628us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.059693 (+ 65852us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.112063 (+ 52370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51240}
I20260709 14:50:28.125576  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3244754255c4494a97a20d6ba372923c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:28.127362  1697 leader_election.cc:343] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.127235  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 5242c2facbbb46338963db64018b812e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 14:50:28.114140  1777 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.132977  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 575fecfb4a7e4812b9b8f6fb093936b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.133869  1717 tablet_service.cc:1511] Processing CreateTablet for tablet 630ab4f3cb154a0bb2fa8c41fbe95434 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 14:50:28.131661  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f577518d474c0782bcd1edaefde746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.135408  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630ab4f3cb154a0bb2fa8c41fbe95434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.139050  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5242c2facbbb46338963db64018b812e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.139334  1569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 37) took 1003 ms. Trace:
I20260709 14:50:28.125025  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6015635506e44421bfc21fc36bdb0191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.139719  1569 rpcz_store.cc:276] 0709 14:50:27.135606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.684846 (+549240us) service_pool.cc:224] Handling call
0709 14:50:27.707406 (+ 22560us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.712526 (+  5120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:27.919078 (+206552us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.066726 (+147648us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.139257 (+ 72531us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3223}
I20260709 14:50:28.142133  1647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08c454fd6fc4270ad8e5ca2542757b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.147296  1649 tablet_service.cc:1511] Processing CreateTablet for tablet 35708989e3c345b3a0bad8e5ecebc998 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:28.148725  1646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb4e4a3db5f46deb623118788ef2fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.148990  1650 tablet_service.cc:1511] Processing CreateTablet for tablet bc051130230a45eba628e68c39f0cda3 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:28.163369  1575 tablet_service.cc:1511] Processing CreateTablet for tablet 6015635506e44421bfc21fc36bdb0191 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:28.164466  1569 tablet_service.cc:1511] Processing CreateTablet for tablet 806dd19b6e18429c8743b9aceda41aa5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:28.165341  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6015635506e44421bfc21fc36bdb0191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.167490  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3244754255c4494a97a20d6ba372923c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:28.168298  1660 raft_consensus.cc:3060] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.172160  1774 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.172799  1774 ts_tablet_manager.cc:1403] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.266s	user 0.008s	sys 0.008s
I20260709 14:50:28.177289  1660 raft_consensus.cc:2468] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:28.185467  1697 leader_election.cc:304] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:28.187285  1569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806dd19b6e18429c8743b9aceda41aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.179311  1784 leader_election.cc:290] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:28.192842  1784 raft_consensus.cc:493] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.195888  1784 raft_consensus.cc:515] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.183254  1650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc051130230a45eba628e68c39f0cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.196969  1642 tablet_service.cc:1511] Processing CreateTablet for tablet b689c1f7b39446488c097ffeb75f608c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:28.201408  1643 tablet_service.cc:1511] Processing CreateTablet for tablet 07e44a51657244bdac12c9e76cc53017 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:28.211499  1777 tablet_bootstrap.cc:654] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.211854  1642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b689c1f7b39446488c097ffeb75f608c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.147169  1571 tablet_service.cc:1511] Processing CreateTablet for tablet f5b7cdd0e0a24e54b17023ff094ef854 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:28.204985  1721 tablet_service.cc:1511] Processing CreateTablet for tablet b0e50386aa8642078375fbb0a9fb3f82 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:28.218358  1574 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54174 (request call id 39) took 1028 ms. Trace:
I20260709 14:50:28.218773  1574 rpcz_store.cc:276] 0709 14:50:27.190247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.920342 (+730095us) service_pool.cc:224] Handling call
0709 14:50:27.922323 (+  1981us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.922441 (+   118us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.089377 (+166936us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.217807 (+128430us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.218291 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.219563  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e50386aa8642078375fbb0a9fb3f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.220891  1574 tablet_service.cc:1511] Processing CreateTablet for tablet d08c454fd6fc4270ad8e5ca2542757b0 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:28.222301  1574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08c454fd6fc4270ad8e5ca2542757b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.223870  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b7cdd0e0a24e54b17023ff094ef854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.194165  1722 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 35) took 1085 ms. Trace:
I20260709 14:50:28.216511  1643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e44a51657244bdac12c9e76cc53017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.231065  1722 rpcz_store.cc:276] 0709 14:50:27.108650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.746479 (+637829us) service_pool.cc:224] Handling call
0709 14:50:27.860407 (+113928us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.860515 (+   108us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.074135 (+213620us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.193528 (+119393us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.194084 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.233476  1722 tablet_service.cc:1511] Processing CreateTablet for tablet 5242c2facbbb46338963db64018b812e (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 14:50:28.235399  1570 tablet_service.cc:1511] Processing CreateTablet for tablet e4fc279a847e48d5a240ecf6cc61c035 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:28.218037  1716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 38) took 1032 ms. Trace:
I20260709 14:50:28.236939  1570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fc279a847e48d5a240ecf6cc61c035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.237030  1775 tablet_bootstrap.cc:654] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.243300  1797 raft_consensus.cc:2804] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:28.243932  1797 raft_consensus.cc:697] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:28.244853  1797 consensus_queue.cc:237] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.250979  1649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35708989e3c345b3a0bad8e5ecebc998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.252748  1715 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 37) took 1096 ms. Trace:
I20260709 14:50:28.253268  1715 rpcz_store.cc:276] 0709 14:50:27.156327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.780734 (+624407us) service_pool.cc:224] Handling call
0709 14:50:27.783060 (+  2326us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.783182 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.024693 (+241511us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.252121 (+227428us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.252643 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.255601  1567 tablet_service.cc:1511] Processing CreateTablet for tablet ebb4e4a3db5f46deb623118788ef2fca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:28.241905  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5242c2facbbb46338963db64018b812e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:28.259649  1697 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 74298.3us
I20260709 14:50:28.261799  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.261899  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:28.264516  1697 leader_election.cc:343] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:28.265290  1697 leader_election.cc:343] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.265697  1697 leader_election.cc:304] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:28.236387  1716 rpcz_store.cc:276] 0709 14:50:27.188730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.884056 (+695326us) service_pool.cc:224] Handling call
0709 14:50:27.886102 (+  2046us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.886224 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.058421 (+172197us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.217394 (+158973us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.217958 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.252853  1784 leader_election.cc:290] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817)
I20260709 14:50:28.257176  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb4e4a3db5f46deb623118788ef2fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.278869  1719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54258 (request call id 39) took 1052 ms. Trace:
I20260709 14:50:28.279275  1719 rpcz_store.cc:276] 0709 14:50:27.226158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:27.947695 (+721537us) service_pool.cc:224] Handling call
0709 14:50:27.949629 (+  1934us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:27.949762 (+   133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:28.121579 (+171817us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:28.278261 (+156682us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:28.278762 (+   501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:50:28.286129  1715 tablet_service.cc:1511] Processing CreateTablet for tablet c7f577518d474c0782bcd1edaefde746 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:28.287706  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f577518d474c0782bcd1edaefde746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.287629  1719 tablet_service.cc:1511] Processing CreateTablet for tablet f5b7cdd0e0a24e54b17023ff094ef854 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:28.278512  1774 raft_consensus.cc:359] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.291579  1774 raft_consensus.cc:385] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.292035  1774 raft_consensus.cc:740] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.293164  1774 consensus_queue.cc:260] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.274756  1784 raft_consensus.cc:2749] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.320312  1724 tablet_service.cc:1511] Processing CreateTablet for tablet 6015635506e44421bfc21fc36bdb0191 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:28.321074  1718 tablet_service.cc:1511] Processing CreateTablet for tablet ebb4e4a3db5f46deb623118788ef2fca (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:28.324446  1774 ts_tablet_manager.cc:1434] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.151s	user 0.009s	sys 0.002s
I20260709 14:50:28.325925  1716 tablet_service.cc:1511] Processing CreateTablet for tablet 806dd19b6e18429c8743b9aceda41aa5 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:28.325922  1724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6015635506e44421bfc21fc36bdb0191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.328552  1716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806dd19b6e18429c8743b9aceda41aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.329766  1774 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.335529  1718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb4e4a3db5f46deb623118788ef2fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.336297  1723 tablet_service.cc:1511] Processing CreateTablet for tablet e4fc279a847e48d5a240ecf6cc61c035 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:28.342904  1723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fc279a847e48d5a240ecf6cc61c035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.359511  1720 tablet_service.cc:1511] Processing CreateTablet for tablet d08c454fd6fc4270ad8e5ca2542757b0 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:28.361934  1720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08c454fd6fc4270ad8e5ca2542757b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.350298  1572 tablet_service.cc:1511] Processing CreateTablet for tablet 07e44a51657244bdac12c9e76cc53017 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:28.367983  1572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e44a51657244bdac12c9e76cc53017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.371768  1566 tablet_service.cc:1511] Processing CreateTablet for tablet bc051130230a45eba628e68c39f0cda3 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:28.373317  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc051130230a45eba628e68c39f0cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.375983  1715 tablet_service.cc:1511] Processing CreateTablet for tablet bc051130230a45eba628e68c39f0cda3 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:28.377458  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc051130230a45eba628e68c39f0cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.333460  1575 tablet_service.cc:1511] Processing CreateTablet for tablet 35708989e3c345b3a0bad8e5ecebc998 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:28.387171  1719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b7cdd0e0a24e54b17023ff094ef854. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:28.398952  1697 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 133667us
I20260709 14:50:28.399751  1575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35708989e3c345b3a0bad8e5ecebc998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.448881  1777 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.449458  1777 ts_tablet_manager.cc:1403] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.336s	user 0.028s	sys 0.014s
I20260709 14:50:28.411214  1571 tablet_service.cc:1511] Processing CreateTablet for tablet b689c1f7b39446488c097ffeb75f608c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:28.450397  1775 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.460120  1775 ts_tablet_manager.cc:1403] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.422s	user 0.017s	sys 0.011s
I20260709 14:50:28.465108  1721 tablet_service.cc:1511] Processing CreateTablet for tablet 07e44a51657244bdac12c9e76cc53017 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:28.466187  1774 tablet_bootstrap.cc:654] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.466781  1777 raft_consensus.cc:359] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.467624  1571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b689c1f7b39446488c097ffeb75f608c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.467671  1777 raft_consensus.cc:385] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.468293  1777 raft_consensus.cc:740] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.469174  1777 consensus_queue.cc:260] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.475382  1777 ts_tablet_manager.cc:1434] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.006s
I20260709 14:50:28.476599  1777 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.446797  1722 tablet_service.cc:1511] Processing CreateTablet for tablet 35708989e3c345b3a0bad8e5ecebc998 (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:28.484146  1722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35708989e3c345b3a0bad8e5ecebc998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.486946  1482 catalog_manager.cc:5697] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:28.487690  1775 raft_consensus.cc:359] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.488507  1775 raft_consensus.cc:385] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.488842  1775 raft_consensus.cc:740] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.489719  1775 consensus_queue.cc:260] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.445516  1717 tablet_service.cc:1511] Processing CreateTablet for tablet b689c1f7b39446488c097ffeb75f608c (DEFAULT_TABLE table=test-workload [id=e9535cd3eed14d769897d7960aaf847a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:28.491868  1717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b689c1f7b39446488c097ffeb75f608c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.494879  1721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e44a51657244bdac12c9e76cc53017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:28.498262  1800 raft_consensus.cc:493] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.498940  1800 raft_consensus.cc:515] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.503268  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.504663  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:28.506686  1697 leader_election.cc:343] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.508311  1800 leader_election.cc:290] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:28.509322  1697 leader_election.cc:343] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.510020  1697 leader_election.cc:304] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:28.511602  1800 raft_consensus.cc:2749] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.518180  1775 ts_tablet_manager.cc:1434] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.005s
I20260709 14:50:28.519642  1775 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.526605  1777 tablet_bootstrap.cc:654] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.533143  1774 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.533778  1774 ts_tablet_manager.cc:1403] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.204s	user 0.008s	sys 0.007s
I20260709 14:50:28.547560  1775 tablet_bootstrap.cc:654] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.553916  1777 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.554720  1777 ts_tablet_manager.cc:1403] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.006s
I20260709 14:50:28.538043  1774 raft_consensus.cc:359] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.555459  1774 raft_consensus.cc:385] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.555896  1774 raft_consensus.cc:740] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.557868  1774 consensus_queue.cc:260] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.558591  1777 raft_consensus.cc:359] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.560907  1777 raft_consensus.cc:385] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.561304  1777 raft_consensus.cc:740] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.562230  1777 consensus_queue.cc:260] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.564209  1774 ts_tablet_manager.cc:1434] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.006s
I20260709 14:50:28.565524  1774 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.573421  1777 ts_tablet_manager.cc:1434] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 14:50:28.574641  1786 raft_consensus.cc:493] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.575271  1786 raft_consensus.cc:515] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.579728  1775 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.580114  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.580876  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.581632  1660 raft_consensus.cc:2468] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
W20260709 14:50:28.582057  1549 leader_election.cc:343] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.580384  1775 ts_tablet_manager.cc:1403] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.005s
I20260709 14:50:28.582963  1786 leader_election.cc:290] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.583163  1548 leader_election.cc:304] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:28.584462  1786 raft_consensus.cc:2804] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.584862  1786 raft_consensus.cc:493] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.585270  1786 raft_consensus.cc:3060] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.587155  1775 raft_consensus.cc:359] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.588066  1775 raft_consensus.cc:385] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.588408  1775 raft_consensus.cc:740] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.574692  1777 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.590385  1774 tablet_bootstrap.cc:654] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.593410  1775 consensus_queue.cc:260] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.596246  1786 raft_consensus.cc:515] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.600875  1775 ts_tablet_manager.cc:1434] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 14:50:28.603472  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:28.603984  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:28.604583  1660 raft_consensus.cc:3060] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.605824  1779 raft_consensus.cc:493] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.606349  1779 raft_consensus.cc:515] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.608647  1779 leader_election.cc:290] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.610399  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.611480  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:28.612804  1623 leader_election.cc:343] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:28.613507  1624 leader_election.cc:343] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.614161  1624 leader_election.cc:304] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:28.616034  1779 raft_consensus.cc:2749] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.619684  1660 raft_consensus.cc:2468] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.621989  1548 leader_election.cc:304] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:28.625326  1777 tablet_bootstrap.cc:654] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.628886  1811 raft_consensus.cc:493] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.629511  1811 raft_consensus.cc:515] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:28.628938  1549 leader_election.cc:343] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.631320  1786 leader_election.cc:290] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.631899  1811 leader_election.cc:290] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.633167  1812 raft_consensus.cc:2804] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:28.635862  1812 raft_consensus.cc:697] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:28.602275  1775 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.636855  1812 consensus_queue.cc:237] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.634735  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.640509  1774 tablet_bootstrap.cc:492] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.641129  1774 ts_tablet_manager.cc:1403] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.076s	user 0.028s	sys 0.024s
I20260709 14:50:28.641184  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:28.644493  1548 leader_election.cc:343] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:28.645177  1549 leader_election.cc:343] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.645710  1549 leader_election.cc:304] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:28.646795  1775 tablet_bootstrap.cc:654] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.647516  1777 tablet_bootstrap.cc:492] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.647501  1811 raft_consensus.cc:2749] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.647173  1774 raft_consensus.cc:359] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.648345  1777 ts_tablet_manager.cc:1403] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.009s
I20260709 14:50:28.649471  1774 raft_consensus.cc:385] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.649891  1774 raft_consensus.cc:740] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.651019  1774 consensus_queue.cc:260] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.652931  1777 raft_consensus.cc:359] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.654124  1777 raft_consensus.cc:385] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.654618  1777 raft_consensus.cc:740] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.655829  1777 consensus_queue.cc:260] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.660403  1777 ts_tablet_manager.cc:1434] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 14:50:28.660463  1774 ts_tablet_manager.cc:1434] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 14:50:28.667302  1777 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.669493  1775 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.670161  1775 ts_tablet_manager.cc:1403] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.017s
I20260709 14:50:28.670115  1483 catalog_manager.cc:5697] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:28.673393  1774 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.674460  1775 raft_consensus.cc:359] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.675395  1775 raft_consensus.cc:385] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.675722  1775 raft_consensus.cc:740] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.676573  1777 tablet_bootstrap.cc:654] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.676581  1775 consensus_queue.cc:260] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.679941  1775 ts_tablet_manager.cc:1434] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 14:50:28.682457  1775 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.683962  1774 tablet_bootstrap.cc:654] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.691965  1775 tablet_bootstrap.cc:654] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.700475  1774 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.701927  1774 ts_tablet_manager.cc:1403] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.008s
I20260709 14:50:28.702804  1775 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.703496  1775 ts_tablet_manager.cc:1403] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 14:50:28.705476  1774 raft_consensus.cc:359] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.706513  1774 raft_consensus.cc:385] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.706883  1774 raft_consensus.cc:740] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.707517  1775 raft_consensus.cc:359] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.707923  1774 consensus_queue.cc:260] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.708452  1775 raft_consensus.cc:385] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.708914  1775 raft_consensus.cc:740] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.709918  1775 consensus_queue.cc:260] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.712671  1775 ts_tablet_manager.cc:1434] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 14:50:28.713896  1775 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.715035  1774 ts_tablet_manager.cc:1434] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 14:50:28.717113  1774 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.723327  1775 tablet_bootstrap.cc:654] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.725538  1774 tablet_bootstrap.cc:654] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.735720  1775 tablet_bootstrap.cc:492] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.736545  1777 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.736482  1775 ts_tablet_manager.cc:1403] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.009s
I20260709 14:50:28.737295  1777 ts_tablet_manager.cc:1403] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.070s	user 0.041s	sys 0.025s
I20260709 14:50:28.739054  1812 raft_consensus.cc:493] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.739847  1812 raft_consensus.cc:515] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.741381  1775 raft_consensus.cc:359] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.740965  1777 raft_consensus.cc:359] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.742448  1775 raft_consensus.cc:385] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.742707  1777 raft_consensus.cc:385] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.743031  1775 raft_consensus.cc:740] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.743259  1777 raft_consensus.cc:740] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.744233  1777 consensus_queue.cc:260] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.746665  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.747474  1660 raft_consensus.cc:2468] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.744673  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.748623  1777 ts_tablet_manager.cc:1434] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 14:50:28.748658  1734 raft_consensus.cc:2468] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.749131  1548 leader_election.cc:304] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:28.750100  1811 raft_consensus.cc:2804] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.750252  1777 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.750510  1811 raft_consensus.cc:493] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.744347  1775 consensus_queue.cc:260] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.750926  1811 raft_consensus.cc:3060] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.753113  1774 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.753664  1774 ts_tablet_manager.cc:1403] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.004s
I20260709 14:50:28.753705  1775 ts_tablet_manager.cc:1434] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 14:50:28.757257  1774 raft_consensus.cc:359] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.758194  1774 raft_consensus.cc:385] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.758533  1774 raft_consensus.cc:740] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.750893  1812 leader_election.cc:290] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.759985  1777 tablet_bootstrap.cc:654] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.763931  1775 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.765509  1812 raft_consensus.cc:493] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.766098  1812 raft_consensus.cc:515] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.768813  1786 raft_consensus.cc:493] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.769387  1786 raft_consensus.cc:515] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.766002  1811 raft_consensus.cc:515] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.771625  1786 leader_election.cc:290] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.759541  1774 consensus_queue.cc:260] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.774309  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.775209  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.775882  1660 raft_consensus.cc:2468] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.776681  1774 ts_tablet_manager.cc:1434] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 14:50:28.777429  1548 leader_election.cc:304] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:28.778851  1549 leader_election.cc:343] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.778681  1786 raft_consensus.cc:2804] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.780580  1786 raft_consensus.cc:493] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.781087  1786 raft_consensus.cc:3060] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.782954  1775 tablet_bootstrap.cc:654] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.787461  1777 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.784258  1774 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.787864  1779 raft_consensus.cc:493] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.788087  1777 ts_tablet_manager.cc:1403] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.024s
I20260709 14:50:28.788401  1779 raft_consensus.cc:515] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.791043  1779 leader_election.cc:290] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.793016  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:28.795176  1623 leader_election.cc:343] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.794976  1777 raft_consensus.cc:359] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.795812  1777 raft_consensus.cc:385] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.796137  1777 raft_consensus.cc:740] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.797900  1774 tablet_bootstrap.cc:654] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.797670  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.801620  1800 consensus_queue.cc:1048] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:50:28.804169  1624 leader_election.cc:343] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:28.804951  1624 leader_election.cc:304] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:28.808374  1775 tablet_bootstrap.cc:492] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.808902  1775 ts_tablet_manager.cc:1403] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.003s
I20260709 14:50:28.809449  1779 raft_consensus.cc:2749] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.812426  1775 raft_consensus.cc:359] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.813155  1775 raft_consensus.cc:385] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.813447  1775 raft_consensus.cc:740] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.814136  1775 consensus_queue.cc:260] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.816638  1775 ts_tablet_manager.cc:1434] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 14:50:28.817785  1775 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.817831  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:28.818495  1734 raft_consensus.cc:3060] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.818208  1777 consensus_queue.cc:260] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.821435  1812 leader_election.cc:290] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.823952  1774 tablet_bootstrap.cc:492] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.824852  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550babf7e3344307bf7f14bae6eb5b86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.825103  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550babf7e3344307bf7f14bae6eb5b86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.825696  1733 raft_consensus.cc:2468] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.826133  1659 raft_consensus.cc:2468] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.827248  1549 leader_election.cc:304] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:28.828600  1774 ts_tablet_manager.cc:1403] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.006s
I20260709 14:50:28.828501  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1298fa6acd614e99b7cdb2fc4adbf5b9" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:28.829589  1659 raft_consensus.cc:3060] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.833689  1812 raft_consensus.cc:2804] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.834125  1812 raft_consensus.cc:493] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.834484  1812 raft_consensus.cc:3060] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.835201  1734 raft_consensus.cc:2468] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.838625  1549 leader_election.cc:304] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
W20260709 14:50:28.839221  1697 consensus_peers.cc:597] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:28.832018  1774 raft_consensus.cc:359] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.840107  1774 raft_consensus.cc:385] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.840355  1800 raft_consensus.cc:493] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.840982  1800 raft_consensus.cc:515] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.844038  1800 leader_election.cc:290] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:28.840459  1774 raft_consensus.cc:740] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.845575  1774 consensus_queue.cc:260] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.846614  1775 tablet_bootstrap.cc:654] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.846369  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.851706  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438a8ba60c454c089cab93412c5c93a1" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.852555  1774 ts_tablet_manager.cc:1434] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 14:50:28.853148  1658 raft_consensus.cc:2393] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:28.853878  1774 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.855104  1812 raft_consensus.cc:515] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.858556  1697 leader_election.cc:304] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:28.859123  1812 leader_election.cc:290] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.860682  1812 raft_consensus.cc:2804] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:28.861258  1812 raft_consensus.cc:697] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:28.861501  1800 raft_consensus.cc:3060] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.862254  1812 consensus_queue.cc:237] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.863303  1774 tablet_bootstrap.cc:654] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.865599  1811 leader_election.cc:290] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.866297  1811 raft_consensus.cc:493] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.866777  1811 raft_consensus.cc:515] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.867897  1797 raft_consensus.cc:493] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.868652  1797 raft_consensus.cc:515] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.869396  1811 leader_election.cc:290] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.871464  1797 leader_election.cc:290] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:28.877095  1786 raft_consensus.cc:515] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.883206  1659 raft_consensus.cc:2468] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.884981  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:28.886116  1585 raft_consensus.cc:2393] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:28.888432  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.888936  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.889209  1659 raft_consensus.cc:2468] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:28.889629  1733 raft_consensus.cc:2468] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.891008  1697 leader_election.cc:304] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:28.891268  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550babf7e3344307bf7f14bae6eb5b86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:28.892194  1734 raft_consensus.cc:3060] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.892740  1800 raft_consensus.cc:2749] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:28.895190  1797 raft_consensus.cc:2804] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.895831  1797 raft_consensus.cc:493] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.896275  1797 raft_consensus.cc:3060] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.903262  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550babf7e3344307bf7f14bae6eb5b86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:28.904074  1659 raft_consensus.cc:3060] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.910460  1786 leader_election.cc:290] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.913575  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:28.914538  1658 raft_consensus.cc:3060] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.917876  1777 ts_tablet_manager.cc:1434] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.129s	user 0.028s	sys 0.031s
I20260709 14:50:28.919539  1734 raft_consensus.cc:2468] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.919996  1777 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.919528  1797 raft_consensus.cc:515] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.921684  1797 leader_election.cc:290] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:28.923480  1775 tablet_bootstrap.cc:492] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:28.924077  1775 ts_tablet_manager.cc:1403] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.107s	user 0.030s	sys 0.020s
I20260709 14:50:28.928248  1775 raft_consensus.cc:359] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.929093  1775 raft_consensus.cc:385] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.929455  1775 raft_consensus.cc:740] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.930469  1775 consensus_queue.cc:260] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:28.932425  1774 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:28.933053  1774 ts_tablet_manager.cc:1403] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.079s	user 0.029s	sys 0.008s
I20260709 14:50:28.932871  1479 catalog_manager.cc:5697] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:28.934042  1777 tablet_bootstrap.cc:654] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.936205  1774 raft_consensus.cc:359] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.943377  1774 raft_consensus.cc:385] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.943920  1774 raft_consensus.cc:740] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.945246  1658 raft_consensus.cc:2468] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.939805  1775 ts_tablet_manager.cc:1434] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 14:50:28.946708  1775 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:28.946138  1774 consensus_queue.cc:260] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.949559  1774 ts_tablet_manager.cc:1434] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 14:50:28.951030  1774 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:28.942514  1659 raft_consensus.cc:2468] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:28.937418  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:28.953910  1548 leader_election.cc:304] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:28.956243  1826 raft_consensus.cc:2804] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:28.956416  1548 leader_election.cc:304] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:28.960419  1774 tablet_bootstrap.cc:654] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.957159  1826 raft_consensus.cc:697] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:28.967501  1660 raft_consensus.cc:2468] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:28.968019  1786 raft_consensus.cc:2804] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:28.973241  1548 leader_election.cc:304] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:28.973423  1786 raft_consensus.cc:697] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:28.974483  1812 raft_consensus.cc:2804] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:28.974968  1812 raft_consensus.cc:493] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:28.975365  1812 raft_consensus.cc:3060] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:28.975888  1826 consensus_queue.cc:237] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.978618  1777 tablet_bootstrap.cc:492] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:28.979209  1777 ts_tablet_manager.cc:1403] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.017s
I20260709 14:50:28.984086  1775 tablet_bootstrap.cc:654] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:28.974987  1786 consensus_queue.cc:237] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.988660  1777 raft_consensus.cc:359] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:28.966674  1697 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 75752.8us
I20260709 14:50:28.991091  1777 raft_consensus.cc:385] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:28.990414  1779 raft_consensus.cc:493] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:28.992209  1777 raft_consensus.cc:740] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:28.992929  1779 raft_consensus.cc:515] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.993283  1777 consensus_queue.cc:260] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:28.995302  1779 leader_election.cc:290] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:28.996554  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:28.997891  1777 ts_tablet_manager.cc:1434] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 14:50:28.998976  1777 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:28.997265  1734 raft_consensus.cc:2468] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:28.999221  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.001089  1624 leader_election.cc:304] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:29.002866  1623 leader_election.cc:343] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.004150  1812 raft_consensus.cc:515] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:29.007155  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:29.008596  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.010098  1658 raft_consensus.cc:2393] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:29.010977  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.012583  1659 raft_consensus.cc:3060] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.012902  1774 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.013054  1779 raft_consensus.cc:2804] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.013422  1774 ts_tablet_manager.cc:1403] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.018s
W20260709 14:50:29.012049  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:29.013424  1779 raft_consensus.cc:493] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.015326  1779 raft_consensus.cc:3060] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.019462  1777 tablet_bootstrap.cc:654] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.007233  1812 leader_election.cc:290] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.023916  1774 raft_consensus.cc:359] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.026466  1774 raft_consensus.cc:385] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.027064  1774 raft_consensus.cc:740] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.030017  1774 consensus_queue.cc:260] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.033474  1774 ts_tablet_manager.cc:1434] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 14:50:29.037329  1774 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.043133  1812 consensus_queue.cc:1048] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.044718  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b844b9b6088d424d8cd83fa504010c9c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.045562  1734 raft_consensus.cc:2393] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:29.038302  1659 raft_consensus.cc:2468] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:29.047585  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c784884b0ef3459b95c5a9028bed9e61" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.048205  1734 raft_consensus.cc:3060] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.049741  1548 leader_election.cc:304] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:29.050689  1826 raft_consensus.cc:2804] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.051373  1826 raft_consensus.cc:697] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:29.052371  1826 consensus_queue.cc:237] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:29.060444  1779 raft_consensus.cc:515] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.065696  1697 leader_election.cc:304] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.066227  1734 raft_consensus.cc:2468] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:29.073725  1775 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.074467  1831 raft_consensus.cc:2749] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.075670  1775 ts_tablet_manager.cc:1403] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.129s	user 0.028s	sys 0.007s
I20260709 14:50:29.077553  1774 tablet_bootstrap.cc:654] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.081789  1831 raft_consensus.cc:493] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.082916  1831 raft_consensus.cc:515] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.085255  1831 leader_election.cc:290] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.084810  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:29.094774  1779 leader_election.cc:290] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.097512  1777 tablet_bootstrap.cc:492] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.098310  1777 ts_tablet_manager.cc:1403] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.100s	user 0.030s	sys 0.035s
I20260709 14:50:29.099448  1775 raft_consensus.cc:359] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.100620  1775 raft_consensus.cc:385] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.101032  1775 raft_consensus.cc:740] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
W20260709 14:50:29.102152  1623 leader_election.cc:343] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.103850  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.104727  1734 raft_consensus.cc:3060] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.102147  1775 consensus_queue.cc:260] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.108304  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.109084  1660 raft_consensus.cc:2468] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:29.110173  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.112212  1697 leader_election.cc:304] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:29.114457  1831 raft_consensus.cc:2804] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.114895  1831 raft_consensus.cc:493] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.115242  1831 raft_consensus.cc:3060] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.115681  1777 raft_consensus.cc:359] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.116838  1777 raft_consensus.cc:385] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.117256  1777 raft_consensus.cc:740] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.108480  1775 ts_tablet_manager.cc:1434] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.004s
I20260709 14:50:29.118772  1777 consensus_queue.cc:260] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.126132  1779 raft_consensus.cc:493] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:29.126873  1697 leader_election.cc:343] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.113763  1774 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.128417  1774 ts_tablet_manager.cc:1403] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.017s
I20260709 14:50:29.128541  1777 ts_tablet_manager.cc:1434] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.001s
I20260709 14:50:29.130314  1777 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.124267  1482 catalog_manager.cc:5697] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:29.125888  1831 raft_consensus.cc:515] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.134961  1779 raft_consensus.cc:515] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.134457  1482 catalog_manager.cc:5697] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:29.137936  1482 catalog_manager.cc:5697] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:29.134547  1774 raft_consensus.cc:359] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.138396  1831 leader_election.cc:290] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.139158  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:29.138967  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:29.144059  1697 leader_election.cc:343] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.144395  1585 raft_consensus.cc:2468] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:29.147040  1623 leader_election.cc:304] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:29.140816  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.151859  1659 raft_consensus.cc:3060] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.153545  1824 raft_consensus.cc:2804] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.154084  1824 raft_consensus.cc:493] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.154498  1824 raft_consensus.cc:3060] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.142196  1774 raft_consensus.cc:385] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.121183  1775 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.139577  1779 leader_election.cc:290] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.141801  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.166380  1734 raft_consensus.cc:2468] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:29.166584  1733 raft_consensus.cc:2468] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:29.159718  1774 raft_consensus.cc:740] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.172194  1774 consensus_queue.cc:260] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.178820  1831 raft_consensus.cc:493] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.179656  1831 raft_consensus.cc:515] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.182024  1624 leader_election.cc:304] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:29.182726  1831 leader_election.cc:290] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.186985  1826 consensus_queue.cc:1048] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.187819  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.190193  1659 raft_consensus.cc:2468] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:29.186909  1775 tablet_bootstrap.cc:654] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:29.190781  1697 leader_election.cc:343] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.193802  1774 ts_tablet_manager.cc:1434] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.001s
I20260709 14:50:29.184257  1824 raft_consensus.cc:515] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.185417  1779 raft_consensus.cc:2804] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.195209  1774 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.197234  1824 leader_election.cc:290] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.198006  1697 leader_election.cc:304] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:29.199340  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.200412  1831 raft_consensus.cc:2804] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.205178  1734 raft_consensus.cc:3060] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.205312  1831 raft_consensus.cc:697] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:29.206279  1831 consensus_queue.cc:237] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.210963  1779 raft_consensus.cc:697] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:29.211424  1777 tablet_bootstrap.cc:654] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.212304  1779 consensus_queue.cc:237] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.215543  1774 tablet_bootstrap.cc:654] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.221460  1734 raft_consensus.cc:2468] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:29.203420  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73244150135b4bb5aeeba3dc4b94fab7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:29.262710  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.267977  1585 raft_consensus.cc:3060] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.277045  1624 leader_election.cc:304] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:29.280234  1697 leader_election.cc:343] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.283609  1697 leader_election.cc:304] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.228264  1775 tablet_bootstrap.cc:492] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.286370  1775 ts_tablet_manager.cc:1403] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.165s	user 0.013s	sys 0.010s
I20260709 14:50:29.287250  1826 raft_consensus.cc:493] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.288166  1826 raft_consensus.cc:515] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.289430  1797 raft_consensus.cc:2749] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.298681  1777 tablet_bootstrap.cc:492] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.298992  1584 raft_consensus.cc:3060] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.302445  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.302237  1827 consensus_queue.cc:1048] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.300997  1826 leader_election.cc:290] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.305202  1775 raft_consensus.cc:359] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.306245  1775 raft_consensus.cc:385] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.302893  1774 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.293848  1779 raft_consensus.cc:2804] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.306655  1775 raft_consensus.cc:740] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.302276  1777 ts_tablet_manager.cc:1403] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.172s	user 0.021s	sys 0.001s
I20260709 14:50:29.307549  1779 raft_consensus.cc:697] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:29.312381  1779 consensus_queue.cc:237] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.313632  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe3cb63a07e4ce6b38156b136569cca" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.314703  1733 raft_consensus.cc:2393] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:29.317128  1774 ts_tablet_manager.cc:1403] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.006s
I20260709 14:50:29.320179  1549 leader_election.cc:304] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.321579  1826 raft_consensus.cc:3060] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.289374  1482 catalog_manager.cc:5697] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:29.319191  1775 consensus_queue.cc:260] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.328938  1774 raft_consensus.cc:359] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.329919  1774 raft_consensus.cc:385] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.330482  1774 raft_consensus.cc:740] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.330030  1777 raft_consensus.cc:359] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.331032  1777 raft_consensus.cc:385] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.331406  1777 raft_consensus.cc:740] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.332199  1777 consensus_queue.cc:260] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.334074  1826 raft_consensus.cc:2749] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.336416  1777 ts_tablet_manager.cc:1434] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 14:50:29.338234  1777 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.337881  1786 consensus_queue.cc:1048] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.331715  1774 consensus_queue.cc:260] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.345744  1585 raft_consensus.cc:2468] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:29.353897  1775 ts_tablet_manager.cc:1434] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.067s	user 0.011s	sys 0.001s
I20260709 14:50:29.354914  1774 ts_tablet_manager.cc:1434] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.004s
I20260709 14:50:29.355679  1775 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.356571  1774 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.364646  1777 tablet_bootstrap.cc:654] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.386075  1479 catalog_manager.cc:5697] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:29.392416  1482 catalog_manager.cc:5697] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:29.393536  1777 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.394094  1777 ts_tablet_manager.cc:1403] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.006s
I20260709 14:50:29.397696  1774 tablet_bootstrap.cc:654] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.404486  1777 raft_consensus.cc:359] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.405416  1777 raft_consensus.cc:385] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.405752  1777 raft_consensus.cc:740] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.406630  1777 consensus_queue.cc:260] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.408326  1774 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.408816  1774 ts_tablet_manager.cc:1403] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.004s
I20260709 14:50:29.409837  1777 ts_tablet_manager.cc:1434] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 14:50:29.411103  1777 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.411881  1774 raft_consensus.cc:359] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.412648  1774 raft_consensus.cc:385] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.412585  1826 raft_consensus.cc:493] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.413254  1826 raft_consensus.cc:515] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.415616  1826 leader_election.cc:290] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.416635  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.417639  1659 raft_consensus.cc:2393] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:29.420030  1777 tablet_bootstrap.cc:654] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.422674  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc44ef62c96a40118ff58c994395ed5a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.425386  1549 leader_election.cc:304] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.427255  1826 raft_consensus.cc:3060] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.437966  1777 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.437966  1826 raft_consensus.cc:2749] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.438862  1777 ts_tablet_manager.cc:1403] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.000s
I20260709 14:50:29.439023  1775 tablet_bootstrap.cc:654] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.442121  1774 raft_consensus.cc:740] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.442554  1777 raft_consensus.cc:359] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.443369  1777 raft_consensus.cc:385] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.443094  1774 consensus_queue.cc:260] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.443632  1777 raft_consensus.cc:740] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.444507  1777 consensus_queue.cc:260] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.445230  1831 raft_consensus.cc:493] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.446171  1774 ts_tablet_manager.cc:1434] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.037s	user 0.001s	sys 0.006s
I20260709 14:50:29.445972  1831 raft_consensus.cc:515] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.448097  1774 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.448661  1831 leader_election.cc:290] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.447024  1777 ts_tablet_manager.cc:1434] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:50:29.451997  1777 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.452356  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.453195  1659 raft_consensus.cc:2468] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:29.455808  1774 tablet_bootstrap.cc:654] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.459549  1777 tablet_bootstrap.cc:654] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.463162  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.463999  1584 raft_consensus.cc:2468] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:29.466449  1697 leader_election.cc:304] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:29.468267  1797 raft_consensus.cc:2804] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.468698  1797 raft_consensus.cc:493] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.469050  1797 raft_consensus.cc:3060] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.476568  1827 consensus_queue.cc:1048] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.478916  1827 consensus_queue.cc:1048] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.493474  1797 raft_consensus.cc:515] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.496327  1775 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.496908  1775 ts_tablet_manager.cc:1403] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.142s	user 0.008s	sys 0.007s
I20260709 14:50:29.502385  1774 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.504015  1774 ts_tablet_manager.cc:1403] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.009s
I20260709 14:50:29.504858  1775 raft_consensus.cc:359] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.505769  1775 raft_consensus.cc:385] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.506013  1775 raft_consensus.cc:740] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.507777  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:29.507709  1786 consensus_queue.cc:1048] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.508519  1584 raft_consensus.cc:3060] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.520305  1775 consensus_queue.cc:260] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.520788  1774 raft_consensus.cc:359] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.521667  1774 raft_consensus.cc:385] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.521992  1774 raft_consensus.cc:740] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.525843  1775 ts_tablet_manager.cc:1434] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 14:50:29.528086  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea7a74d9aa14ef38365e872e3efb562" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.528990  1774 consensus_queue.cc:260] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.529881  1659 raft_consensus.cc:3060] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.529852  1827 consensus_queue.cc:1048] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.523422  1797 leader_election.cc:290] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.544363  1774 ts_tablet_manager.cc:1434] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.001s
I20260709 14:50:29.546193  1774 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.546968  1827 consensus_queue.cc:1048] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.555658  1777 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.556309  1777 ts_tablet_manager.cc:1403] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.105s	user 0.041s	sys 0.032s
I20260709 14:50:29.559067  1775 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.564414  1584 raft_consensus.cc:2468] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:29.569692  1697 leader_election.cc:304] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:29.571164  1797 raft_consensus.cc:2804] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.571918  1797 raft_consensus.cc:697] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:29.577381  1777 raft_consensus.cc:359] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.577968  1777 raft_consensus.cc:385] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.578215  1777 raft_consensus.cc:740] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.579336  1777 consensus_queue.cc:260] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.580413  1827 raft_consensus.cc:493] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.581067  1827 raft_consensus.cc:515] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.581760  1777 ts_tablet_manager.cc:1434] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 14:50:29.583076  1777 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.576192  1797 consensus_queue.cc:237] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.585230  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.585587  1827 leader_election.cc:290] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.585278  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:29.589514  1549 leader_election.cc:343] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:29.591326  1548 leader_election.cc:343] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.591888  1548 leader_election.cc:304] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.599272  1865 raft_consensus.cc:2749] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.582729  1659 raft_consensus.cc:2468] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:29.614283  1831 consensus_queue.cc:1048] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.617100  1812 consensus_queue.cc:1048] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 14:50:29.619056  1777 tablet_bootstrap.cc:654] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.624893  1775 tablet_bootstrap.cc:654] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.630362  1774 tablet_bootstrap.cc:654] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.647769  1797 consensus_queue.cc:1048] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.650440  1797 raft_consensus.cc:493] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.651419  1797 raft_consensus.cc:515] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.651769  1779 consensus_queue.cc:1048] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:29.653831  1797 leader_election.cc:290] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.644019  1481 catalog_manager.cc:5697] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:29.661337  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:29.664810  1697 leader_election.cc:343] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.670724  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:29.672684  1697 leader_election.cc:343] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.673189  1697 leader_election.cc:304] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.676265  1797 raft_consensus.cc:2749] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.682786  1777 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.683243  1774 tablet_bootstrap.cc:492] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:29.683463  1777 ts_tablet_manager.cc:1403] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.101s	user 0.019s	sys 0.033s
I20260709 14:50:29.684051  1774 ts_tablet_manager.cc:1403] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.004s
I20260709 14:50:29.685153  1775 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.685851  1775 ts_tablet_manager.cc:1403] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.023s
I20260709 14:50:29.687140  1777 raft_consensus.cc:359] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.687924  1777 raft_consensus.cc:385] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.688225  1777 raft_consensus.cc:740] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.688164  1774 raft_consensus.cc:359] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.689373  1774 raft_consensus.cc:385] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.689332  1777 consensus_queue.cc:260] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.689896  1774 raft_consensus.cc:740] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.693269  1777 ts_tablet_manager.cc:1434] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:50:29.693960  1775 raft_consensus.cc:359] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.694504  1774 consensus_queue.cc:260] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.694949  1775 raft_consensus.cc:385] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.694463  1777 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.695511  1775 raft_consensus.cc:740] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.701915  1775 consensus_queue.cc:260] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.711419  1775 ts_tablet_manager.cc:1434] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 14:50:29.712793  1775 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.714008  1777 tablet_bootstrap.cc:654] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.720739  1774 ts_tablet_manager.cc:1434] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 14:50:29.728053  1779 consensus_queue.cc:1048] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.729923  1774 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.733662  1779 consensus_queue.cc:1048] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.755374  1797 raft_consensus.cc:493] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.756031  1797 raft_consensus.cc:515] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.760149  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.762341  1865 raft_consensus.cc:493] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.763010  1865 raft_consensus.cc:515] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.765798  1865 leader_election.cc:290] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.767995  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.769011  1659 raft_consensus.cc:2468] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:29.771157  1548 leader_election.cc:304] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:29.772728  1697 leader_election.cc:343] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.773691  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.775373  1865 raft_consensus.cc:2804] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.778275  1732 raft_consensus.cc:2468] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:29.780375  1797 leader_election.cc:290] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.779385  1865 raft_consensus.cc:493] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.783589  1865 raft_consensus.cc:3060] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.783931  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.784825  1660 raft_consensus.cc:2468] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:29.787139  1800 raft_consensus.cc:493] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.787832  1775 tablet_bootstrap.cc:654] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.787761  1800 raft_consensus.cc:515] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.790005  1697 leader_election.cc:304] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:29.790648  1800 leader_election.cc:290] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.791277  1800 raft_consensus.cc:2804] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.791638  1800 raft_consensus.cc:493] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.791977  1800 raft_consensus.cc:3060] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.792629  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.794555  1777 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.792619  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:29.795645  1697 leader_election.cc:343] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:29.796953  1697 leader_election.cc:343] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.797125  1777 ts_tablet_manager.cc:1403] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.103s	user 0.031s	sys 0.035s
I20260709 14:50:29.797464  1697 leader_election.cc:304] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.803042  1797 raft_consensus.cc:2749] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.802748  1777 raft_consensus.cc:359] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.803771  1777 raft_consensus.cc:385] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.803710  1845 consensus_queue.cc:1048] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.804236  1777 raft_consensus.cc:740] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.804045  1865 raft_consensus.cc:515] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.805258  1777 consensus_queue.cc:260] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.808274  1777 ts_tablet_manager.cc:1434] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 14:50:29.815793  1775 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.816437  1775 ts_tablet_manager.cc:1403] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.003s
I20260709 14:50:29.823904  1774 tablet_bootstrap.cc:654] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.825798  1777 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.826526  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.827232  1660 raft_consensus.cc:3060] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.830945  1775 raft_consensus.cc:359] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.834241  1775 raft_consensus.cc:385] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.834597  1775 raft_consensus.cc:740] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.829316  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c69d6d1223974de892e79da6d6c27d86" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.835634  1733 raft_consensus.cc:3060] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.835634  1775 consensus_queue.cc:260] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.838575  1775 ts_tablet_manager.cc:1434] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 14:50:29.839733  1775 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.840461  1777 tablet_bootstrap.cc:654] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.829535  1865 leader_election.cc:290] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.856887  1733 raft_consensus.cc:2468] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:29.858527  1549 leader_election.cc:304] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:29.859704  1865 raft_consensus.cc:2804] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.860306  1865 raft_consensus.cc:697] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:29.865980  1660 raft_consensus.cc:2468] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:29.866345  1775 tablet_bootstrap.cc:654] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:29.861284  1865 consensus_queue.cc:237] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.882516  1826 raft_consensus.cc:493] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.882516  1827 raft_consensus.cc:493] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.883734  1826 raft_consensus.cc:515] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.883876  1827 raft_consensus.cc:515] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.888314  1826 leader_election.cc:290] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.888671  1827 leader_election.cc:290] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.885612  1779 raft_consensus.cc:493] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:29.889851  1779 raft_consensus.cc:515] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.891309  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:29.892076  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.892360  1800 raft_consensus.cc:515] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.892704  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:29.894977  1549 leader_election.cc:343] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:29.895551  1548 leader_election.cc:343] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.895759  1800 leader_election.cc:290] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:29.896067  1548 leader_election.cc:304] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.897359  1827 raft_consensus.cc:2749] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.897136  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:29.899425  1697 leader_election.cc:343] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.900804  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:29.901125  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.901541  1659 raft_consensus.cc:3060] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.902014  1734 raft_consensus.cc:2468] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:29.902954  1779 leader_election.cc:290] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.912132  1549 leader_election.cc:304] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:29.914688  1827 raft_consensus.cc:2804] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:29.914819  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:29.915242  1827 raft_consensus.cc:493] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:29.915639  1827 raft_consensus.cc:3060] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:29.920586  1548 leader_election.cc:343] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.921672  1774 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
W20260709 14:50:29.922413  1624 leader_election.cc:343] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.923525  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:29.924368  1774 ts_tablet_manager.cc:1403] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.195s	user 0.026s	sys 0.015s
W20260709 14:50:29.925791  1623 leader_election.cc:343] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.926321  1623 leader_election.cc:304] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:29.926887  1659 raft_consensus.cc:2468] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:29.927736  1845 raft_consensus.cc:2749] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:29.928074  1777 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:29.928577  1777 ts_tablet_manager.cc:1403] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.001s
I20260709 14:50:29.929141  1697 leader_election.cc:304] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:29.930047  1800 raft_consensus.cc:2804] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.930516  1800 raft_consensus.cc:697] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:29.930719  1774 raft_consensus.cc:359] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.930986  1827 raft_consensus.cc:515] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.931473  1774 raft_consensus.cc:385] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.931638  1777 raft_consensus.cc:359] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.931782  1800 consensus_queue.cc:237] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.932526  1777 raft_consensus.cc:385] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.932907  1777 raft_consensus.cc:740] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.931805  1774 raft_consensus.cc:740] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.933753  1777 consensus_queue.cc:260] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.925956  1775 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:29.934303  1774 consensus_queue.cc:260] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.935030  1775 ts_tablet_manager.cc:1403] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.096s	user 0.007s	sys 0.011s
I20260709 14:50:29.938566  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
W20260709 14:50:29.940629  1548 leader_election.cc:343] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:29.944190  1775 raft_consensus.cc:359] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.945127  1775 raft_consensus.cc:385] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:29.945461  1775 raft_consensus.cc:740] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:29.945791  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2550480abe0c4d12b8c11bf82ec5039b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:29.946303  1775 consensus_queue.cc:260] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.946816  1733 raft_consensus.cc:3060] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:29.948518  1481 catalog_manager.cc:5697] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:29.949640  1775 ts_tablet_manager.cc:1434] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 14:50:29.950812  1775 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:29.959262  1827 leader_election.cc:290] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:29.958951  1774 ts_tablet_manager.cc:1434] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 14:50:29.960772  1733 raft_consensus.cc:2468] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:29.961304  1774 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:29.962608  1549 leader_election.cc:304] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:29.972658  1777 ts_tablet_manager.cc:1434] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 14:50:29.973970  1777 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:29.972420  1483 catalog_manager.cc:5697] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:29.975445  1826 raft_consensus.cc:2804] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:29.975999  1826 raft_consensus.cc:697] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:29.976164  1800 consensus_queue.cc:1048] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:29.976943  1826 consensus_queue.cc:237] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:29.983369  1774 tablet_bootstrap.cc:654] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.010426  1777 tablet_bootstrap.cc:654] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.018959  1775 tablet_bootstrap.cc:654] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.024278  1797 consensus_queue.cc:1048] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.030504  1777 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.031198  1777 ts_tablet_manager.cc:1403] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.003s
I20260709 14:50:30.012557  1483 catalog_manager.cc:5697] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:30.032094  1775 tablet_bootstrap.cc:492] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.032647  1775 ts_tablet_manager.cc:1403] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.007s
I20260709 14:50:30.034365  1777 raft_consensus.cc:359] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.035161  1777 raft_consensus.cc:385] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.035475  1777 raft_consensus.cc:740] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.036260  1777 consensus_queue.cc:260] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.037810  1775 raft_consensus.cc:359] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.039160  1777 ts_tablet_manager.cc:1434] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:50:30.041389  1775 raft_consensus.cc:385] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.041894  1775 raft_consensus.cc:740] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
W20260709 14:50:30.050191  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:30.052335  1775 consensus_queue.cc:260] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.054553  1774 tablet_bootstrap.cc:492] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.055197  1774 ts_tablet_manager.cc:1403] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.012s
I20260709 14:50:30.056496  1777 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:30.054987  1775 ts_tablet_manager.cc:1434] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 14:50:30.061451  1774 raft_consensus.cc:359] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.062139  1774 raft_consensus.cc:385] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.062400  1774 raft_consensus.cc:740] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.063130  1774 consensus_queue.cc:260] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.065289  1774 ts_tablet_manager.cc:1434] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 14:50:30.070814  1775 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.073698  1777 tablet_bootstrap.cc:654] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.079140  1775 tablet_bootstrap.cc:654] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.079895  1826 raft_consensus.cc:493] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.080449  1826 raft_consensus.cc:515] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.083446  1774 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:30.084314  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.085049  1733 raft_consensus.cc:2468] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:30.084314  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.086737  1660 raft_consensus.cc:2468] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:30.087417  1845 raft_consensus.cc:493] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.088299  1548 leader_election.cc:304] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.088152  1845 raft_consensus.cc:515] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.089087  1826 leader_election.cc:290] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.090009  1827 raft_consensus.cc:2804] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.090407  1827 raft_consensus.cc:493] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.090778  1827 raft_consensus.cc:3060] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.091385  1845 leader_election.cc:290] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.094798  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.098089  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.099216  1733 raft_consensus.cc:2468] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:30.101116  1624 leader_election.cc:304] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.102280  1779 raft_consensus.cc:493] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.103012  1779 raft_consensus.cc:515] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.102308  1827 raft_consensus.cc:515] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:30.104338  1623 leader_election.cc:343] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.105868  1845 raft_consensus.cc:2804] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.106331  1845 raft_consensus.cc:493] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.106709  1779 leader_election.cc:290] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.106750  1827 leader_election.cc:290] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.116660  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:30.117655  1660 raft_consensus.cc:3060] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.106739  1845 raft_consensus.cc:3060] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.125420  1777 tablet_bootstrap.cc:492] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.126060  1777 ts_tablet_manager.cc:1403] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.070s	user 0.030s	sys 0.008s
I20260709 14:50:30.127059  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08aaabb4b40d415dacd518d43a09869a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:30.127777  1733 raft_consensus.cc:3060] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.129277  1777 raft_consensus.cc:359] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.129917  1777 raft_consensus.cc:385] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.130091  1777 raft_consensus.cc:740] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.130775  1777 consensus_queue.cc:260] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.135406  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.137197  1777 ts_tablet_manager.cc:1434] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 14:50:30.138576  1777 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
W20260709 14:50:30.144846  1624 leader_election.cc:343] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.149761  1831 raft_consensus.cc:493] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.150441  1831 raft_consensus.cc:515] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.152328  1779 raft_consensus.cc:493] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.152827  1831 leader_election.cc:290] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:30.152902  1779 raft_consensus.cc:515] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.155546  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.156286  1585 raft_consensus.cc:2468] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:30.156840  1779 leader_election.cc:290] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.156745  1660 raft_consensus.cc:2468] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:30.146605  1845 raft_consensus.cc:515] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.158995  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:30.160398  1623 leader_election.cc:343] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.160913  1845 leader_election.cc:290] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.161764  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.162032  1697 leader_election.cc:304] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:30.162220  1548 leader_election.cc:304] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.162163  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:30.163236  1774 tablet_bootstrap.cc:654] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.162510  1660 raft_consensus.cc:2468] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:30.163717  1865 raft_consensus.cc:2804] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.163763  1585 raft_consensus.cc:3060] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.165760  1865 raft_consensus.cc:697] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:30.165340  1797 raft_consensus.cc:2804] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.167858  1797 raft_consensus.cc:493] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.167891  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.167747  1865 consensus_queue.cc:237] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.168851  1797 raft_consensus.cc:3060] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.169433  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4987548a10d4e82b06659b89058e65c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:30.170126  1732 raft_consensus.cc:3060] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.167872  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:30.171398  1623 leader_election.cc:343] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.171859  1623 leader_election.cc:304] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
W20260709 14:50:30.173909  1624 leader_election.cc:343] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.174484  1624 leader_election.cc:304] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:30.175581  1779 raft_consensus.cc:2749] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.176259  1779 raft_consensus.cc:2749] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.176389  1733 raft_consensus.cc:2468] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:30.182143  1775 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.187574  1775 ts_tablet_manager.cc:1403] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.117s	user 0.033s	sys 0.020s
I20260709 14:50:30.190286  1775 raft_consensus.cc:359] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.186197  1732 raft_consensus.cc:2468] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:30.190943  1775 raft_consensus.cc:385] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.190578  1774 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.191282  1775 raft_consensus.cc:740] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.191731  1774 ts_tablet_manager.cc:1403] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.109s	user 0.013s	sys 0.009s
I20260709 14:50:30.192165  1624 leader_election.cc:304] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.192081  1775 consensus_queue.cc:260] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.193048  1845 raft_consensus.cc:2804] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.193699  1845 raft_consensus.cc:697] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:30.194422  1775 ts_tablet_manager.cc:1434] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 14:50:30.183424  1777 tablet_bootstrap.cc:654] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.194554  1845 consensus_queue.cc:237] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.195669  1775 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.197396  1797 raft_consensus.cc:515] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.201637  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:30.202309  1584 raft_consensus.cc:3060] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.203590  1775 tablet_bootstrap.cc:654] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.207993  1774 raft_consensus.cc:359] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.209004  1774 raft_consensus.cc:385] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.209331  1774 raft_consensus.cc:740] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.210108  1774 consensus_queue.cc:260] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.204780  1585 raft_consensus.cc:2468] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:30.212167  1775 tablet_bootstrap.cc:492] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.212491  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422982f0dc6b4d3ca2fdaf28d0411514" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:30.212735  1775 ts_tablet_manager.cc:1403] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 14:50:30.213138  1660 raft_consensus.cc:3060] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.216220  1774 ts_tablet_manager.cc:1434] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 14:50:30.216140  1775 raft_consensus.cc:359] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.217074  1775 raft_consensus.cc:385] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.217495  1774 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:30.217401  1775 raft_consensus.cc:740] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.218580  1775 consensus_queue.cc:260] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.219475  1483 catalog_manager.cc:5697] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:30.222148  1797 leader_election.cc:290] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:30.228615  1775 ts_tablet_manager.cc:1434] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 14:50:30.229691  1660 raft_consensus.cc:2468] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:30.230579  1775 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.231304  1697 leader_election.cc:304] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.232266  1797 raft_consensus.cc:2804] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.235423  1584 raft_consensus.cc:2468] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:30.236264  1481 catalog_manager.cc:5697] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:30.237599  1797 raft_consensus.cc:697] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:30.225826  1774 tablet_bootstrap.cc:654] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.238670  1797 consensus_queue.cc:237] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.250468  1775 tablet_bootstrap.cc:654] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.260385  1774 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.260598  1777 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.260926  1774 ts_tablet_manager.cc:1403] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.006s
I20260709 14:50:30.261092  1777 ts_tablet_manager.cc:1403] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.123s	user 0.007s	sys 0.008s
I20260709 14:50:30.264001  1777 raft_consensus.cc:359] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.264861  1777 raft_consensus.cc:385] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.265154  1777 raft_consensus.cc:740] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.266037  1775 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.266448  1775 ts_tablet_manager.cc:1403] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.006s
I20260709 14:50:30.266415  1777 consensus_queue.cc:260] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.269076  1777 ts_tablet_manager.cc:1434] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:50:30.268401  1774 raft_consensus.cc:359] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.269833  1774 raft_consensus.cc:385] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.270124  1774 raft_consensus.cc:740] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.270711  1777 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:30.271113  1775 raft_consensus.cc:359] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.271018  1774 consensus_queue.cc:260] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.271893  1775 raft_consensus.cc:385] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.272210  1775 raft_consensus.cc:740] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.273859  1774 ts_tablet_manager.cc:1434] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 14:50:30.272845  1775 consensus_queue.cc:260] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.275020  1774 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:30.276715  1775 ts_tablet_manager.cc:1434] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 14:50:30.277798  1775 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.278354  1777 tablet_bootstrap.cc:654] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.284786  1774 tablet_bootstrap.cc:654] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.286537  1775 tablet_bootstrap.cc:654] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.288272  1865 raft_consensus.cc:493] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.288875  1865 raft_consensus.cc:515] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.292749  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.293023  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.293509  1660 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:30.293628  1732 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:30.294910  1549 leader_election.cc:304] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:30.296245  1777 tablet_bootstrap.cc:492] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.296151  1827 raft_consensus.cc:2804] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.296870  1777 ts_tablet_manager.cc:1403] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.008s
I20260709 14:50:30.297509  1827 raft_consensus.cc:493] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.266526  1479 catalog_manager.cc:5697] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:30.298305  1827 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.296556  1865 leader_election.cc:290] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.302678  1777 raft_consensus.cc:359] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.307832  1777 raft_consensus.cc:385] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.308228  1777 raft_consensus.cc:740] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.308931  1777 consensus_queue.cc:260] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.310601  1777 ts_tablet_manager.cc:1434] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.000s
I20260709 14:50:30.316949  1777 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:30.321497  1827 raft_consensus.cc:515] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.325429  1827 leader_election.cc:290] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.326259  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:30.327304  1660 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.327960  1777 tablet_bootstrap.cc:654] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.326284  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:30.333767  1732 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.313427  1797 raft_consensus.cc:493] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.334543  1831 consensus_queue.cc:1048] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.335052  1775 tablet_bootstrap.cc:492] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.335592  1775 ts_tablet_manager.cc:1403] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.013s
I20260709 14:50:30.339327  1775 raft_consensus.cc:359] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.340104  1775 raft_consensus.cc:385] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.340400  1775 raft_consensus.cc:740] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.341301  1775 consensus_queue.cc:260] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.342900  1774 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.344368  1775 ts_tablet_manager.cc:1434] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:50:30.346557  1775 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.351392  1774 ts_tablet_manager.cc:1403] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.013s
I20260709 14:50:30.356803  1777 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.357447  1777 ts_tablet_manager.cc:1403] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.004s
I20260709 14:50:30.358503  1865 raft_consensus.cc:493] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.359148  1865 raft_consensus.cc:515] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.361771  1732 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:30.362109  1777 raft_consensus.cc:359] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.362895  1777 raft_consensus.cc:385] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.363096  1775 tablet_bootstrap.cc:654] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.363211  1777 raft_consensus.cc:740] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.363416  1549 leader_election.cc:304] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:30.363786  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.364427  1826 raft_consensus.cc:2804] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.364053  1777 consensus_queue.cc:260] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.364936  1826 raft_consensus.cc:697] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:30.357007  1774 raft_consensus.cc:359] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.366997  1774 raft_consensus.cc:385] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.366070  1826 consensus_queue.cc:237] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.367336  1774 raft_consensus.cc:740] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.368515  1774 consensus_queue.cc:260] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.370847  1774 ts_tablet_manager.cc:1434] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 14:50:30.372354  1774 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:30.372856  1812 consensus_queue.cc:1048] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.377720  1865 leader_election.cc:290] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.382141  1774 tablet_bootstrap.cc:654] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.334486  1797 raft_consensus.cc:515] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.389183  1657 raft_consensus.cc:3060] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.392753  1797 leader_election.cc:290] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:30.401403  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7fe23a70e9483db6bb31a0a6ab1096" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.416867  1548 leader_election.cc:304] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:30.418219  1865 raft_consensus.cc:2749] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.424525  1483 catalog_manager.cc:5697] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:30.403829  1826 consensus_queue.cc:1048] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.409348  1660 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:30.412827  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.414476  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.436208  1656 raft_consensus.cc:2468] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:30.434263  1827 consensus_queue.cc:1048] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:30.437623  1826 raft_consensus.cc:493] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.438169  1826 raft_consensus.cc:515] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.439005  1779 raft_consensus.cc:493] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:30.439404  1697 leader_election.cc:343] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.439519  1779 raft_consensus.cc:515] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.441478  1697 leader_election.cc:304] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:30.441823  1779 leader_election.cc:290] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.442755  1831 raft_consensus.cc:2804] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.443151  1831 raft_consensus.cc:493] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.443509  1831 raft_consensus.cc:3060] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.443675  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:30.445291  1623 leader_election.cc:343] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.446352  1827 raft_consensus.cc:493] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.446852  1827 raft_consensus.cc:515] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.450146  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.449136  1779 raft_consensus.cc:493] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.452099  1779 raft_consensus.cc:515] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.455513  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.456293  1584 raft_consensus.cc:2468] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:30.457921  1623 leader_election.cc:304] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.458917  1845 raft_consensus.cc:2804] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.459335  1845 raft_consensus.cc:493] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.459781  1845 raft_consensus.cc:3060] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:30.462630  1624 leader_election.cc:343] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.463596  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.466635  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:30.474969  1549 leader_election.cc:343] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.461256  1779 leader_election.cc:290] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.474586  1624 leader_election.cc:304] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:30.469461  1584 raft_consensus.cc:3060] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.480260  1777 ts_tablet_manager.cc:1434] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.122s	user 0.017s	sys 0.053s
I20260709 14:50:30.482007  1777 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
W20260709 14:50:30.491879  1624 leader_election.cc:343] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.494339  1800 consensus_queue.cc:1048] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.496598  1845 raft_consensus.cc:515] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.497781  1775 tablet_bootstrap.cc:492] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.497601  1831 raft_consensus.cc:515] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.498272  1775 ts_tablet_manager.cc:1403] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.152s	user 0.035s	sys 0.065s
I20260709 14:50:30.499197  1779 raft_consensus.cc:2749] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.470161  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.501966  1660 raft_consensus.cc:2468] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:30.499778  1831 leader_election.cc:290] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:30.505136  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.477377  1827 leader_election.cc:290] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.502372  1775 raft_consensus.cc:359] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.506601  1775 raft_consensus.cc:385] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.507038  1775 raft_consensus.cc:740] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.492065  1777 tablet_bootstrap.cc:654] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.468864  1797 raft_consensus.cc:493] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.507823  1775 consensus_queue.cc:260] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.500675  1826 leader_election.cc:290] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.508525  1548 leader_election.cc:304] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:30.520370  1775 ts_tablet_manager.cc:1434] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 14:50:30.527642  1826 raft_consensus.cc:2804] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.528384  1826 raft_consensus.cc:493] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.532105  1775 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.533362  1774 tablet_bootstrap.cc:492] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.534148  1774 ts_tablet_manager.cc:1403] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.162s	user 0.016s	sys 0.037s
I20260709 14:50:30.528891  1826 raft_consensus.cc:3060] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.515465  1797 raft_consensus.cc:515] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.535554  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:30.552564  1827 consensus_queue.cc:1048] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.555361  1584 raft_consensus.cc:3060] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.558928  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:30.545322  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.545678  1845 leader_election.cc:290] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:30.575937  1549 leader_election.cc:343] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.583006  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:30.583932  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:30.584611  1660 raft_consensus.cc:3060] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:30.595625  1548 leader_election.cc:343] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.596421  1548 leader_election.cc:304] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:30.637558  1908 raft_consensus.cc:2749] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.639196  1831 raft_consensus.cc:493] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.640697  1831 raft_consensus.cc:515] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.661682  1774 raft_consensus.cc:359] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.665449  1774 raft_consensus.cc:385] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.665953  1774 raft_consensus.cc:740] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.668813  1774 consensus_queue.cc:260] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:30.660615  1624 leader_election.cc:343] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.680909  1774 ts_tablet_manager.cc:1434] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.146s	user 0.009s	sys 0.009s
I20260709 14:50:30.693427  1774 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
W20260709 14:50:30.656800  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 61257.3us
I20260709 14:50:30.729133  1775 tablet_bootstrap.cc:654] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.726050  1826 raft_consensus.cc:515] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.733623  1777 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:30.736841  1777 ts_tablet_manager.cc:1403] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.255s	user 0.035s	sys 0.023s
I20260709 14:50:30.734196  1845 consensus_queue.cc:1048] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 14:50:30.734965  1831 leader_election.cc:290] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:30.737465  1797 leader_election.cc:290] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:30.743474  1697 leader_election.cc:343] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.733662  1660 raft_consensus.cc:2468] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:30.754012  1777 raft_consensus.cc:359] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.755366  1777 raft_consensus.cc:385] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.755769  1777 raft_consensus.cc:740] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.747000  1826 leader_election.cc:290] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.743310  1584 raft_consensus.cc:2468] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:30.775738  1623 leader_election.cc:304] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:30.776755  1777 consensus_queue.cc:260] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.777031  1845 raft_consensus.cc:2804] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.778623  1845 raft_consensus.cc:697] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:30.784484  1797 consensus_queue.cc:1048] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:30.784305  1845 consensus_queue.cc:237] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.785735  1777 ts_tablet_manager.cc:1434] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.048s	user 0.010s	sys 0.001s
I20260709 14:50:30.788794  1777 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:30.800236  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:30.804443  1729 raft_consensus.cc:3060] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.812068  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.813000  1585 raft_consensus.cc:2468] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:30.815181  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.843240  1775 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:30.843748  1775 ts_tablet_manager.cc:1403] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.312s	user 0.023s	sys 0.009s
I20260709 14:50:30.845974  1921 consensus_queue.cc:1048] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:50:30.857018  1729 raft_consensus.cc:2468] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:30.856799  1482 catalog_manager.cc:5697] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:30.859606  1775 raft_consensus.cc:359] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.860483  1775 raft_consensus.cc:385] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:30.860777  1775 raft_consensus.cc:740] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:30.861495  1775 consensus_queue.cc:260] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.876430  1775 ts_tablet_manager.cc:1434] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 14:50:30.881932  1826 consensus_queue.cc:1048] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.882922  1777 tablet_bootstrap.cc:654] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:30.897959  1775 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:30.904498  1697 leader_election.cc:304] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:30.907284  1774 tablet_bootstrap.cc:654] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:30.897418  1623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43817, user_credentials={real_user=slave}} blocked reactor thread for 48406.5us
I20260709 14:50:30.914142  1917 raft_consensus.cc:2804] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.916004  1917 raft_consensus.cc:493] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.916410  1917 raft_consensus.cc:3060] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.916051  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.912880  1826 raft_consensus.cc:493] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.917757  1826 raft_consensus.cc:515] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.920171  1826 leader_election.cc:290] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.923898  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.925911  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af1fd4b621c4a69afab73a4d89fa1a6" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
W20260709 14:50:30.923293  1697 leader_election.cc:343] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.930527  1929 raft_consensus.cc:493] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.932159  1929 raft_consensus.cc:515] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.933394  1659 raft_consensus.cc:2468] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:30.935743  1929 leader_election.cc:290] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.938277  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.939489  1580 raft_consensus.cc:2468] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:30.929389  1656 raft_consensus.cc:3060] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.946664  1623 leader_election.cc:304] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:30.947571  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:30.950244  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:30.953562  1549 leader_election.cc:304] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:30.955549  1827 raft_consensus.cc:2804] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.956164  1827 raft_consensus.cc:697] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:30.956542  1921 consensus_queue.cc:1048] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.957115  1827 consensus_queue.cc:237] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.966264  1779 raft_consensus.cc:2804] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:30.966790  1779 raft_consensus.cc:493] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:30.967216  1779 raft_consensus.cc:3060] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:30.968851  1929 raft_consensus.cc:493] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:30.969473  1929 raft_consensus.cc:515] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.972498  1929 leader_election.cc:290] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:30.975028  1774 tablet_bootstrap.cc:492] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:30.975145  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:30.976572  1697 leader_election.cc:304] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:30.977416  1774 ts_tablet_manager.cc:1403] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.288s	user 0.020s	sys 0.012s
W20260709 14:50:30.977530  1623 leader_election.cc:343] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.979292  1800 raft_consensus.cc:2804] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:30.979977  1800 raft_consensus.cc:697] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
W20260709 14:50:30.982604  1697 leader_election.cc:343] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:30.981884  1800 consensus_queue.cc:237] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.983947  1697 leader_election.cc:304] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:30.985515  1915 raft_consensus.cc:2749] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:30.978641  1917 raft_consensus.cc:515] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:30.987129  1931 consensus_queue.cc:1048] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:30.980860  1831 consensus_queue.cc:1048] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 14:50:30.998106  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:31.000648  1548 leader_election.cc:343] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.003542  1775 tablet_bootstrap.cc:654] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.007458  1733 raft_consensus.cc:2393] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:31.009671  1774 raft_consensus.cc:359] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.010524  1774 raft_consensus.cc:385] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.010865  1774 raft_consensus.cc:740] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.011595  1774 consensus_queue.cc:260] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.011247  1917 leader_election.cc:290] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.023134  1774 ts_tablet_manager.cc:1434] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 14:50:31.024634  1774 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:31.025315  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.027120  1479 catalog_manager.cc:5697] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:31.034633  1777 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:31.035279  1777 ts_tablet_manager.cc:1403] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.247s	user 0.020s	sys 0.016s
I20260709 14:50:31.040485  1779 raft_consensus.cc:515] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.051817  1779 leader_election.cc:290] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.052254  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:31.052986  1581 raft_consensus.cc:3060] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.053532  1656 raft_consensus.cc:2468] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:31.074462  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:31.059707  1549 leader_election.cc:343] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.082808  1549 leader_election.cc:304] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.083561  1777 raft_consensus.cc:359] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.084409  1777 raft_consensus.cc:385] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.084723  1777 raft_consensus.cc:740] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.084821  1812 raft_consensus.cc:2749] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.085474  1777 consensus_queue.cc:260] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.066743  1581 raft_consensus.cc:2468] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:31.088708  1623 leader_election.cc:304] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:31.093062  1779 raft_consensus.cc:2804] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:31.093642  1779 raft_consensus.cc:697] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:31.094548  1779 consensus_queue.cc:237] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.088708  1777 ts_tablet_manager.cc:1434] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.001s
I20260709 14:50:31.118664  1826 consensus_queue.cc:1048] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:31.117388  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:31.120658  1733 raft_consensus.cc:2393] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:31.123502  1777 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:31.123917  1774 tablet_bootstrap.cc:654] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.143846  1777 tablet_bootstrap.cc:654] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.152148  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ccd774316a4d69b235f2eddf58ddcc" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
W20260709 14:50:31.155791  1624 leader_election.cc:343] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.156347  1624 leader_election.cc:304] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:31.169371  1931 raft_consensus.cc:2749] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.174638  1775 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.175271  1775 ts_tablet_manager.cc:1403] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.298s	user 0.031s	sys 0.069s
I20260709 14:50:31.178215  1775 raft_consensus.cc:359] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.179306  1775 raft_consensus.cc:385] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.179638  1775 raft_consensus.cc:740] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
W20260709 14:50:31.179821  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:31.180387  1775 consensus_queue.cc:260] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.180660  1479 catalog_manager.cc:5697] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:31.182273  1482 catalog_manager.cc:5697] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:31.183687  1775 ts_tablet_manager.cc:1434] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:50:31.185000  1775 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.185333  1797 raft_consensus.cc:493] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.185878  1797 raft_consensus.cc:515] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.189075  1697 leader_election.cc:304] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.189726  1797 leader_election.cc:290] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.191826  1797 raft_consensus.cc:2749] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.195149  1774 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.195770  1774 ts_tablet_manager.cc:1403] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.171s	user 0.015s	sys 0.015s
I20260709 14:50:31.201512  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.202913  1581 raft_consensus.cc:2393] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:31.204872  1931 raft_consensus.cc:493] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.206285  1931 raft_consensus.cc:515] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.208714  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c555043e8447c4baadb1583c60a7e6" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.212343  1931 leader_election.cc:290] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.214552  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:31.216435  1623 leader_election.cc:343] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.217870  1697 leader_election.cc:304] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.219772  1775 tablet_bootstrap.cc:654] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.222046  1917 raft_consensus.cc:3060] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.222669  1774 raft_consensus.cc:359] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.223505  1774 raft_consensus.cc:385] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.223770  1774 raft_consensus.cc:740] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.224498  1774 consensus_queue.cc:260] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.225267  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.227404  1777 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:31.227540  1774 ts_tablet_manager.cc:1434] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.001s
I20260709 14:50:31.228112  1777 ts_tablet_manager.cc:1403] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.105s	user 0.013s	sys 0.007s
I20260709 14:50:31.228940  1774 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:31.232483  1917 raft_consensus.cc:2749] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.232183  1777 raft_consensus.cc:359] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:31.233870  1624 leader_election.cc:343] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.234048  1777 raft_consensus.cc:385] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.234450  1624 leader_election.cc:304] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:31.234651  1777 raft_consensus.cc:740] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.235476  1931 raft_consensus.cc:2749] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.235777  1777 consensus_queue.cc:260] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.238189  1777 ts_tablet_manager.cc:1434] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 14:50:31.238559  1774 tablet_bootstrap.cc:654] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.251888  1777 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:31.273020  1775 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.273669  1775 ts_tablet_manager.cc:1403] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.017s
I20260709 14:50:31.276625  1775 raft_consensus.cc:359] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.277359  1775 raft_consensus.cc:385] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.277602  1775 raft_consensus.cc:740] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.278235  1775 consensus_queue.cc:260] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.281464  1775 ts_tablet_manager.cc:1434] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 14:50:31.282560  1775 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.284150  1931 consensus_queue.cc:1048] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:31.291360  1775 tablet_bootstrap.cc:654] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.297173  1927 consensus_queue.cc:1048] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:31.334949  1777 tablet_bootstrap.cc:654] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.340598  1774 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.341267  1774 ts_tablet_manager.cc:1403] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.113s	user 0.026s	sys 0.033s
I20260709 14:50:31.344990  1775 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.345691  1775 ts_tablet_manager.cc:1403] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.063s	user 0.021s	sys 0.021s
I20260709 14:50:31.348026  1774 raft_consensus.cc:359] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.348987  1774 raft_consensus.cc:385] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.349318  1774 raft_consensus.cc:740] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.349601  1775 raft_consensus.cc:359] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.350400  1775 raft_consensus.cc:385] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.350185  1774 consensus_queue.cc:260] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.350718  1775 raft_consensus.cc:740] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.352278  1775 consensus_queue.cc:260] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.355548  1774 ts_tablet_manager.cc:1434] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.003s
I20260709 14:50:31.357234  1775 ts_tablet_manager.cc:1434] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 14:50:31.358345  1775 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.366098  1774 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:31.369879  1775 tablet_bootstrap.cc:654] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.385798  1774 tablet_bootstrap.cc:654] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.394439  1777 tablet_bootstrap.cc:492] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:31.398701  1777 ts_tablet_manager.cc:1403] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.147s	user 0.018s	sys 0.012s
I20260709 14:50:31.401584  1775 tablet_bootstrap.cc:492] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.402238  1775 ts_tablet_manager.cc:1403] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.011s
I20260709 14:50:31.402436  1777 raft_consensus.cc:359] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.403332  1777 raft_consensus.cc:385] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.403652  1777 raft_consensus.cc:740] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.404495  1777 consensus_queue.cc:260] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.409235  1777 ts_tablet_manager.cc:1434] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 14:50:31.409031  1775 raft_consensus.cc:359] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.409868  1775 raft_consensus.cc:385] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.409965  1774 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.410171  1775 raft_consensus.cc:740] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.410627  1774 ts_tablet_manager.cc:1403] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.008s
I20260709 14:50:31.411067  1775 consensus_queue.cc:260] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.413946  1774 raft_consensus.cc:359] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.415380  1774 raft_consensus.cc:385] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.415732  1774 raft_consensus.cc:740] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.416625  1774 consensus_queue.cc:260] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.418284  1775 ts_tablet_manager.cc:1434] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 14:50:31.418927  1774 ts_tablet_manager.cc:1434] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 14:50:31.419418  1775 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.420018  1774 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:31.430971  1917 raft_consensus.cc:493] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.431545  1917 raft_consensus.cc:515] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.433575  1917 leader_election.cc:290] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.435303  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.443769  1774 tablet_bootstrap.cc:654] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:31.455827  1697 leader_election.cc:343] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.447036  1777 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:31.436405  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.457214  1656 raft_consensus.cc:2468] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:31.462044  1697 leader_election.cc:304] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:31.462913  1917 raft_consensus.cc:2804] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.463333  1917 raft_consensus.cc:493] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.463685  1917 raft_consensus.cc:3060] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.466063  1826 raft_consensus.cc:493] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.466195  1777 tablet_bootstrap.cc:654] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.466208  1775 tablet_bootstrap.cc:654] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.466631  1826 raft_consensus.cc:515] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.468945  1826 leader_election.cc:290] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.471179  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.474213  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.475102  1734 raft_consensus.cc:2468] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
W20260709 14:50:31.476353  1548 leader_election.cc:343] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.477921  1549 leader_election.cc:304] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.479422  1826 raft_consensus.cc:2804] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.479864  1774 tablet_bootstrap.cc:492] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.479877  1826 raft_consensus.cc:493] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.480553  1774 ts_tablet_manager.cc:1403] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.018s
I20260709 14:50:31.480597  1826 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.482002  1775 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.482566  1775 ts_tablet_manager.cc:1403] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.013s
I20260709 14:50:31.484123  1800 consensus_queue.cc:1048] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:31.484499  1774 raft_consensus.cc:359] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.485306  1774 raft_consensus.cc:385] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.485605  1774 raft_consensus.cc:740] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.486596  1774 consensus_queue.cc:260] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.487838  1775 raft_consensus.cc:359] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.495018  1775 raft_consensus.cc:385] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.496320  1777 tablet_bootstrap.cc:492] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:31.495946  1917 raft_consensus.cc:515] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.496634  1775 raft_consensus.cc:740] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.496896  1777 ts_tablet_manager.cc:1403] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.012s
I20260709 14:50:31.497694  1775 consensus_queue.cc:260] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.497174  1826 raft_consensus.cc:515] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.500525  1775 ts_tablet_manager.cc:1434] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 14:50:31.500293  1777 raft_consensus.cc:359] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.503644  1775 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.502985  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:31.506742  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:31.507733  1548 leader_election.cc:343] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.509310  1774 ts_tablet_manager.cc:1434] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.002s
I20260709 14:50:31.509315  1826 leader_election.cc:290] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.511027  1774 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:31.512733  1775 tablet_bootstrap.cc:654] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.501093  1777 raft_consensus.cc:385] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.517081  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:31.517935  1734 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.516197  1777 raft_consensus.cc:740] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.522663  1777 consensus_queue.cc:260] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.524734  1585 raft_consensus.cc:3060] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.526343  1917 leader_election.cc:290] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.529431  1774 tablet_bootstrap.cc:654] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.532639  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a043e41dae8347b3988abf806a866b74" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:31.535065  1656 raft_consensus.cc:3060] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.535820  1777 ts_tablet_manager.cc:1434] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.039s	user 0.012s	sys 0.000s
I20260709 14:50:31.537083  1777 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:31.541105  1585 raft_consensus.cc:2468] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:31.545991  1827 consensus_queue.cc:1048] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:31.547240  1826 raft_consensus.cc:493] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.547869  1826 raft_consensus.cc:515] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.551326  1826 leader_election.cc:290] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.555734  1775 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.556597  1775 ts_tablet_manager.cc:1403] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.003s
I20260709 14:50:31.549961  1734 raft_consensus.cc:2468] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:31.558272  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.559111  1657 raft_consensus.cc:2468] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:31.563926  1775 raft_consensus.cc:359] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.564764  1775 raft_consensus.cc:385] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.565052  1775 raft_consensus.cc:740] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.565717  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.565809  1775 consensus_queue.cc:260] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.580526  1697 leader_election.cc:304] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:31.592137  1827 raft_consensus.cc:493] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.595713  1775 ts_tablet_manager.cc:1434] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.002s
I20260709 14:50:31.587953  1656 raft_consensus.cc:2468] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:31.592300  1800 raft_consensus.cc:493] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.574328  1931 raft_consensus.cc:493] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.594800  1732 raft_consensus.cc:2468] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:31.595021  1917 raft_consensus.cc:2804] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:31.604223  1548 leader_election.cc:304] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:31.605125  1917 raft_consensus.cc:697] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:31.601907  1800 raft_consensus.cc:515] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.605723  1908 raft_consensus.cc:2804] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.606196  1908 raft_consensus.cc:493] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.605959  1812 consensus_queue.cc:1048] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:31.606554  1908 raft_consensus.cc:3060] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.607430  1800 leader_election.cc:290] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.595310  1827 raft_consensus.cc:515] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.612219  1583 raft_consensus.cc:3060] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.613313  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.613426  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.614277  1656 raft_consensus.cc:2468] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:31.614281  1585 raft_consensus.cc:2468] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:31.617720  1774 tablet_bootstrap.cc:492] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.618283  1774 ts_tablet_manager.cc:1403] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.007s
I20260709 14:50:31.611922  1931 raft_consensus.cc:515] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.619206  1827 leader_election.cc:290] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.575073  1549 leader_election.cc:304] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.619900  1697 leader_election.cc:304] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:31.610049  1917 consensus_queue.cc:237] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.622149  1797 raft_consensus.cc:2804] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.622584  1797 raft_consensus.cc:493] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.626376  1775 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:31.626283  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.627303  1774 raft_consensus.cc:359] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.628121  1774 raft_consensus.cc:385] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.630722  1774 raft_consensus.cc:740] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.631858  1774 consensus_queue.cc:260] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.625656  1797 raft_consensus.cc:3060] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.648152  1931 leader_election.cc:290] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.633064  1827 raft_consensus.cc:2804] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:31.649425  1827 raft_consensus.cc:697] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:31.650391  1827 consensus_queue.cc:237] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.650741  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.665478  1777 tablet_bootstrap.cc:654] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.636579  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.665328  1800 raft_consensus.cc:493] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.689039  1732 raft_consensus.cc:2468] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:31.697362  1931 raft_consensus.cc:493] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.697962  1931 raft_consensus.cc:515] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.706866  1931 leader_election.cc:290] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:31.696096  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 121147us
I20260709 14:50:31.695273  1800 raft_consensus.cc:515] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.722316  1800 leader_election.cc:290] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:31.727432  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.730566  1732 raft_consensus.cc:2468] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:31.723079  1624 leader_election.cc:304] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:31.732272  1931 raft_consensus.cc:2804] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.732690  1931 raft_consensus.cc:493] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.733778  1931 raft_consensus.cc:3060] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.733819  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93fe5d42f4a4b38a4d1adc56b5fcabb" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.723099  1908 raft_consensus.cc:515] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.749961  1774 ts_tablet_manager.cc:1434] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.131s	user 0.018s	sys 0.021s
I20260709 14:50:31.776420  1775 tablet_bootstrap.cc:654] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.779577  1774 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
W20260709 14:50:31.787010  1623 leader_election.cc:343] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.782646  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.787032  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.794028  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:31.803849  1658 raft_consensus.cc:2468] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:31.791363  1797 raft_consensus.cc:515] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:31.800945  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 64716.5us
I20260709 14:50:31.800235  1585 raft_consensus.cc:2468] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:31.813707  1962 consensus_queue.cc:1048] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:31.799765  1624 leader_election.cc:304] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:31.818143  1697 leader_election.cc:304] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:31.818771  1927 raft_consensus.cc:2804] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.819267  1927 raft_consensus.cc:493] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.819607  1927 raft_consensus.cc:3060] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.822359  1964 raft_consensus.cc:2804] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:31.822597  1549 leader_election.cc:304] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:31.822947  1964 raft_consensus.cc:493] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:31.823347  1964 raft_consensus.cc:3060] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.824640  1779 consensus_queue.cc:1048] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:31.827126  1929 raft_consensus.cc:493] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.829774  1929 raft_consensus.cc:515] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.832106  1929 leader_election.cc:290] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.838676  1777 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:31.839489  1777 ts_tablet_manager.cc:1403] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.303s	user 0.020s	sys 0.004s
I20260709 14:50:31.837579  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.838189  1971 raft_consensus.cc:2749] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.851812  1908 leader_election.cc:290] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.867235  1927 raft_consensus.cc:515] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.872795  1929 consensus_queue.cc:1048] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:31.885632  1774 tablet_bootstrap.cc:654] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:31.887733  1483 catalog_manager.cc:5697] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:31.886731  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:31.892099  1729 raft_consensus.cc:3060] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:31.888142  1624 leader_election.cc:343] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.900094  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:31.900799  1580 raft_consensus.cc:3060] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.904001  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:31.904731  1582 raft_consensus.cc:3060] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.922163  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.898576  1927 leader_election.cc:290] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:31.937952  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:31.939296  1775 tablet_bootstrap.cc:492] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:31.940148  1775 ts_tablet_manager.cc:1403] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.314s	user 0.016s	sys 0.021s
I20260709 14:50:31.948268  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5deb3961d984f8481c1ee3e463f8a3b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:31.949077  1657 raft_consensus.cc:3060] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:31.941359  1797 leader_election.cc:290] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:31.941222  1623 leader_election.cc:343] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:31.950649  1777 raft_consensus.cc:359] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.953166  1777 raft_consensus.cc:385] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.953445  1777 raft_consensus.cc:740] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.956862  1623 leader_election.cc:304] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:31.958878  1779 raft_consensus.cc:2749] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:31.961978  1657 raft_consensus.cc:2468] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:31.963603  1729 raft_consensus.cc:2468] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:31.965591  1549 leader_election.cc:304] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:31.966507  1971 raft_consensus.cc:2804] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:31.967104  1971 raft_consensus.cc:697] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:31.967988  1971 consensus_queue.cc:237] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.970209  1774 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:31.969945  1777 consensus_queue.cc:260] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.970708  1774 ts_tablet_manager.cc:1403] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.196s	user 0.023s	sys 0.015s
I20260709 14:50:31.966208  1964 raft_consensus.cc:515] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.976965  1774 raft_consensus.cc:359] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.977870  1774 raft_consensus.cc:385] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.977067  1931 raft_consensus.cc:515] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.978204  1774 raft_consensus.cc:740] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.981377  1908 raft_consensus.cc:493] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:31.981992  1908 raft_consensus.cc:515] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.979857  1774 consensus_queue.cc:260] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.983490  1777 ts_tablet_manager.cc:1434] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.144s	user 0.002s	sys 0.007s
I20260709 14:50:31.982788  1775 raft_consensus.cc:359] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.985966  1775 raft_consensus.cc:385] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:31.986335  1775 raft_consensus.cc:740] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:31.987165  1775 consensus_queue.cc:260] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:31.988196  1777 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:31.994011  1774 ts_tablet_manager.cc:1434] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 14:50:31.990891  1730 raft_consensus.cc:2393] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:31.990612  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:32.000526  1975 raft_consensus.cc:493] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.001150  1975 raft_consensus.cc:515] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.003082  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:31.989414  1908 leader_election.cc:290] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:32.004451  1549 leader_election.cc:343] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.005580  1975 leader_election.cc:290] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.005373  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.005470  1797 raft_consensus.cc:493] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.007373  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.008262  1797 raft_consensus.cc:515] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.009086  1580 raft_consensus.cc:2468] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.010629  1697 leader_election.cc:304] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:32.010874  1797 leader_election.cc:290] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.013792  1963 raft_consensus.cc:2804] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.014611  1963 raft_consensus.cc:697] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:32.015631  1963 consensus_queue.cc:237] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.019001  1775 ts_tablet_manager.cc:1434] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 14:50:32.015337  1931 leader_election.cc:290] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.016033  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:31.987629  1582 raft_consensus.cc:2468] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:32.021771  1775 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:32.021703  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:32.023145  1964 leader_election.cc:290] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.027210  1623 leader_election.cc:304] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:32.031325  1779 raft_consensus.cc:2804] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.031913  1779 raft_consensus.cc:697] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
W20260709 14:50:32.028373  1623 leader_election.cc:343] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.032884  1779 consensus_queue.cc:237] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:32.050880  1697 leader_election.cc:343] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.056270  1774 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:32.054113  1479 catalog_manager.cc:5697] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:32.084472  1797 consensus_queue.cc:1048] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.085567  1775 tablet_bootstrap.cc:654] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.090006  1774 tablet_bootstrap.cc:654] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:32.097415  1548 leader_election.cc:343] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.106568  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.098392  1548 leader_election.cc:304] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:32.112809  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:32.113767  1732 raft_consensus.cc:3060] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.110852  1908 raft_consensus.cc:2749] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.114564  1729 raft_consensus.cc:2393] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:32.122314  1908 consensus_queue.cc:1048] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:32.129801  1732 raft_consensus.cc:2468] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:32.131575  1779 raft_consensus.cc:493] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.139506  1779 raft_consensus.cc:515] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.140076  1777 tablet_bootstrap.cc:654] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.146344  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:32.148253  1623 leader_election.cc:343] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.152550  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce56315bb31441e1bbe7db96a8ce4410" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.156194  1658 raft_consensus.cc:3060] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.142735  1779 leader_election.cc:290] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.183892  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd1126d930d488dab086eb5d690385a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.179200  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.186879  1657 raft_consensus.cc:2468] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.188396  1658 raft_consensus.cc:2468] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.192404  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.211833  1775 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:32.212450  1775 ts_tablet_manager.cc:1403] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.191s	user 0.038s	sys 0.006s
I20260709 14:50:32.215384  1624 leader_election.cc:304] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.216761  1624 leader_election.cc:304] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:32.222484  1927 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.229728  1985 raft_consensus.cc:2804] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.230396  1985 raft_consensus.cc:697] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:32.234186  1775 raft_consensus.cc:359] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.234984  1775 raft_consensus.cc:385] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.235275  1775 raft_consensus.cc:740] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.231331  1985 consensus_queue.cc:237] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.236034  1775 consensus_queue.cc:260] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.238499  1775 ts_tablet_manager.cc:1434] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 14:50:32.240149  1775 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:32.242617  1927 raft_consensus.cc:2749] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.250440  1777 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.251133  1777 ts_tablet_manager.cc:1403] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.263s	user 0.030s	sys 0.024s
I20260709 14:50:32.251650  1483 catalog_manager.cc:5697] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
W20260709 14:50:32.254640  1624 leader_election.cc:343] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.255239  1624 leader_election.cc:304] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.259049  1927 raft_consensus.cc:2749] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.261426  1777 raft_consensus.cc:359] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.262410  1777 raft_consensus.cc:385] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.262722  1777 raft_consensus.cc:740] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.263154  1697 leader_election.cc:304] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:32.263441  1777 consensus_queue.cc:260] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.264420  1963 raft_consensus.cc:2749] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.265836  1777 ts_tablet_manager.cc:1434] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 14:50:32.266969  1777 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.269199  1697 leader_election.cc:304] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:32.274816  1482 catalog_manager.cc:5697] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:32.275259  1963 raft_consensus.cc:2804] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:32.281451  1963 raft_consensus.cc:493] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:32.281875  1963 raft_consensus.cc:3060] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.285681  1774 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:32.286309  1774 ts_tablet_manager.cc:1403] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.230s	user 0.016s	sys 0.029s
I20260709 14:50:32.290416  1774 raft_consensus.cc:359] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.291312  1774 raft_consensus.cc:385] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.291556  1774 raft_consensus.cc:740] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.295960  1479 catalog_manager.cc:5697] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
W20260709 14:50:32.300063  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:32.299854  1774 consensus_queue.cc:260] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.299954  1963 raft_consensus.cc:515] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.302173  1774 ts_tablet_manager.cc:1434] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 14:50:32.304917  1963 leader_election.cc:290] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.307363  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:32.309697  1697 leader_election.cc:343] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.314102  1775 tablet_bootstrap.cc:654] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.314954  1774 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:32.317831  1826 consensus_queue.cc:1048] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.319517  1964 consensus_queue.cc:1048] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:32.321408  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.322372  1658 raft_consensus.cc:3060] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.323570  1482 catalog_manager.cc:5697] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:32.324412  1777 tablet_bootstrap.cc:654] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.338239  1775 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:32.338928  1775 ts_tablet_manager.cc:1403] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.000s
I20260709 14:50:32.341871  1775 raft_consensus.cc:359] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.342613  1775 raft_consensus.cc:385] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.342901  1775 raft_consensus.cc:740] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.345939  1775 consensus_queue.cc:260] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.350953  1775 ts_tablet_manager.cc:1434] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:50:32.355906  1775 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:32.349546  1658 raft_consensus.cc:2468] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.367342  1697 leader_election.cc:304] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:32.368728  1964 raft_consensus.cc:2804] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.369325  1964 raft_consensus.cc:697] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:32.370380  1964 consensus_queue.cc:237] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.370904  1774 tablet_bootstrap.cc:654] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.374408  1777 tablet_bootstrap.cc:492] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.375029  1777 ts_tablet_manager.cc:1403] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.013s
I20260709 14:50:32.378298  1777 raft_consensus.cc:359] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.379990  1777 raft_consensus.cc:385] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.380355  1777 raft_consensus.cc:740] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.381210  1777 consensus_queue.cc:260] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.398098  1777 ts_tablet_manager.cc:1434] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 14:50:32.400591  1777 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.401475  1775 tablet_bootstrap.cc:654] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.409328  1777 tablet_bootstrap.cc:654] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.442430  1482 catalog_manager.cc:5697] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:32.448163  1774 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:32.448918  1774 ts_tablet_manager.cc:1403] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.134s	user 0.037s	sys 0.023s
I20260709 14:50:32.452435  1774 raft_consensus.cc:359] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.453433  1774 raft_consensus.cc:385] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.456683  1774 raft_consensus.cc:740] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.457444  1774 consensus_queue.cc:260] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.457942  1777 tablet_bootstrap.cc:492] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.458709  1777 ts_tablet_manager.cc:1403] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.010s
I20260709 14:50:32.460086  1774 ts_tablet_manager.cc:1434] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 14:50:32.461659  1777 raft_consensus.cc:359] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.462393  1965 consensus_queue.cc:1048] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:32.463224  1774 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:32.462392  1777 raft_consensus.cc:385] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.464525  1777 raft_consensus.cc:740] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.465190  1777 consensus_queue.cc:260] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.468225  1777 ts_tablet_manager.cc:1434] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 14:50:32.469679  1777 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.492302  1777 tablet_bootstrap.cc:654] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.504515  1777 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.505075  1777 ts_tablet_manager.cc:1403] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.036s	user 0.004s	sys 0.009s
I20260709 14:50:32.505689  1985 consensus_queue.cc:1048] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.505899  1812 consensus_queue.cc:1048] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.508445  1964 raft_consensus.cc:493] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.512270  1964 raft_consensus.cc:515] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.512120  1777 raft_consensus.cc:359] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.514750  1964 leader_election.cc:290] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.514799  1777 raft_consensus.cc:385] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.516105  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.516812  1580 raft_consensus.cc:2468] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.515601  1777 raft_consensus.cc:740] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.518115  1697 leader_election.cc:304] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:32.518177  1777 consensus_queue.cc:260] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.519760  1964 raft_consensus.cc:2804] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:32.520138  1964 raft_consensus.cc:493] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:32.520502  1964 raft_consensus.cc:3060] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.520879  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.521559  1658 raft_consensus.cc:2468] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.528266  1777 ts_tablet_manager.cc:1434] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 14:50:32.533857  1777 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.538522  1775 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:32.539134  1775 ts_tablet_manager.cc:1403] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.184s	user 0.047s	sys 0.018s
I20260709 14:50:32.547585  1775 raft_consensus.cc:359] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.552937  1775 raft_consensus.cc:385] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.553380  1775 raft_consensus.cc:740] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.549079  1964 raft_consensus.cc:515] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.557027  1964 leader_election.cc:290] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.556943  1985 consensus_queue.cc:1048] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.552554  1777 tablet_bootstrap.cc:654] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.559329  1774 tablet_bootstrap.cc:654] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.562783  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:32.563036  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4ff68bffd45df8bbdafb01119e3aa" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.564487  1659 raft_consensus.cc:3060] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.554167  1775 consensus_queue.cc:260] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.564175  1964 consensus_queue.cc:1048] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.564222  1585 raft_consensus.cc:3060] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.579694  1659 raft_consensus.cc:2468] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.582165  1775 ts_tablet_manager.cc:1434] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 14:50:32.582008  1697 leader_election.cc:304] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:32.583974  1797 raft_consensus.cc:2804] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.584515  1797 raft_consensus.cc:697] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:32.587430  1775 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:32.588560  1985 raft_consensus.cc:493] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.585381  1797 consensus_queue.cc:237] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.589114  1985 raft_consensus.cc:515] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.597147  1985 leader_election.cc:290] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.598419  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.598853  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.607635  1775 tablet_bootstrap.cc:654] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:32.609975  1624 leader_election.cc:343] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.610671  1964 consensus_queue.cc:1048] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:50:32.612794  1623 leader_election.cc:343] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.613272  1623 leader_election.cc:304] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.613615  1777 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.614143  1777 ts_tablet_manager.cc:1403] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.007s
I20260709 14:50:32.615381  1985 raft_consensus.cc:2749] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.622946  1777 raft_consensus.cc:359] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.624167  1777 raft_consensus.cc:385] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.624469  1777 raft_consensus.cc:740] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.624965  1777 consensus_queue.cc:260] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.635557  1777 ts_tablet_manager.cc:1434] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.006s
I20260709 14:50:32.636711  1777 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.637938  1585 raft_consensus.cc:2468] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.649694  1774 tablet_bootstrap.cc:492] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:32.650293  1774 ts_tablet_manager.cc:1403] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.187s	user 0.020s	sys 0.015s
I20260709 14:50:32.653185  1774 raft_consensus.cc:359] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.659283  1774 raft_consensus.cc:385] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.662901  1826 raft_consensus.cc:493] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.663226  1985 raft_consensus.cc:493] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.663525  1826 raft_consensus.cc:515] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.663729  1985 raft_consensus.cc:515] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.665824  1479 catalog_manager.cc:5697] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:32.666734  1826 leader_election.cc:290] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.667690  1655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.667712  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.658890  1777 tablet_bootstrap.cc:654] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.665881  1985 leader_election.cc:290] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:32.670814  1624 leader_election.cc:343] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.659616  1774 raft_consensus.cc:740] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.671875  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.672608  1585 raft_consensus.cc:2468] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:32.672626  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd3d3a0c81c4e6e83d700d44aa3d2fc" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.672524  1774 consensus_queue.cc:260] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.673629  1730 raft_consensus.cc:2393] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:32.674345  1623 leader_election.cc:304] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.674770  1549 leader_election.cc:304] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:32.675596  1774 ts_tablet_manager.cc:1434] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.008s
I20260709 14:50:32.675658  1826 raft_consensus.cc:3060] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.675837  1985 raft_consensus.cc:2804] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:32.677109  1985 raft_consensus.cc:493] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:32.677273  1774 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:32.677516  1985 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:32.683174  1611 tablet.cc:2406] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 14:50:32.685566  1826 raft_consensus.cc:2749] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.690341  1774 tablet_bootstrap.cc:654] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.699234  1971 raft_consensus.cc:493] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.699848  1971 raft_consensus.cc:515] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.700742  1777 tablet_bootstrap.cc:492] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.701289  1777 ts_tablet_manager.cc:1403] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.000s
I20260709 14:50:32.695891  1985 raft_consensus.cc:515] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.704239  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:32.705044  1580 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.707683  1777 raft_consensus.cc:359] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.708456  1777 raft_consensus.cc:385] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.708715  1777 raft_consensus.cc:740] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.711678  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.709415  1777 consensus_queue.cc:260] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.712461  1927 consensus_queue.cc:1048] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:32.710716  1971 leader_election.cc:290] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.715166  1777 ts_tablet_manager.cc:1434] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 14:50:32.716454  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.716841  1777 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
W20260709 14:50:32.717847  1549 leader_election.cc:343] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:32.728104  1548 leader_election.cc:343] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.728652  1548 leader_election.cc:304] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:32.734781  1777 tablet_bootstrap.cc:654] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.733168  1971 raft_consensus.cc:2749] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.740053  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:32.743831  1774 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:32.744400  1774 ts_tablet_manager.cc:1403] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.009s
W20260709 14:50:32.745499  1624 leader_election.cc:343] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.733230  1985 leader_election.cc:290] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.748201  1963 raft_consensus.cc:493] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.750080  1963 raft_consensus.cc:515] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.752620  1963 leader_election.cc:290] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.753974  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.754940  1585 raft_consensus.cc:2468] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.757750  1580 raft_consensus.cc:2468] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:32.758999  1697 leader_election.cc:304] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:32.759182  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.754334  1775 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:32.759876  1659 raft_consensus.cc:2468] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.760339  1775 ts_tablet_manager.cc:1403] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.173s	user 0.011s	sys 0.013s
I20260709 14:50:32.760262  1797 raft_consensus.cc:2804] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:32.755740  1774 raft_consensus.cc:359] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.761307  1774 raft_consensus.cc:385] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.761620  1774 raft_consensus.cc:740] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.760144  1623 leader_election.cc:304] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.760939  1797 raft_consensus.cc:493] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:32.767076  1797 raft_consensus.cc:3060] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.767208  1927 raft_consensus.cc:2804] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.769333  1927 raft_consensus.cc:697] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:32.770383  1927 consensus_queue.cc:237] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.777611  1774 consensus_queue.cc:260] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.763005  1985 consensus_queue.cc:1048] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 14:50:32.798235  1774 ts_tablet_manager.cc:1434] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.001s
I20260709 14:50:32.800794  1482 catalog_manager.cc:5697] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:32.806627  1775 raft_consensus.cc:359] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.808015  1775 raft_consensus.cc:385] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.810045  1775 raft_consensus.cc:740] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.810745  1775 consensus_queue.cc:260] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.819475  1774 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:32.831079  1963 raft_consensus.cc:493] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.831909  1963 raft_consensus.cc:515] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.835049  1963 leader_election.cc:290] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.841441  1777 tablet_bootstrap.cc:492] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.842183  1777 ts_tablet_manager.cc:1403] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.020s
I20260709 14:50:32.845031  1775 ts_tablet_manager.cc:1434] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.004s
I20260709 14:50:32.831825  1797 raft_consensus.cc:515] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.852967  1797 leader_election.cc:290] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.845463  1777 raft_consensus.cc:359] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.854110  1777 raft_consensus.cc:385] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.854429  1777 raft_consensus.cc:740] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.855168  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.855407  1777 consensus_queue.cc:260] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.856524  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:32.857179  1582 raft_consensus.cc:3060] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.857906  1777 ts_tablet_manager.cc:1434] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 14:50:32.859397  1777 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:32.859658  1585 raft_consensus.cc:2468] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.863430  1985 raft_consensus.cc:493] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.864028  1985 raft_consensus.cc:515] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.866605  1777 tablet_bootstrap.cc:654] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.867383  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:32.870965  1623 leader_election.cc:343] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.870813  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d58c741dd1442bf9096d719c5d54058" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.872300  1659 raft_consensus.cc:3060] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.873797  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:32.875682  1624 leader_election.cc:343] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.876168  1624 leader_election.cc:304] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.876914  1927 raft_consensus.cc:2749] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.877664  1697 leader_election.cc:304] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:32.880620  1775 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:32.876924  1985 leader_election.cc:290] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.882061  1774 tablet_bootstrap.cc:654] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.882406  1797 raft_consensus.cc:2804] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:32.885632  1797 raft_consensus.cc:493] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:32.887107  1797 raft_consensus.cc:3060] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.900982  1655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.901901  1655 raft_consensus.cc:2468] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:32.921511  1985 raft_consensus.cc:493] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:32.922735  1985 raft_consensus.cc:515] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.925319  1985 leader_election.cc:290] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:32.941407  1582 raft_consensus.cc:2468] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.944482  1797 raft_consensus.cc:515] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.952917  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:32.956743  1659 raft_consensus.cc:2468] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.956655  1775 tablet_bootstrap.cc:654] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:32.957136  1797 leader_election.cc:290] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:32.959724  1697 leader_election.cc:304] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
W20260709 14:50:32.960843  1624 leader_election.cc:343] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.962374  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:32.961562  1963 raft_consensus.cc:2804] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.964903  1963 raft_consensus.cc:697] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:32.965656  1659 raft_consensus.cc:3060] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:32.966105  1963 consensus_queue.cc:237] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.973914  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:32.975728  1964 consensus_queue.cc:1048] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:32.976327  1774 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:32.977046  1774 ts_tablet_manager.cc:1403] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.017s
I20260709 14:50:32.978677  1777 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:32.979251  1777 ts_tablet_manager.cc:1403] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.059s
W20260709 14:50:32.982657  1623 leader_election.cc:343] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:32.982378  1777 raft_consensus.cc:359] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.983263  1777 raft_consensus.cc:385] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:32.983232  1623 leader_election.cc:304] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:32.983736  1777 raft_consensus.cc:740] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:32.984735  1777 consensus_queue.cc:260] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:32.985256  1985 raft_consensus.cc:2749] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:32.988164  1659 raft_consensus.cc:2468] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:32.992084  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915bcb7025e246cf9db5e084d3115085" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.000072  1580 raft_consensus.cc:3060] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.000957  1697 leader_election.cc:304] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:33.002188  1964 raft_consensus.cc:2804] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:32.993554  1582 raft_consensus.cc:3060] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.018311  1483 catalog_manager.cc:5697] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:33.021763  1964 raft_consensus.cc:697] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:33.022974  1964 consensus_queue.cc:237] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.023597  1826 raft_consensus.cc:493] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.024092  1826 raft_consensus.cc:515] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.026678  1777 ts_tablet_manager.cc:1434] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 14:50:33.035161  1826 leader_election.cc:290] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.035945  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:33.037990  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f4b54cf7034f25bdd1182b79e561d7" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:32.988067  1774 raft_consensus.cc:359] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.040704  1774 raft_consensus.cc:385] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.040880  1548 leader_election.cc:304] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.042052  1826 raft_consensus.cc:2749] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:33.041023  1774 raft_consensus.cc:740] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.045255  2026 consensus_queue.cc:1048] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.049098  1775 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.049755  1775 ts_tablet_manager.cc:1403] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.169s	user 0.023s	sys 0.009s
I20260709 14:50:33.043429  1774 consensus_queue.cc:260] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.052752  1775 raft_consensus.cc:359] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.053515  1775 raft_consensus.cc:385] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.053876  1775 raft_consensus.cc:740] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.055058  1775 consensus_queue.cc:260] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.058096  1775 ts_tablet_manager.cc:1434] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:50:33.058723  1774 ts_tablet_manager.cc:1434] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
I20260709 14:50:33.059723  1775 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.022025  1580 raft_consensus.cc:2468] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:33.062970  1777 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:33.069252  1774 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.069445  1775 tablet_bootstrap.cc:654] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.085747  1777 tablet_bootstrap.cc:654] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.098202  1775 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.098542  1774 tablet_bootstrap.cc:654] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.099017  1775 ts_tablet_manager.cc:1403] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.015s
I20260709 14:50:33.104586  1777 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:33.105502  1777 ts_tablet_manager.cc:1403] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.066s	user 0.006s	sys 0.010s
I20260709 14:50:33.106294  1797 raft_consensus.cc:493] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.106793  1797 raft_consensus.cc:515] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.108830  1797 leader_election.cc:290] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.109386  1775 raft_consensus.cc:359] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.110332  1775 raft_consensus.cc:385] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.110669  1775 raft_consensus.cc:740] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.110925  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.111646  1585 raft_consensus.cc:2468] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:33.111855  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.112365  1775 consensus_queue.cc:260] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.113333  1697 leader_election.cc:343] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.117362  1774 tablet_bootstrap.cc:492] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:33.114799  1777 raft_consensus.cc:359] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.117894  1697 leader_election.cc:304] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.117942  1774 ts_tablet_manager.cc:1403] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.005s
I20260709 14:50:33.118407  1777 raft_consensus.cc:385] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.118777  1797 raft_consensus.cc:2804] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.119256  1797 raft_consensus.cc:493] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.119596  1797 raft_consensus.cc:3060] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.120061  1775 ts_tablet_manager.cc:1434] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 14:50:33.121629  1779 raft_consensus.cc:493] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.121306  1774 raft_consensus.cc:359] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.122094  1774 raft_consensus.cc:385] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.122447  1774 raft_consensus.cc:740] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.122160  1779 raft_consensus.cc:515] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.123262  1774 consensus_queue.cc:260] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.124507  1779 leader_election.cc:290] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.125638  1774 ts_tablet_manager.cc:1434] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 14:50:33.126688  1774 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.128726  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.128814  1479 catalog_manager.cc:5697] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
W20260709 14:50:33.131294  1623 leader_election.cc:343] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.135211  1775 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.135456  1774 tablet_bootstrap.cc:654] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.135212  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:33.137483  1624 leader_election.cc:343] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.137989  1624 leader_election.cc:304] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:33.139251  1779 raft_consensus.cc:2749] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.141005  2027 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.141613  2027 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.147192  1826 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.147868  1826 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.152364  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.153270  2027 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.152803  1775 tablet_bootstrap.cc:654] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.154626  1826 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:33.155630  1548 leader_election.cc:343] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.164454  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.165280  1585 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:33.165544  1774 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:33.166107  1774 ts_tablet_manager.cc:1403] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.000s
I20260709 14:50:33.167335  1775 tablet_bootstrap.cc:492] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.170720  1775 ts_tablet_manager.cc:1403] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.008s
I20260709 14:50:33.167382  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.174626  1797 raft_consensus.cc:515] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.175593  1775 raft_consensus.cc:359] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.169970  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:33.176618  1775 raft_consensus.cc:385] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.176684  1797 leader_election.cc:290] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.176801  1729 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:33.177099  1775 raft_consensus.cc:740] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.175784  1774 raft_consensus.cc:359] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.177723  1774 raft_consensus.cc:385] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.178023  1774 raft_consensus.cc:740] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.118816  1777 raft_consensus.cc:740] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.177970  1697 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:33.179211  2027 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.179554  1964 consensus_queue.cc:1048] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.178102  1775 consensus_queue.cc:260] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.179024  1774 consensus_queue.cc:260] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.181334  1697 leader_election.cc:343] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.180298  1777 consensus_queue.cc:260] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.179585  2027 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.183499  2027 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.183109  1775 ts_tablet_manager.cc:1434] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:50:33.185490  1777 ts_tablet_manager.cc:1434] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.080s	user 0.006s	sys 0.002s
I20260709 14:50:33.186758  1775 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.192768  2027 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.193341  1777 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:33.183108  1774 ts_tablet_manager.cc:1434] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 14:50:33.195964  1774 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.198223  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.198975  1659 raft_consensus.cc:3060] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.204681  1549 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.204731  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b259d0baa0c4493eba83ff1de96cf93f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.206982  1585 raft_consensus.cc:3060] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.206570  1826 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.211092  1826 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.211489  1826 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.213794  1797 consensus_queue.cc:1048] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.218061  1659 raft_consensus.cc:2468] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:33.219704  1697 leader_election.cc:304] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:33.220634  2027 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.221364  2027 raft_consensus.cc:2804] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:33.221771  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.224170  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.234169  1826 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.238050  1729 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.222877  2027 raft_consensus.cc:697] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
W20260709 14:50:33.240391  1697 leader_election.cc:343] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.239993  1582 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:33.249984  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.251456  1775 tablet_bootstrap.cc:654] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.251245  1697 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
W20260709 14:50:33.252547  1548 leader_election.cc:343] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.252535  1826 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.255199  1777 tablet_bootstrap.cc:654] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.255366  1797 raft_consensus.cc:2749] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.259099  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:33.262241  1733 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:33.262203  1585 raft_consensus.cc:2468] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:33.265046  1549 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.272576  1971 raft_consensus.cc:2749] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.275702  1774 tablet_bootstrap.cc:654] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.253430  2027 consensus_queue.cc:237] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.282987  1779 raft_consensus.cc:493] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.283846  1779 raft_consensus.cc:515] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.293002  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:33.299932  1623 leader_election.cc:343] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.306766  1797 raft_consensus.cc:493] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.307675  1797 raft_consensus.cc:515] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.306685  1985 consensus_queue.cc:1048] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:33.310904  1779 leader_election.cc:290] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.319096  1797 leader_election.cc:290] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.321346  1655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.327936  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.328758  1580 raft_consensus.cc:2468] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:33.321501  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:33.331431  1624 leader_election.cc:343] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.331729  1775 tablet_bootstrap.cc:492] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.331885  1624 leader_election.cc:304] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:33.332245  1775 ts_tablet_manager.cc:1403] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.033s
I20260709 14:50:33.332743  1985 raft_consensus.cc:2749] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.339736  1697 leader_election.cc:304] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.342867  1797 raft_consensus.cc:2804] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:50:33.345687  1985 consensus_queue.cc:1048] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:33.346107  1797 raft_consensus.cc:493] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45d5272855ec440cbd07d4d3995c68d3)
I20260709 14:50:33.346727  1797 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:33.355746  1775 raft_consensus.cc:359] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.357903  1775 raft_consensus.cc:385] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.358213  1775 raft_consensus.cc:740] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.362692  1775 consensus_queue.cc:260] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.367271  1774 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:33.367835  1774 ts_tablet_manager.cc:1403] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.172s	user 0.021s	sys 0.009s
I20260709 14:50:33.376525  1483 catalog_manager.cc:5697] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:33.383877  1775 ts_tablet_manager.cc:1434] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 14:50:33.384745  2026 consensus_queue.cc:1048] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:33.377879  1797 raft_consensus.cc:515] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.388346  2027 consensus_queue.cc:1048] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.390611  1729 raft_consensus.cc:1240] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Rejecting Update request from peer 45d5272855ec440cbd07d4d3995c68d3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:50:33.393596  2020 consensus_queue.cc:1059] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.394528  2020 raft_consensus.cc:3055] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:33.394767  1775 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.394972  2020 raft_consensus.cc:740] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:33.395381  1774 raft_consensus.cc:359] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.396147  1774 raft_consensus.cc:385] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.395964  2020 consensus_queue.cc:260] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.397219  2020 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:33.396463  1774 raft_consensus.cc:740] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.406672  1797 leader_election.cc:290] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.457868  1774 consensus_queue.cc:260] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.424015  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.463433  1774 ts_tablet_manager.cc:1434] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.095s	user 0.005s	sys 0.010s
I20260709 14:50:33.469375  1774 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.469064  1964 consensus_queue.cc:1048] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:33.471457  1964 raft_consensus.cc:493] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.472857  1964 raft_consensus.cc:515] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.477411  1964 leader_election.cc:290] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.479368  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c265ae69744159a2cbd5fea8af0d07" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.480751  1985 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.481282  1985 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.483516  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.484890  1985 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.482685  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.487872  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:33.488088  1697 leader_election.cc:343] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.488675  1732 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:33.489862  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.490623  1581 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:33.487936  1585 raft_consensus.cc:2468] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:33.499415  1775 tablet_bootstrap.cc:654] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.510906  1659 raft_consensus.cc:2468] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 2.
I20260709 14:50:33.512857  1697 leader_election.cc:304] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:33.514544  2027 raft_consensus.cc:2804] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:33.516306  2027 raft_consensus.cc:697] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:33.516963  1963 consensus_queue.cc:1048] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 14:50:33.525624  2027 consensus_queue.cc:237] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.540304  1623 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:33.542182  1697 leader_election.cc:304] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.543460  1797 raft_consensus.cc:2804] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.543844  1797 raft_consensus.cc:493] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.544166  1797 raft_consensus.cc:3060] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.553510  1774 tablet_bootstrap.cc:654] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.554994  2026 raft_consensus.cc:493] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.564935  2026 raft_consensus.cc:515] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.567444  2026 leader_election.cc:290] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.547356  2020 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:33.568662  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:33.577049  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.577942  1659 raft_consensus.cc:2468] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:33.579140  1826 raft_consensus.cc:493] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.579684  1826 raft_consensus.cc:515] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.581669  1775 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.582434  1775 ts_tablet_manager.cc:1403] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.010s
I20260709 14:50:33.583364  1826 leader_election.cc:290] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.586802  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.586588  1775 raft_consensus.cc:359] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.588639  1775 raft_consensus.cc:385] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.589056  1775 raft_consensus.cc:740] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.589948  1775 consensus_queue.cc:260] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.591292  1697 leader_election.cc:304] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:33.592654  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.593256  1659 raft_consensus.cc:2468] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:33.556123  1777 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:33.595239  1777 ts_tablet_manager.cc:1403] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.402s	user 0.027s	sys 0.017s
I20260709 14:50:33.596841  1479 catalog_manager.cc:5697] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 1 to 2, leader changed from 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194) to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:33.601306  2026 raft_consensus.cc:2804] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.601727  2026 raft_consensus.cc:493] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.602072  2026 raft_consensus.cc:3060] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.594413  1548 leader_election.cc:304] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:33.604528  1775 ts_tablet_manager.cc:1434] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.004s
I20260709 14:50:33.598803  1797 raft_consensus.cc:515] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.604115  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:33.646271  1797 leader_election.cc:290] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:33.645885  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 51546.9us
W20260709 14:50:33.648654  1549 leader_election.cc:343] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.648854  1812 raft_consensus.cc:2804] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.649453  1812 raft_consensus.cc:493] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.649933  1812 raft_consensus.cc:3060] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.665727  1775 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.670367  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.671226  1580 raft_consensus.cc:3060] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.665230  1777 raft_consensus.cc:359] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.673328  1697 leader_election.cc:343] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.673393  1777 raft_consensus.cc:385] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.675515  1777 raft_consensus.cc:740] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.677531  1777 consensus_queue.cc:260] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.683080  2020 raft_consensus.cc:2749] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.695899  2026 raft_consensus.cc:515] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.698220  2026 leader_election.cc:290] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.700320  1777 ts_tablet_manager.cc:1434] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.007s
I20260709 14:50:33.701509  1777 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:33.702755  1812 raft_consensus.cc:515] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.706284  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.703356  1580 raft_consensus.cc:2468] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
W20260709 14:50:33.710997  1623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43817, user_credentials={real_user=slave}} blocked reactor thread for 57073.9us
I20260709 14:50:33.714586  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.719491  1774 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
W20260709 14:50:33.719282  1697 leader_election.cc:343] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.720201  1697 leader_election.cc:304] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:33.723080  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.723578  1962 raft_consensus.cc:2804] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:33.723811  1659 raft_consensus.cc:3060] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.724151  1962 raft_consensus.cc:697] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:33.723032  1774 ts_tablet_manager.cc:1403] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.255s	user 0.016s	sys 0.006s
I20260709 14:50:33.725042  1962 consensus_queue.cc:237] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.736937  1697 leader_election.cc:343] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.743593  1812 leader_election.cc:290] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.746563  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:33.750006  1774 raft_consensus.cc:359] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.749677  1549 leader_election.cc:343] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.750948  1774 raft_consensus.cc:385] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.751400  1774 raft_consensus.cc:740] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.752292  1774 consensus_queue.cc:260] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.752782  1659 raft_consensus.cc:2468] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:33.754155  2027 consensus_queue.cc:1048] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:33.754947  1774 ts_tablet_manager.cc:1434] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 14:50:33.755844  1775 tablet_bootstrap.cc:654] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.756278  1774 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.762506  1777 tablet_bootstrap.cc:654] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.766443  1774 tablet_bootstrap.cc:654] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.774710  1775 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.775475  1775 ts_tablet_manager.cc:1403] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.008s
I20260709 14:50:33.775470  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:33.776397  1657 raft_consensus.cc:3060] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.782447  1777 tablet_bootstrap.cc:492] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:33.783279  1777 ts_tablet_manager.cc:1403] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.003s
I20260709 14:50:33.789323  1777 raft_consensus.cc:359] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.790352  1777 raft_consensus.cc:385] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.785869  1775 raft_consensus.cc:359] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.790591  1697 leader_election.cc:304] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:33.790899  1775 raft_consensus.cc:385] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.790792  2026 raft_consensus.cc:493] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.791492  2026 raft_consensus.cc:515] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.792835  1775 raft_consensus.cc:740] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.790644  1777 raft_consensus.cc:740] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.798921  2026 leader_election.cc:290] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:33.794692  1777 consensus_queue.cc:260] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.804457  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:33.799939  1775 consensus_queue.cc:260] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.806879  1697 leader_election.cc:343] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.808248  1777 ts_tablet_manager.cc:1434] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 14:50:33.802322  2020 raft_consensus.cc:493] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:33.812212  1775 ts_tablet_manager.cc:1434] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 14:50:33.807360  2026 consensus_queue.cc:1048] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 14:50:33.822348  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.824509  1775 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
W20260709 14:50:33.826999  1697 leader_election.cc:343] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.827661  1697 leader_election.cc:304] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:33.802322  2027 raft_consensus.cc:2804] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:33.804194  1657 raft_consensus.cc:2468] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:33.839936  2020 raft_consensus.cc:515] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.839880  2026 raft_consensus.cc:2749] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:33.844131  2027 raft_consensus.cc:697] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:33.845458  2027 consensus_queue.cc:237] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.809899  1777 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:33.821970  1482 catalog_manager.cc:5697] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
W20260709 14:50:33.852803  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 52564us
I20260709 14:50:33.853123  1774 tablet_bootstrap.cc:492] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:33.857646  1774 ts_tablet_manager.cc:1403] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.102s	user 0.018s	sys 0.038s
I20260709 14:50:33.859563  2020 leader_election.cc:290] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.861949  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:33.862773  1582 raft_consensus.cc:2468] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:33.864171  1623 leader_election.cc:304] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:33.864629  1548 leader_election.cc:304] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:33.866840  1812 raft_consensus.cc:2804] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:33.867923  1812 raft_consensus.cc:697] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:33.868445  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:33.868829  1812 consensus_queue.cc:237] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:33.865141  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 59687.6us
W20260709 14:50:33.873386  1624 leader_election.cc:343] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.866391  2020 raft_consensus.cc:2804] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:33.874691  2020 raft_consensus.cc:493] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:33.875130  2020 raft_consensus.cc:3060] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.911625  1775 tablet_bootstrap.cc:654] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.916605  1777 tablet_bootstrap.cc:654] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.918743  2020 raft_consensus.cc:515] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.906405  1774 raft_consensus.cc:359] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.922689  1774 raft_consensus.cc:385] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.922449  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:33.923069  1774 raft_consensus.cc:740] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.923278  1582 raft_consensus.cc:3060] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:33.925619  1774 consensus_queue.cc:260] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.929018  1479 catalog_manager.cc:5697] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:33.932837  1582 raft_consensus.cc:2468] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:33.940766  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:33.942176  1623 leader_election.cc:304] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:33.942917  1624 leader_election.cc:343] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:33.943008  1779 raft_consensus.cc:2804] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:33.943619  1779 raft_consensus.cc:697] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:33.944514  1779 consensus_queue.cc:237] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.939721  2020 leader_election.cc:290] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:33.947183  1774 ts_tablet_manager.cc:1434] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.001s
I20260709 14:50:33.950088  1774 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:33.949280  1483 catalog_manager.cc:5697] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:33.952663  1775 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:33.953367  1775 ts_tablet_manager.cc:1403] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.001s
I20260709 14:50:33.955436  1777 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:33.955961  1777 ts_tablet_manager.cc:1403] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.146s	user 0.017s	sys 0.009s
I20260709 14:50:33.956305  1775 raft_consensus.cc:359] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.956931  1775 raft_consensus.cc:385] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.957176  1775 raft_consensus.cc:740] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.957932  1775 consensus_queue.cc:260] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.962024  1777 raft_consensus.cc:359] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.970707  1774 tablet_bootstrap.cc:654] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:33.971350  1777 raft_consensus.cc:385] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:33.971715  1777 raft_consensus.cc:740] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:33.972455  1777 consensus_queue.cc:260] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:33.974889  1777 ts_tablet_manager.cc:1434] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 14:50:33.976853  1775 ts_tablet_manager.cc:1434] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 14:50:33.978705  1777 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:33.983001  1775 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:33.987929  1482 catalog_manager.cc:5697] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:33.999791  1777 tablet_bootstrap.cc:654] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.013065  1582 raft_consensus.cc:3060] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:34.015165  1775 tablet_bootstrap.cc:654] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.020622  1777 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.021210  1777 ts_tablet_manager.cc:1403] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.043s	user 0.003s	sys 0.015s
I20260709 14:50:34.022660  1779 raft_consensus.cc:493] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.023543  1582 raft_consensus.cc:1275] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Refusing update from remote peer 36d78753ff96438e9cc2808f05853294: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:34.023299  1779 raft_consensus.cc:515] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.025102  1777 raft_consensus.cc:359] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.026072  1777 raft_consensus.cc:385] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.026445  1777 raft_consensus.cc:740] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.025350  1962 consensus_queue.cc:1048] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:34.027334  1775 tablet_bootstrap.cc:492] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.027263  1777 consensus_queue.cc:260] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.027895  1775 ts_tablet_manager.cc:1403] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.001s
I20260709 14:50:34.027961  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.027297  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.028685  1580 raft_consensus.cc:2468] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:34.029656  1777 ts_tablet_manager.cc:1434] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
W20260709 14:50:34.031153  1624 leader_election.cc:343] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.031793  1777 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:34.031878  1775 raft_consensus.cc:359] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.032617  1775 raft_consensus.cc:385] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.032909  1775 raft_consensus.cc:740] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.033895  1774 tablet_bootstrap.cc:492] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:34.033684  1775 consensus_queue.cc:260] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.034407  1774 ts_tablet_manager.cc:1403] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.085s	user 0.017s	sys 0.006s
I20260709 14:50:34.035985  1775 ts_tablet_manager.cc:1434] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 14:50:34.036280  1623 leader_election.cc:304] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.037563  1775 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:34.037725  2020 raft_consensus.cc:2804] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.038239  2020 raft_consensus.cc:493] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.037851  1774 raft_consensus.cc:359] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.038578  2020 raft_consensus.cc:3060] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.038635  1774 raft_consensus.cc:385] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.039077  1774 raft_consensus.cc:740] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.039850  1774 consensus_queue.cc:260] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.045598  1775 tablet_bootstrap.cc:654] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.045598  1777 tablet_bootstrap.cc:654] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.025625  1779 leader_election.cc:290] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.048207  1962 raft_consensus.cc:493] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.058245  1774 ts_tablet_manager.cc:1434] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 14:50:34.057256  2048 raft_consensus.cc:493] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.063794  2048 raft_consensus.cc:515] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.055866  1962 raft_consensus.cc:515] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.065907  1774 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:34.066628  1962 leader_election.cc:290] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.069208  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.069931  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.070778  1653 raft_consensus.cc:2393] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:34.073071  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a4bcec2d91452aa995a62be8b6c26b" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.073947  1583 raft_consensus.cc:2393] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:34.074592  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4130f75a588248fba88156660a11b6e8" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.076431  1623 leader_election.cc:304] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.077189  1779 raft_consensus.cc:3060] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.072459  2048 leader_election.cc:290] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.080423  1697 leader_election.cc:304] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:34.086094  2026 raft_consensus.cc:3060] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.103029  1779 raft_consensus.cc:2749] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:34.104727  1653 raft_consensus.cc:1275] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Refusing update from remote peer 36d78753ff96438e9cc2808f05853294: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:34.111037  2026 raft_consensus.cc:2749] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:34.118080  2027 consensus_queue.cc:1048] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:34.124161  1775 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.124789  1775 ts_tablet_manager.cc:1403] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.023s
I20260709 14:50:34.131357  1775 raft_consensus.cc:359] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.132172  1775 raft_consensus.cc:385] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.133672  1775 raft_consensus.cc:740] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.135571  2048 raft_consensus.cc:493] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.136117  2048 raft_consensus.cc:515] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.136178  1775 consensus_queue.cc:260] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.138341  2048 leader_election.cc:290] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.139142  1775 ts_tablet_manager.cc:1434] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 14:50:34.111039  2020 raft_consensus.cc:515] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.141584  1775 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:34.141299  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.143213  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:34.143931  1585 raft_consensus.cc:3060] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:34.145879  1623 leader_election.cc:343] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.143270  1774 tablet_bootstrap.cc:654] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.147076  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:34.156384  1624 leader_election.cc:343] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.156906  1624 leader_election.cc:304] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.159003  2048 raft_consensus.cc:2749] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:34.160602  2020 leader_election.cc:290] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.163890  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:34.166689  1585 raft_consensus.cc:2468] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
W20260709 14:50:34.169854  1624 leader_election.cc:343] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.171248  1623 leader_election.cc:304] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.172076  1779 raft_consensus.cc:2804] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:34.173080  1779 raft_consensus.cc:697] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:34.174191  1777 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.174134  1779 consensus_queue.cc:237] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.174999  1777 ts_tablet_manager.cc:1403] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.143s	user 0.020s	sys 0.010s
I20260709 14:50:34.172158  1775 tablet_bootstrap.cc:654] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.178742  1777 raft_consensus.cc:359] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.186563  1777 raft_consensus.cc:385] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.187100  1777 raft_consensus.cc:740] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.190892  1777 consensus_queue.cc:260] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.196977  1777 ts_tablet_manager.cc:1434] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 14:50:34.201737  1777 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:34.207749  1483 catalog_manager.cc:5697] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:34.211457  1774 tablet_bootstrap.cc:492] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:34.212047  1774 ts_tablet_manager.cc:1403] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.013s
I20260709 14:50:34.213554  1777 tablet_bootstrap.cc:654] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.216048  1774 raft_consensus.cc:359] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.216799  1774 raft_consensus.cc:385] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.217075  1774 raft_consensus.cc:740] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.217728  1774 consensus_queue.cc:260] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.211443  1775 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.220021  1775 ts_tablet_manager.cc:1403] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.004s
I20260709 14:50:34.222926  1774 ts_tablet_manager.cc:1434] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 14:50:34.225366  1775 raft_consensus.cc:359] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.226256  1775 raft_consensus.cc:385] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.226519  1775 raft_consensus.cc:740] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.227573  1812 raft_consensus.cc:493] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.227259  1775 consensus_queue.cc:260] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.228123  1812 raft_consensus.cc:515] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.229351  1775 ts_tablet_manager.cc:1434] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:50:34.230489  1812 leader_election.cc:290] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.231387  1774 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:34.234463  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.235864  1657 raft_consensus.cc:2393] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:34.256525  1777 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.257143  1777 ts_tablet_manager.cc:1403] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.026s
I20260709 14:50:34.257400  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce2a67caa54429fb9510b8ee027fba5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.260178  1777 raft_consensus.cc:359] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.267535  1777 raft_consensus.cc:385] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.267897  1777 raft_consensus.cc:740] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.268615  1777 consensus_queue.cc:260] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.271515  1549 leader_election.cc:304] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:34.261529  1775 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:34.263175  1774 tablet_bootstrap.cc:654] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.272684  1812 raft_consensus.cc:3060] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.271051  1777 ts_tablet_manager.cc:1434] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 14:50:34.288872  1777 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:34.297128  2026 consensus_queue.cc:1048] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:34.299754  1775 tablet_bootstrap.cc:654] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.307024  1777 tablet_bootstrap.cc:654] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.309355  1812 raft_consensus.cc:2749] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:34.321928  1775 tablet_bootstrap.cc:492] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.322059  1777 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.322640  1775 ts_tablet_manager.cc:1403] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.003s
I20260709 14:50:34.323215  1777 ts_tablet_manager.cc:1403] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.010s
I20260709 14:50:34.327147  1777 raft_consensus.cc:359] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.328325  1777 raft_consensus.cc:385] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.326094  1775 raft_consensus.cc:359] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.328619  1777 raft_consensus.cc:740] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.328804  1775 raft_consensus.cc:385] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.329115  1774 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:34.328837  2026 consensus_queue.cc:1048] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:34.329193  1775 raft_consensus.cc:740] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.329766  1774 ts_tablet_manager.cc:1403] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.099s	user 0.015s	sys 0.004s
I20260709 14:50:34.329411  1777 consensus_queue.cc:260] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.330013  1775 consensus_queue.cc:260] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.332546  1775 ts_tablet_manager.cc:1434] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 14:50:34.334029  1774 raft_consensus.cc:359] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.335132  1774 raft_consensus.cc:385] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.335403  1774 raft_consensus.cc:740] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.338956  1775 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:34.346019  2056 consensus_queue.cc:1048] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:34.347960  1775 tablet_bootstrap.cc:654] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.349514  1777 ts_tablet_manager.cc:1434] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
W20260709 14:50:34.351387  1624 consensus_peers.cc:597] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Couldn't send request to peer 36d78753ff96438e9cc2808f05853294. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:34.352631  1777 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:34.350373  2068 consensus_queue.cc:1048] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 14:50:34.350328  1774 consensus_queue.cc:260] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.389005  1779 raft_consensus.cc:493] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.389685  1779 raft_consensus.cc:515] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.399896  1964 raft_consensus.cc:493] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.400537  1964 raft_consensus.cc:515] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.401552  1774 ts_tablet_manager.cc:1434] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.000s
I20260709 14:50:34.402642  1774 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:34.407845  1779 leader_election.cc:290] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.402634  1964 leader_election.cc:290] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.413362  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.415997  2056 raft_consensus.cc:493] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.416697  2056 raft_consensus.cc:515] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.418169  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.418951  1580 raft_consensus.cc:2468] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:34.420025  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.420745  1583 raft_consensus.cc:2468] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:34.421931  2027 consensus_queue.cc:1048] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:34.423207  1623 leader_election.cc:304] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:34.424067  1779 raft_consensus.cc:2804] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.424449  1779 raft_consensus.cc:493] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.424751  1779 raft_consensus.cc:3060] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.425958  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.426173  1697 leader_election.cc:304] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:34.426682  1657 raft_consensus.cc:2468] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:34.427560  2027 raft_consensus.cc:2804] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.427975  2027 raft_consensus.cc:493] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.428328  2027 raft_consensus.cc:3060] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.438679  1812 consensus_queue.cc:1048] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:50:34.474239  1624 leader_election.cc:343] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.477797  1779 raft_consensus.cc:515] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.480948  1774 tablet_bootstrap.cc:654] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.434062  2056 leader_election.cc:290] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.488044  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.488821  1653 raft_consensus.cc:2468] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:34.487317  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.497185  1777 tablet_bootstrap.cc:654] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.497994  1548 leader_election.cc:304] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:34.503882  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:34.504658  1584 raft_consensus.cc:3060] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.513471  1779 leader_election.cc:290] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.517320  1775 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.517918  1775 ts_tablet_manager.cc:1403] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.019s
I20260709 14:50:34.514947  2027 raft_consensus.cc:515] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.519436  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:34.506131  2056 raft_consensus.cc:2804] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.521904  2056 raft_consensus.cc:493] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:50:34.522346  1624 leader_election.cc:343] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.523725  2027 leader_election.cc:290] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.524525  1775 raft_consensus.cc:359] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.544842  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
W20260709 14:50:34.539878  1549 leader_election.cc:343] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.549489  1583 raft_consensus.cc:3060] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.526535  1779 consensus_queue.cc:1048] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 14:50:34.570376  1775 raft_consensus.cc:385] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.570799  1775 raft_consensus.cc:740] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.571499  1775 consensus_queue.cc:260] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.526948  2056 raft_consensus.cc:3060] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.538338  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c0c3916ca4477fb2e82c720b4c1e87" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:34.527606  1584 raft_consensus.cc:2468] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:34.616580  1775 ts_tablet_manager.cc:1434] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.098s	user 0.001s	sys 0.013s
I20260709 14:50:34.598927  1653 raft_consensus.cc:3060] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.624053  1623 leader_election.cc:304] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.651214  1775 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
W20260709 14:50:34.659340  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:34.631503  2020 raft_consensus.cc:2804] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:34.685062  2020 raft_consensus.cc:697] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:34.685902  1583 raft_consensus.cc:2468] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:34.686116  2020 consensus_queue.cc:237] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.698596  1777 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.699162  1777 ts_tablet_manager.cc:1403] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.347s	user 0.040s	sys 0.012s
I20260709 14:50:34.702275  2056 raft_consensus.cc:515] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.705559  2056 leader_election.cc:290] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.707255  1774 tablet_bootstrap.cc:492] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:34.707834  1774 ts_tablet_manager.cc:1403] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.305s	user 0.019s	sys 0.038s
I20260709 14:50:34.712874  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:34.713531  1659 raft_consensus.cc:3060] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.710397  1963 raft_consensus.cc:493] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.716496  1971 raft_consensus.cc:493] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.727566  1777 raft_consensus.cc:359] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.728819  1777 raft_consensus.cc:385] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.729157  1777 raft_consensus.cc:740] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.733669  1774 raft_consensus.cc:359] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.734391  1774 raft_consensus.cc:385] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.734432  1697 leader_election.cc:304] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:34.735328  2026 raft_consensus.cc:2804] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:34.732998  2020 consensus_queue.cc:1048] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:34.738116  2068 raft_consensus.cc:493] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.738452  2026 raft_consensus.cc:697] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:34.738679  2068 raft_consensus.cc:515] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.739492  2026 consensus_queue.cc:237] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.723191  1963 raft_consensus.cc:515] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.735484  1777 consensus_queue.cc:260] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.740938  2068 leader_election.cc:290] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.734678  1774 raft_consensus.cc:740] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.743367  2048 raft_consensus.cc:493] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.744011  2048 raft_consensus.cc:515] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.726008  1971 raft_consensus.cc:515] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.753396  2048 leader_election.cc:290] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.753031  1774 consensus_queue.cc:260] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.754238  1777 ts_tablet_manager.cc:1434] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.055s	user 0.003s	sys 0.007s
I20260709 14:50:34.755940  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.756299  1777 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:34.756639  1583 raft_consensus.cc:2468] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:34.757776  1659 raft_consensus.cc:2468] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:34.756138  1653 raft_consensus.cc:2468] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:34.757975  1623 leader_election.cc:304] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:34.758996  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.759709  2048 raft_consensus.cc:2804] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.759001  1548 leader_election.cc:304] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:34.760252  2048 raft_consensus.cc:493] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.760668  2048 raft_consensus.cc:3060] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.759966  1483 catalog_manager.cc:5697] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:34.759708  1729 raft_consensus.cc:2468] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:34.761510  1826 raft_consensus.cc:2804] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:34.761982  1826 raft_consensus.cc:697] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:34.762799  1826 consensus_queue.cc:237] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.763793  1774 ts_tablet_manager.cc:1434] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 14:50:34.764920  1774 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:34.776118  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:34.776795  2056 raft_consensus.cc:493] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.777452  2056 raft_consensus.cc:515] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.779379  1971 leader_election.cc:290] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.780472  2056 leader_election.cc:290] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.754762  1963 leader_election.cc:290] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.781477  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.784526  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:34.787465  1549 leader_election.cc:343] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.799916  1774 tablet_bootstrap.cc:654] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.800977  1775 tablet_bootstrap.cc:654] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.806929  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.807745  1733 raft_consensus.cc:2468] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:34.814719  1482 catalog_manager.cc:5697] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:34.819674  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:34.826228  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.826994  1659 raft_consensus.cc:2468] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:34.826896  1549 leader_election.cc:304] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
W20260709 14:50:34.827975  1549 leader_election.cc:343] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.830214  2048 raft_consensus.cc:515] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.831887  1729 raft_consensus.cc:3060] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.832545  2048 leader_election.cc:290] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.833060  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:34.833603  1584 raft_consensus.cc:3060] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.840204  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66b38ba9a15445b787fe3e5738e58a0f" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.842748  1584 raft_consensus.cc:2468] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:34.843974  1971 raft_consensus.cc:2764] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 14:50:34.846683  1774 tablet_bootstrap.cc:492] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:34.847388  1774 ts_tablet_manager.cc:1403] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.000s
W20260709 14:50:34.848699  1697 leader_election.cc:343] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.851266  1623 leader_election.cc:304] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:34.852149  2020 raft_consensus.cc:2804] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:34.852654  2020 raft_consensus.cc:697] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:34.853539  2020 consensus_queue.cc:237] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.859177  1777 tablet_bootstrap.cc:654] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.859498  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.864326  1730 raft_consensus.cc:3060] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.864892  1775 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.865442  1775 ts_tablet_manager.cc:1403] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.235s	user 0.013s	sys 0.004s
I20260709 14:50:34.850438  1774 raft_consensus.cc:359] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.860630  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.867484  1657 raft_consensus.cc:2468] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:34.866539  1774 raft_consensus.cc:385] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.868351  1774 raft_consensus.cc:740] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.869236  1774 consensus_queue.cc:260] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.871173  1697 leader_election.cc:304] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:34.879982  1963 raft_consensus.cc:493] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45d5272855ec440cbd07d4d3995c68d3)
I20260709 14:50:34.884527  1963 raft_consensus.cc:515] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.887132  1962 raft_consensus.cc:2804] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.887785  1962 raft_consensus.cc:493] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.888253  1962 raft_consensus.cc:3060] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.892597  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75a712ecfe4deeb97bc690ad53e39c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:34.892719  1775 raft_consensus.cc:359] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.893395  1729 raft_consensus.cc:3060] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.893981  1774 ts_tablet_manager.cc:1434] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 14:50:34.893554  1775 raft_consensus.cc:385] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.894589  1775 raft_consensus.cc:740] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.895623  1775 consensus_queue.cc:260] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.898545  1774 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:34.898748  1775 ts_tablet_manager.cc:1434] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.008s
I20260709 14:50:34.902673  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.903640  1963 leader_election.cc:290] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.904927  1775 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:34.913419  1697 leader_election.cc:304] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:34.919368  1962 raft_consensus.cc:515] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.920351  1963 raft_consensus.cc:493] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:34.920902  1963 raft_consensus.cc:515] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.924542  1963 leader_election.cc:290] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.927254  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:34.928223  1583 raft_consensus.cc:2393] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:34.929152  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:34.930330  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9d4f77ab264824824b9de6802cec54" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:34.934604  1697 leader_election.cc:343] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.939463  1729 raft_consensus.cc:2468] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
W20260709 14:50:34.934343  2096 log.cc:927] Time spent T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 14:50:34.940579  2026 raft_consensus.cc:2749] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:34.940968  1775 tablet_bootstrap.cc:654] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.942737  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e64847f9b324f7faefa87f278434672" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:34.943981  1548 leader_election.cc:304] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:34.944205  1962 leader_election.cc:290] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:34.945273  1826 raft_consensus.cc:2804] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:34.945662  1826 raft_consensus.cc:493] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:34.946008  1826 raft_consensus.cc:3060] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:34.963773  1479 catalog_manager.cc:5697] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:34.981241  1826 raft_consensus.cc:515] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.984406  1826 leader_election.cc:290] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:34.985085  1777 tablet_bootstrap.cc:492] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:34.985698  1777 ts_tablet_manager.cc:1403] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.230s	user 0.010s	sys 0.016s
I20260709 14:50:34.985982  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
W20260709 14:50:34.987708  1549 leader_election.cc:343] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:34.988744  1775 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:34.988801  1774 tablet_bootstrap.cc:654] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:34.989706  1777 raft_consensus.cc:359] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.989326  1775 ts_tablet_manager.cc:1403] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.014s
I20260709 14:50:34.990490  1777 raft_consensus.cc:385] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.990808  1777 raft_consensus.cc:740] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.991627  1777 consensus_queue.cc:260] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.994457  1775 raft_consensus.cc:359] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.995648  1697 leader_election.cc:304] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:34.995934  1775 raft_consensus.cc:385] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:34.996259  1775 raft_consensus.cc:740] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:34.997220  1777 ts_tablet_manager.cc:1434] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 14:50:34.997073  2026 raft_consensus.cc:2749] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:34.997008  1775 consensus_queue.cc:260] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:34.995707  1482 catalog_manager.cc:5697] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:34.998378  1777 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:35.002462  1775 ts_tablet_manager.cc:1434] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:50:35.003717  1775 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.007380  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:35.008144  1657 raft_consensus.cc:3060] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.009487  1697 leader_election.cc:304] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:35.011135  2026 raft_consensus.cc:3060] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.017926  1657 raft_consensus.cc:2468] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:35.018560  1775 tablet_bootstrap.cc:654] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.020500  2026 raft_consensus.cc:2749] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.028497  2092 consensus_queue.cc:1048] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.032006  2092 raft_consensus.cc:493] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.032495  2092 raft_consensus.cc:515] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.022112  1777 tablet_bootstrap.cc:654] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.026119  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:35.035897  1657 raft_consensus.cc:3060] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.036000  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:35.038486  1623 leader_election.cc:343] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.043327  2092 leader_election.cc:290] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.044106  1775 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.044678  1775 ts_tablet_manager.cc:1403] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.009s
I20260709 14:50:35.044801  1657 raft_consensus.cc:2468] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:35.047392  1774 tablet_bootstrap.cc:492] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:35.047966  1774 ts_tablet_manager.cc:1403] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.004s
I20260709 14:50:35.048069  1548 leader_election.cc:304] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.048543  1775 raft_consensus.cc:359] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.049248  1826 raft_consensus.cc:2804] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:35.049305  1775 raft_consensus.cc:385] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.049779  1775 raft_consensus.cc:740] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.049871  1826 raft_consensus.cc:697] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:35.050621  1775 consensus_queue.cc:260] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.050798  1826 consensus_queue.cc:237] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.052950  1775 ts_tablet_manager.cc:1434] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:50:35.054354  1775 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.055776  1774 raft_consensus.cc:359] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.059221  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.059311  1697 leader_election.cc:304] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:35.060458  1962 raft_consensus.cc:2804] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:35.061021  1962 raft_consensus.cc:697] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:35.056571  1774 raft_consensus.cc:385] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.062378  1962 consensus_queue.cc:237] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.063112  1774 raft_consensus.cc:740] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
W20260709 14:50:35.065438  1624 leader_election.cc:343] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.065979  1624 leader_election.cc:304] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.067250  2092 raft_consensus.cc:2749] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.068816  1774 consensus_queue.cc:260] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.073865  1777 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:35.076072  1777 ts_tablet_manager.cc:1403] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.078s	user 0.009s	sys 0.005s
I20260709 14:50:35.074433  1774 ts_tablet_manager.cc:1434] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.005s
I20260709 14:50:35.078321  1774 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:35.080890  1777 raft_consensus.cc:359] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.081537  1777 raft_consensus.cc:385] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.081867  1777 raft_consensus.cc:740] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.076794  1775 tablet_bootstrap.cc:654] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.082926  1777 consensus_queue.cc:260] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.085959  1777 ts_tablet_manager.cc:1434] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:50:35.087002  1777 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:35.094782  1777 tablet_bootstrap.cc:654] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.110715  2092 raft_consensus.cc:493] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.111595  2092 raft_consensus.cc:515] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.114185  2092 leader_election.cc:290] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.115222  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.116146  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.117383  1777 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
W20260709 14:50:35.117872  1624 leader_election.cc:343] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.118808  1775 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.119426  1775 ts_tablet_manager.cc:1403] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.003s
I20260709 14:50:35.118963  1482 catalog_manager.cc:5697] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
W20260709 14:50:35.120069  1623 leader_election.cc:343] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.120550  1623 leader_election.cc:304] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.117959  1777 ts_tablet_manager.cc:1403] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.011s
I20260709 14:50:35.121685  2092 raft_consensus.cc:2749] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.124044  1775 raft_consensus.cc:359] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.125020  1775 raft_consensus.cc:385] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.124781  1777 raft_consensus.cc:359] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.125494  1777 raft_consensus.cc:385] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.125700  1775 raft_consensus.cc:740] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.125783  1777 raft_consensus.cc:740] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.127086  1774 tablet_bootstrap.cc:654] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.126669  1775 consensus_queue.cc:260] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.126855  1777 consensus_queue.cc:260] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.129449  1775 ts_tablet_manager.cc:1434] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 14:50:35.129791  1777 ts_tablet_manager.cc:1434] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 14:50:35.130700  1775 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.131160  1777 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:35.137722  1775 tablet_bootstrap.cc:654] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.140141  1479 catalog_manager.cc:5697] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:35.143217  1777 tablet_bootstrap.cc:654] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.152455  1775 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.153084  1775 ts_tablet_manager.cc:1403] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 14:50:35.156167  1775 raft_consensus.cc:359] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.156977  1775 raft_consensus.cc:385] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.157310  1775 raft_consensus.cc:740] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.158039  1775 consensus_queue.cc:260] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.166066  1775 ts_tablet_manager.cc:1434] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 14:50:35.167515  1775 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.168584  2092 raft_consensus.cc:493] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.169152  2092 raft_consensus.cc:515] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.171573  2092 leader_election.cc:290] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.174606  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:35.176767  1623 leader_election.cc:343] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.178766  1775 tablet_bootstrap.cc:654] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.181368  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.198693  1775 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.199482  1775 ts_tablet_manager.cc:1403] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.013s
I20260709 14:50:35.202572  1775 raft_consensus.cc:359] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.203539  1775 raft_consensus.cc:385] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.203899  1775 raft_consensus.cc:740] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.204638  1775 consensus_queue.cc:260] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.210675  1775 ts_tablet_manager.cc:1434] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 14:50:35.211956  1812 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.212744  1812 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.218269  1812 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.230440  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.231643  1653 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:35.234814  1826 raft_consensus.cc:493] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.235669  1826 raft_consensus.cc:515] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.244578  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.245801  1653 raft_consensus.cc:2468] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
W20260709 14:50:35.247223  1624 leader_election.cc:343] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.247800  1624 leader_election.cc:304] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.248179  1962 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.248529  2092 raft_consensus.cc:2749] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.248715  1962 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.238688  1826 leader_election.cc:290] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.253364  1962 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:35.253437  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.253628  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.254495  1729 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
W20260709 14:50:35.265558  1549 consensus_peers.cc:597] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe -> Peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Couldn't send request to peer 36d78753ff96438e9cc2808f05853294. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:35.266584  1775 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
W20260709 14:50:35.266575  1549 leader_election.cc:343] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.271267  1548 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:35.280560  1548 leader_election.cc:304] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.284387  1812 raft_consensus.cc:2804] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:35.284813  1812 raft_consensus.cc:493] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:35.285228  1812 raft_consensus.cc:3060] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.288370  2057 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:50:35.288990  2057 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:35.289354  2057 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:35.264386  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.296923  1779 consensus_queue.cc:1048] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.306947  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.307896  1659 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:35.322655  1779 consensus_queue.cc:1048] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.325627  1812 raft_consensus.cc:515] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.326853  2092 consensus_queue.cc:1048] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:50:35.331887  1812 leader_election.cc:290] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.344255  1826 raft_consensus.cc:493] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.345052  1826 raft_consensus.cc:515] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.346306  1774 tablet_bootstrap.cc:492] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:35.346879  1774 ts_tablet_manager.cc:1403] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.269s	user 0.024s	sys 0.052s
I20260709 14:50:35.348003  1775 tablet_bootstrap.cc:654] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.349251  1826 leader_election.cc:290] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.354491  1777 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:35.355139  1777 ts_tablet_manager.cc:1403] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.224s	user 0.047s	sys 0.120s
I20260709 14:50:35.357940  1777 raft_consensus.cc:359] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.358687  1777 raft_consensus.cc:385] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.359010  1777 raft_consensus.cc:740] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.359654  1777 consensus_queue.cc:260] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.364351  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.364790  1697 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:35.362344  1777 ts_tablet_manager.cc:1434] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 14:50:35.366583  1777 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:35.366663  1730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:35.367969  1797 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:50:35.368386  1797 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:35.368727  1797 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:50:35.372376  1549 leader_election.cc:343] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.376157  2086 consensus_queue.cc:1048] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.379237  1797 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.381392  1774 raft_consensus.cc:359] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.382375  1774 raft_consensus.cc:385] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.382704  1774 raft_consensus.cc:740] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.384423  1774 consensus_queue.cc:260] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.388324  1797 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:35.392674  2026 raft_consensus.cc:493] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.393321  2026 raft_consensus.cc:515] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.395488  2026 leader_election.cc:290] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:35.396080  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:35.341166  2057 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.402869  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6915b13ffb704edcac0fd629381d253e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.403723  1659 raft_consensus.cc:2393] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:35.405975  1777 tablet_bootstrap.cc:654] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.401242  2092 raft_consensus.cc:493] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.424085  2092 raft_consensus.cc:515] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.427042  1653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:35.427783  1653 raft_consensus.cc:3060] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.428225  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b7cdd0e0a24e54b17023ff094ef854" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.429117  1585 raft_consensus.cc:2468] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:35.431910  1623 leader_election.cc:304] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:35.411583  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.434769  1774 ts_tablet_manager.cc:1434] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.001s
I20260709 14:50:35.438894  1583 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 2: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:35.440599  2048 raft_consensus.cc:2804] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:35.447796  1774 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
W20260709 14:50:35.448942  1624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 45410.8us
I20260709 14:50:35.441352  2048 raft_consensus.cc:493] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:35.443125  2092 leader_election.cc:290] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.454452  2048 raft_consensus.cc:3060] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.481698  1655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:35.482481  1655 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:35.483124  1775 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.483686  1775 ts_tablet_manager.cc:1403] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.217s	user 0.045s	sys 0.035s
W20260709 14:50:35.491591  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 45694.7us
I20260709 14:50:35.467034  2123 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.528529  2123 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.527733  2122 raft_consensus.cc:493] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.578683  1775 raft_consensus.cc:359] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.589731  1775 raft_consensus.cc:385] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.591961  1775 raft_consensus.cc:740] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.577431  2057 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.600404  1775 consensus_queue.cc:260] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.567310  2122 raft_consensus.cc:515] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.611562  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "575fecfb4a7e4812b9b8f6fb093936b5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.612478  1658 raft_consensus.cc:2393] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:35.612706  1775 ts_tablet_manager.cc:1434] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.129s	user 0.016s	sys 0.003s
I20260709 14:50:35.614056  1775 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.614601  1777 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:35.615185  1777 ts_tablet_manager.cc:1403] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.249s	user 0.029s	sys 0.019s
I20260709 14:50:35.611725  2125 raft_consensus.cc:493] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.601187  1729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:35.629748  2122 leader_election.cc:290] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.608665  2123 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.630760  1729 raft_consensus.cc:2393] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 2: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
I20260709 14:50:35.646085  1777 raft_consensus.cc:359] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.648064  1777 raft_consensus.cc:385] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.648478  1777 raft_consensus.cc:740] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.649505  1777 consensus_queue.cc:260] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.641525  1656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:35.668160  1777 ts_tablet_manager.cc:1434] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 14:50:35.635056  2125 raft_consensus.cc:515] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.666551  1548 leader_election.cc:304] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:35.674795  1697 leader_election.cc:304] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:35.653353  2026 consensus_queue.cc:1048] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 14:50:35.677543  1826 consensus_queue.cc:1048] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.691601  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.673025  1582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.680580  1971 raft_consensus.cc:3060] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.691382  1963 raft_consensus.cc:3060] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:35.726166  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 48276.6us
I20260709 14:50:35.727108  1548 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:35.727936  1812 raft_consensus.cc:2749] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 14:50:35.732173  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:35.733426  1659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.715078  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b7cdd0e0a24e54b17023ff094ef854" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:35.737109  1548 leader_election.cc:343] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.738557  1812 consensus_queue.cc:1048] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 14:50:35.741505  1623 leader_election.cc:343] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:35.742183  1623 leader_election.cc:343] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.751691  1775 tablet_bootstrap.cc:654] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.735863  1653 raft_consensus.cc:2468] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:35.744081  1655 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 2.
I20260709 14:50:35.763211  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.752013  2125 leader_election.cc:290] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.770990  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:35.773767  1549 leader_election.cc:343] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.774506  1549 leader_election.cc:304] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:35.775820  1774 tablet_bootstrap.cc:654] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.775947  1826 raft_consensus.cc:2749] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.780350  2048 raft_consensus.cc:515] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.782961  1963 raft_consensus.cc:2749] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.787832  1777 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:35.796697  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:35.799826  1777 tablet_bootstrap.cc:654] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.799292  2048 leader_election.cc:290] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:35.813002  1774 tablet_bootstrap.cc:492] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:35.813616  1774 ts_tablet_manager.cc:1403] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.366s	user 0.021s	sys 0.008s
I20260709 14:50:35.815788  1971 raft_consensus.cc:2749] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:35.824846  1774 raft_consensus.cc:359] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.836664  1774 raft_consensus.cc:385] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.838528  1774 raft_consensus.cc:740] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.840816  1774 consensus_queue.cc:260] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.876641  1774 ts_tablet_manager.cc:1434] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.063s	user 0.010s	sys 0.001s
I20260709 14:50:35.884992  1774 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:35.890563  2026 raft_consensus.cc:493] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.891304  2026 raft_consensus.cc:515] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.893594  2026 leader_election.cc:290] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:35.900625  1775 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:35.900841  1963 raft_consensus.cc:493] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.901326  1775 ts_tablet_manager.cc:1403] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.288s	user 0.025s	sys 0.018s
I20260709 14:50:35.901423  1963 raft_consensus.cc:515] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.903878  1963 leader_election.cc:290] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:35.908299  2057 consensus_queue.cc:1048] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:35.911044  1775 raft_consensus.cc:359] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.915305  1774 tablet_bootstrap.cc:654] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:35.911926  1548 leader_election.cc:304] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.921056  1775 raft_consensus.cc:385] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:35.921465  1775 raft_consensus.cc:740] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:35.922160  2086 consensus_queue.cc:1048] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:35.923110  1729 raft_consensus.cc:3060] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.923842  1777 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:35.924374  1777 ts_tablet_manager.cc:1403] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.012s
I20260709 14:50:35.925098  1697 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:35.926889  1826 raft_consensus.cc:2804] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:35.927416  1826 raft_consensus.cc:697] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:35.928205  1826 consensus_queue.cc:237] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.929411  2086 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:35.940790  2048 raft_consensus.cc:493] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:35.941084  1775 consensus_queue.cc:260] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.948264  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.949458  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b7cdd0e0a24e54b17023ff094ef854" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:35.950067  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.939488  1963 consensus_queue.cc:1048] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 14:50:35.953430  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610d6977f8694bfe8fcccaa5d37d222a" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:35.953562  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b7cdd0e0a24e54b17023ff094ef854" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:35.954259  1581 raft_consensus.cc:3060] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:35.941416  2048 raft_consensus.cc:515] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.957355  1775 ts_tablet_manager.cc:1434] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 14:50:35.962817  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5242c2facbbb46338963db64018b812e" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:35.964035  1651 raft_consensus.cc:2393] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:35.967626  1775 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:35.941841  2086 raft_consensus.cc:697] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
W20260709 14:50:35.976657  1624 leader_election.cc:343] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:35.980787  1624 leader_election.cc:343] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.981312  1624 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.990545  1775 tablet_bootstrap.cc:654] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:35.993353  1624 leader_election.cc:343] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.993956  1624 leader_election.cc:304] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:35.979239  2127 consensus_queue.cc:1048] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 14:50:35.991506  2086 consensus_queue.cc:237] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:35.992434  2092 raft_consensus.cc:2749] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.002100  1812 raft_consensus.cc:493] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.002671  1812 raft_consensus.cc:515] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:36.004123  1624 leader_election.cc:343] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:35.995522  2048 leader_election.cc:290] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.003158  2020 raft_consensus.cc:2749] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.018821  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:36.036857  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:36.070056  1623 leader_election.cc:343] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:36.081463  1624 leader_election.cc:343] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.080698  1777 raft_consensus.cc:359] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.085858  1812 leader_election.cc:290] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.086272  1777 raft_consensus.cc:385] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.085906  1624 leader_election.cc:304] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:36.086933  1777 raft_consensus.cc:740] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.088311  1777 consensus_queue.cc:260] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.089205  2122 raft_consensus.cc:2749] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.106637  1777 ts_tablet_manager.cc:1434] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.182s	user 0.011s	sys 0.001s
I20260709 14:50:36.107982  1697 leader_election.cc:304] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:36.111181  1697 leader_election.cc:304] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:36.113590  1797 raft_consensus.cc:2749] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.118251  1774 tablet_bootstrap.cc:492] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:36.118875  1774 ts_tablet_manager.cc:1403] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.235s	user 0.011s	sys 0.016s
I20260709 14:50:36.120733  1777 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
W20260709 14:50:36.131840  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 126812us
I20260709 14:50:36.147794  2131 raft_consensus.cc:3060] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.152139  1483 catalog_manager.cc:5697] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:36.156623  1826 raft_consensus.cc:493] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.157225  1826 raft_consensus.cc:515] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:36.178087  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 171569us
I20260709 14:50:36.150070  1581 raft_consensus.cc:2468] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:36.180979  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:36.182123  1826 leader_election.cc:290] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.183305  1774 raft_consensus.cc:359] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.184048  1774 raft_consensus.cc:385] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.186734  1774 raft_consensus.cc:740] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.189033  1774 consensus_queue.cc:260] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.192376  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:36.193246  1652 raft_consensus.cc:2468] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:36.207839  1774 ts_tablet_manager.cc:1434] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 14:50:36.195668  1733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:36.222298  1548 leader_election.cc:304] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
W20260709 14:50:36.229331  1549 leader_election.cc:343] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.233858  1774 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:36.227778  2123 raft_consensus.cc:493] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.251813  2123 raft_consensus.cc:515] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.265074  2123 leader_election.cc:290] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.274200  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:36.284484  2148 raft_consensus.cc:2804] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:36.285352  2148 raft_consensus.cc:493] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:36.285877  2148 raft_consensus.cc:3060] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:36.300804  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 78774.7us
I20260709 14:50:36.305778  1623 leader_election.cc:304] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
W20260709 14:50:36.312785  2143 log.cc:927] Time spent T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 14:50:36.314273  2122 raft_consensus.cc:2804] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:36.343202  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:36.344112  1658 raft_consensus.cc:2468] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:36.299134  1728 raft_consensus.cc:2468] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
W20260709 14:50:36.351997  1624 leader_election.cc:343] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.327800  2122 raft_consensus.cc:697] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:36.360893  1548 leader_election.cc:304] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:36.362571  1774 tablet_bootstrap.cc:654] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:36.364166  1826 raft_consensus.cc:2804] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:36.364603  1826 raft_consensus.cc:493] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:36.364925  1826 raft_consensus.cc:3060] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.365780  2131 raft_consensus.cc:2749] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.358789  2122 consensus_queue.cc:237] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.380227  1826 raft_consensus.cc:515] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.386919  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:36.387771  1576 raft_consensus.cc:2468] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:36.389974  1623 leader_election.cc:304] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:36.398545  1777 tablet_bootstrap.cc:654] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:36.399353  2129 consensus_queue.cc:1048] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:36.403169  1775 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:36.403800  1775 ts_tablet_manager.cc:1403] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.436s	user 0.044s	sys 0.016s
I20260709 14:50:36.397127  2123 raft_consensus.cc:2804] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:36.416543  1826 leader_election.cc:290] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.415776  2123 raft_consensus.cc:493] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:36.434896  1775 raft_consensus.cc:359] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.435751  1775 raft_consensus.cc:385] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.436041  1775 raft_consensus.cc:740] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.439716  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:36.433907  2123 raft_consensus.cc:3060] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.448568  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:36.449350  1658 raft_consensus.cc:3060] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:36.451005  1549 leader_election.cc:343] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:36.467368  2157 log.cc:927] Time spent T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 14:50:36.444074  1775 consensus_queue.cc:260] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.491411  1826 consensus_queue.cc:1048] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:36.476723  2148 raft_consensus.cc:515] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.517148  1775 ts_tablet_manager.cc:1434] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.002s
I20260709 14:50:36.521340  1774 tablet_bootstrap.cc:492] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:36.522001  1774 ts_tablet_manager.cc:1403] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.290s	user 0.010s	sys 0.013s
W20260709 14:50:36.529371  2159 log.cc:927] Time spent T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 14:50:36.537601  2160 log.cc:927] Time spent T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 14:50:36.544222  1775 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:36.563530  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:36.564332  1660 raft_consensus.cc:3060] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.566337  1774 raft_consensus.cc:359] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.571689  1774 raft_consensus.cc:385] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.572167  1774 raft_consensus.cc:740] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.574075  1774 consensus_queue.cc:260] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.577919  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f577518d474c0782bcd1edaefde746" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:36.578708  1728 raft_consensus.cc:3060] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.566316  2148 leader_election.cc:290] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.580856  1658 raft_consensus.cc:2468] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:36.593927  1774 ts_tablet_manager.cc:1434] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.004s
I20260709 14:50:36.594470  1548 leader_election.cc:304] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:36.597836  1777 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:36.598489  1777 ts_tablet_manager.cc:1403] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.478s	user 0.023s	sys 0.006s
I20260709 14:50:36.597898  1826 raft_consensus.cc:2804] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:36.601068  1826 raft_consensus.cc:697] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:36.601801  1826 consensus_queue.cc:237] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.607360  1774 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:36.609004  1658 raft_consensus.cc:1275] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Refusing update from remote peer 36d78753ff96438e9cc2808f05853294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:36.621718  2131 consensus_queue.cc:1048] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:36.625048  1777 raft_consensus.cc:359] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.625900  1777 raft_consensus.cc:385] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.626240  1777 raft_consensus.cc:740] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.627074  1777 consensus_queue.cc:260] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.600719  2123 raft_consensus.cc:515] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.644070  2129 raft_consensus.cc:493] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.646704  1777 ts_tablet_manager.cc:1434] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 14:50:36.644682  2129 raft_consensus.cc:515] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.651191  2129 leader_election.cc:290] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.656231  1576 raft_consensus.cc:1275] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Refusing update from remote peer 36d78753ff96438e9cc2808f05853294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:50:36.657667  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:36.663574  2138 consensus_queue.cc:1048] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 14:50:36.663300  1658 raft_consensus.cc:2468] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:36.654675  1731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:36.669308  1548 leader_election.cc:304] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:36.671608  2147 raft_consensus.cc:2804] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:36.666698  2122 raft_consensus.cc:493] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.672151  2147 raft_consensus.cc:493] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:36.672595  2147 raft_consensus.cc:3060] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.672442  2122 raft_consensus.cc:515] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.673518  1777 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:36.676370  1482 catalog_manager.cc:5697] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 2, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:36.686026  2122 leader_election.cc:290] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.686498  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:36.687913  1549 leader_election.cc:343] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:36.688592  1623 leader_election.cc:343] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.690001  1775 tablet_bootstrap.cc:654] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:36.692854  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:36.710309  1774 tablet_bootstrap.cc:654] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:36.720105  1624 leader_election.cc:343] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.720947  1624 leader_election.cc:304] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:36.712831  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:36.731402  1576 raft_consensus.cc:3060] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:36.738266  2092 raft_consensus.cc:2749] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:36.739547  1728 raft_consensus.cc:2468] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:36.741807  1549 leader_election.cc:304] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:36.742960  1826 raft_consensus.cc:2804] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:36.746312  1826 raft_consensus.cc:697] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:36.747519  1826 consensus_queue.cc:237] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.711788  1660 raft_consensus.cc:2468] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:36.710922  1483 catalog_manager.cc:5697] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:36.765020  2123 leader_election.cc:290] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:36.767287  1777 tablet_bootstrap.cc:654] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:36.767206  1624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 47176.4us
I20260709 14:50:36.798367  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:36.817020  1576 raft_consensus.cc:2468] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:36.818279  1623 leader_election.cc:304] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:36.818429  2123 consensus_queue.cc:1048] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:36.819448  2092 raft_consensus.cc:2804] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 14:50:36.824008  1624 leader_election.cc:343] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:36.961062  1774 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:36.961742  1774 ts_tablet_manager.cc:1403] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.355s	user 0.026s	sys 0.029s
I20260709 14:50:36.970006  1775 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:36.970660  1775 ts_tablet_manager.cc:1403] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.428s	user 0.015s	sys 0.026s
I20260709 14:50:36.976487  1777 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:36.977242  1777 ts_tablet_manager.cc:1403] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.318s	user 0.012s	sys 0.017s
I20260709 14:50:36.980695  1777 raft_consensus.cc:359] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.981467  1777 raft_consensus.cc:385] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.981745  1777 raft_consensus.cc:740] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
W20260709 14:50:36.983098  1548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44999, user_credentials={real_user=slave}} blocked reactor thread for 157152us
I20260709 14:50:36.970733  2147 raft_consensus.cc:515] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.985895  1774 raft_consensus.cc:359] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.986671  1774 raft_consensus.cc:385] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:36.986990  1774 raft_consensus.cc:740] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:36.987646  1774 consensus_queue.cc:260] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.989382  1774 ts_tablet_manager.cc:1434] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 14:50:36.984251  1962 raft_consensus.cc:493] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:36.990473  1774 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:36.990525  1962 raft_consensus.cc:515] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:36.992864  1962 leader_election.cc:290] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:36.994678  2147 leader_election.cc:290] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:37.000298  1774 tablet_bootstrap.cc:654] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.007961  2127 raft_consensus.cc:493] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.008652  2127 raft_consensus.cc:515] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.010855  2127 leader_election.cc:290] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:37.012181  1775 raft_consensus.cc:359] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.013281  1775 raft_consensus.cc:385] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.013631  1775 raft_consensus.cc:740] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.014483  1775 consensus_queue.cc:260] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.021553  1775 ts_tablet_manager.cc:1434] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
W20260709 14:50:37.025799  1624 consensus_peers.cc:597] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Couldn't send request to peer 36d78753ff96438e9cc2808f05853294. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:37.025897  1775 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:37.024096  2092 raft_consensus.cc:697] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:37.030337  2092 consensus_queue.cc:237] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:37.038722  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 14:50:37.040685  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 51571.1us
I20260709 14:50:37.049077  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:37.052637  1777 consensus_queue.cc:260] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.053846  1581 raft_consensus.cc:2468] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:37.054512  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:37.060532  1777 ts_tablet_manager.cc:1434] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 14:50:37.061903  1777 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:37.072701  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:37.073729  1651 raft_consensus.cc:2393] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
W20260709 14:50:37.078579  1624 leader_election.cc:343] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.091940  1623 leader_election.cc:304] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:37.091701  1483 catalog_manager.cc:5697] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:37.093358  2175 raft_consensus.cc:2804] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:37.093747  2175 raft_consensus.cc:493] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:37.094100  2175 raft_consensus.cc:3060] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.095551  1775 tablet_bootstrap.cc:654] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.102945  1774 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:37.104043  1774 ts_tablet_manager.cc:1403] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.114s	user 0.040s	sys 0.058s
I20260709 14:50:37.105562  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630ab4f3cb154a0bb2fa8c41fbe95434" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:37.115367  1774 raft_consensus.cc:359] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:37.130167  1624 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 43749.1us
I20260709 14:50:37.130693  1774 raft_consensus.cc:385] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.131508  1774 raft_consensus.cc:740] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.115154  2175 raft_consensus.cc:515] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.136988  1774 consensus_queue.cc:260] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.138144  2175 leader_election.cc:290] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:37.117419  1777 tablet_bootstrap.cc:654] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.105780  1728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
W20260709 14:50:37.139735  1549 leader_election.cc:343] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.141014  1774 ts_tablet_manager.cc:1434] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 14:50:37.144480  1774 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:37.146418  1479 catalog_manager.cc:5697] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:37.149470  1479 catalog_manager.cc:5697] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:37.158578  1775 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:37.159267  1775 ts_tablet_manager.cc:1403] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.009s
I20260709 14:50:37.161932  1775 raft_consensus.cc:359] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.162655  1775 raft_consensus.cc:385] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.162655  1731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:37.163266  1777 tablet_bootstrap.cc:492] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:37.163779  1777 ts_tablet_manager.cc:1403] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.102s	user 0.005s	sys 0.008s
I20260709 14:50:37.163030  1775 raft_consensus.cc:740] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.164438  1697 leader_election.cc:304] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:37.167785  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08c454fd6fc4270ad8e5ca2542757b0" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:37.168602  1774 tablet_bootstrap.cc:654] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.176855  1775 consensus_queue.cc:260] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.176854  1651 raft_consensus.cc:3060] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.184371  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:37.185184  1578 raft_consensus.cc:3060] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.188138  1775 ts_tablet_manager.cc:1434] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 14:50:37.189113  2026 raft_consensus.cc:3060] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.190759  1651 raft_consensus.cc:2468] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:37.198390  2026 raft_consensus.cc:2749] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:37.176883  1777 raft_consensus.cc:359] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.206105  1777 raft_consensus.cc:385] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.206480  1777 raft_consensus.cc:740] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.207034  1775 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:37.207204  1777 consensus_queue.cc:260] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.209625  1777 ts_tablet_manager.cc:1434] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.001s
I20260709 14:50:37.214355  1774 tablet_bootstrap.cc:492] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:37.214928  1774 ts_tablet_manager.cc:1403] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.002s
I20260709 14:50:37.214056  1578 raft_consensus.cc:2468] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:37.217834  1777 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:37.217963  2056 consensus_queue.cc:1048] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:37.219468  2056 consensus_queue.cc:1048] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:37.223243  1623 leader_election.cc:304] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:37.223882  1775 tablet_bootstrap.cc:654] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.224174  2175 raft_consensus.cc:2804] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:37.226382  2175 raft_consensus.cc:697] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
W20260709 14:50:37.227398  1624 leader_election.cc:343] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.230880  2175 consensus_queue.cc:237] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.238696  2056 consensus_queue.cc:1048] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:37.239023  2147 consensus_queue.cc:1048] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:37.243183  1548 leader_election.cc:304] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 36d78753ff96438e9cc2808f05853294
I20260709 14:50:37.244638  2125 raft_consensus.cc:2804] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:37.249334  1774 raft_consensus.cc:359] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.250159  1774 raft_consensus.cc:385] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.250432  1774 raft_consensus.cc:740] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.255127  1777 tablet_bootstrap.cc:654] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.252096  1774 consensus_queue.cc:260] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.256947  2125 raft_consensus.cc:697] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:37.263348  2123 raft_consensus.cc:493] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.282675  1774 ts_tablet_manager.cc:1434] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.000s
I20260709 14:50:37.289142  1774 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:37.281597  2123 raft_consensus.cc:515] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.262897  2125 consensus_queue.cc:237] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.325135  2123 leader_election.cc:290] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:37.335860  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:37.338692  1623 leader_election.cc:343] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.334904  2086 raft_consensus.cc:493] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.335884  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:37.342221  2086 raft_consensus.cc:515] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:37.345700  1624 leader_election.cc:343] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.346150  1624 leader_election.cc:304] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:37.349114  2086 leader_election.cc:290] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:37.350229  1584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:37.350229  2092 raft_consensus.cc:2749] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:37.351815  1584 raft_consensus.cc:2468] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:37.360786  1774 tablet_bootstrap.cc:654] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.361510  1775 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:37.362128  1775 ts_tablet_manager.cc:1403] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.155s	user 0.016s	sys 0.012s
I20260709 14:50:37.358336  1697 leader_election.cc:304] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:37.364797  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:37.365792  1660 raft_consensus.cc:2468] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:37.366644  1482 catalog_manager.cc:5697] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:37.368934  2138 raft_consensus.cc:2804] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:37.373044  2138 raft_consensus.cc:493] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:37.373586  2138 raft_consensus.cc:3060] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.375855  1775 raft_consensus.cc:359] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.376639  1775 raft_consensus.cc:385] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.376942  1775 raft_consensus.cc:740] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.378578  1775 consensus_queue.cc:260] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.380975  1775 ts_tablet_manager.cc:1434] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 14:50:37.382107  1775 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: Bootstrap starting.
I20260709 14:50:37.399168  2086 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.405707  2086 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.408685  1962 raft_consensus.cc:493] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.409438  1962 raft_consensus.cc:515] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.573460  1479 catalog_manager.cc:5697] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:37.581537  1777 tablet_bootstrap.cc:492] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:37.582146  1777 ts_tablet_manager.cc:1403] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.365s	user 0.015s	sys 0.017s
I20260709 14:50:37.622036  1774 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:37.622699  1774 ts_tablet_manager.cc:1403] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.334s	user 0.019s	sys 0.011s
I20260709 14:50:37.628897  1962 leader_election.cc:290] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:37.645311  2138 raft_consensus.cc:515] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.670499  1774 raft_consensus.cc:359] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.687619  1774 raft_consensus.cc:385] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.688129  1774 raft_consensus.cc:740] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.697379  1774 consensus_queue.cc:260] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.708325  2177 raft_consensus.cc:493] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.709023  2177 raft_consensus.cc:515] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.709403  2138 leader_election.cc:290] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:37.710877  1775 tablet_bootstrap.cc:654] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:37.738024  1777 raft_consensus.cc:359] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.739075  2086 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:37.742748  1777 raft_consensus.cc:385] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:37.749070  2177 leader_election.cc:290] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:37.764801  1774 ts_tablet_manager.cc:1434] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.142s	user 0.020s	sys 0.011s
I20260709 14:50:37.766469  1774 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
W20260709 14:50:37.783332  1549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:38515, user_credentials={real_user=slave}} blocked reactor thread for 55630.1us
I20260709 14:50:37.794523  1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:37.746166  1777 raft_consensus.cc:740] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:37.797567  1731 raft_consensus.cc:3060] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.797756  1777 consensus_queue.cc:260] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:37.805507  1624 leader_election.cc:343] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:37.869153  1777 ts_tablet_manager.cc:1434] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.287s	user 0.008s	sys 0.003s
I20260709 14:50:37.858600  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:37.875478  1581 raft_consensus.cc:2393] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:37.879719  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6015635506e44421bfc21fc36bdb0191" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
W20260709 14:50:37.887656  1624 consensus_peers.cc:597] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Couldn't send request to peer 36d78753ff96438e9cc2808f05853294. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:37.866307  2127 raft_consensus.cc:493] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.905767  2127 raft_consensus.cc:515] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.863643  1657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:37.938395  2020 consensus_queue.cc:1048] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:37.954981  1581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e50386aa8642078375fbb0a9fb3f82" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:37.959151  1581 raft_consensus.cc:3060] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.940450  1657 raft_consensus.cc:3060] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:37.965837  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:37.955245  2147 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:37.969544  2147 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:37.979357  1576 raft_consensus.cc:2468] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:37.995798  1660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:37.996620  1660 raft_consensus.cc:2468] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:37.999140  2147 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.000989  1774 tablet_bootstrap.cc:654] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.002475  1775 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: No bootstrap required, opened a new log
I20260709 14:50:38.003050  1775 ts_tablet_manager.cc:1403] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: Time spent bootstrapping tablet: real 0.621s	user 0.037s	sys 0.053s
I20260709 14:50:38.004197  1777 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:38.013159  1697 leader_election.cc:304] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:38.018432  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:38.026912  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:38.027771  1725 raft_consensus.cc:2468] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.035604  2127 leader_election.cc:290] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.039368  1697 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.040292  2199 raft_consensus.cc:2804] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.040719  2199 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.041064  2199 raft_consensus.cc:3060] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:38.042268  1549 consensus_peers.cc:597] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe -> Peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Couldn't send request to peer 36d78753ff96438e9cc2808f05853294. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:38.044862  2127 consensus_queue.cc:1048] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.049116  2204 raft_consensus.cc:2749] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.056401  1549 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.067445  2210 raft_consensus.cc:2804] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.067937  2210 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.068291  2210 raft_consensus.cc:3060] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:38.076154  1623 leader_election.cc:343] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.076867  1623 leader_election.cc:304] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:38.070520  2200 raft_consensus.cc:493] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.082391  2200 raft_consensus.cc:515] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.083725  1775 raft_consensus.cc:359] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.087895  1775 raft_consensus.cc:385] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.088425  1775 raft_consensus.cc:740] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.089124  1581 raft_consensus.cc:2468] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.092110  1657 raft_consensus.cc:2468] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.083310  2127 raft_consensus.cc:2749] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.093215  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:38.095711  1623 leader_election.cc:343] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.097157  1775 consensus_queue.cc:260] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.100330  1697 leader_election.cc:304] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.105494  2203 raft_consensus.cc:2804] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.108387  1775 ts_tablet_manager.cc:1434] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: Time spent starting tablet: real 0.105s	user 0.009s	sys 0.001s
I20260709 14:50:38.111377  2203 raft_consensus.cc:697] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:38.129316  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
W20260709 14:50:38.131264  1531 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:38.131009  2203 consensus_queue.cc:237] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.137378  2200 leader_election.cc:290] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.137420  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.141111  1658 raft_consensus.cc:2468] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.146796  2127 consensus_queue.cc:1048] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.130983  2210 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.154307  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:38.166745  1624 leader_election.cc:343] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.167392  1624 leader_election.cc:304] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:38.173481  2048 raft_consensus.cc:2749] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.174268  2129 consensus_queue.cc:1048] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.183637  1777 tablet_bootstrap.cc:654] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.184504  1774 tablet_bootstrap.cc:492] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.187000  1774 ts_tablet_manager.cc:1403] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.421s	user 0.018s	sys 0.007s
W20260709 14:50:38.194329  1697 leader_election.cc:343] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.161963  2199 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.208947  1774 raft_consensus.cc:359] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.210225  1774 raft_consensus.cc:385] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.211213  1774 raft_consensus.cc:740] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.212723  1774 consensus_queue.cc:260] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.238445  1774 ts_tablet_manager.cc:1434] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.002s
I20260709 14:50:38.240541  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.241448  1652 raft_consensus.cc:3060] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.245055  1655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.295229  1774 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.299875  1585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:38.300791  1585 raft_consensus.cc:2393] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 36d78753ff96438e9cc2808f05853294 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:38.281442  2210 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.294492  1655 raft_consensus.cc:2468] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:38.308223  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.300405  2199 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.320075  1652 raft_consensus.cc:2468] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:38.310384  1726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fc279a847e48d5a240ecf6cc61c035" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:38.324057  1726 raft_consensus.cc:2393] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 36d78753ff96438e9cc2808f05853294 in this term.
W20260709 14:50:38.323408  2213 log.cc:927] Time spent T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: Append to log took a long time: real 0.182s	user 0.002s	sys 0.010s
I20260709 14:50:38.335711  1774 tablet_bootstrap.cc:654] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.339107  1479 catalog_manager.cc:5697] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:38.342772  1774 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.343436  1774 ts_tablet_manager.cc:1403] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.000s
I20260709 14:50:38.343333  1548 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.344745  1777 tablet_bootstrap.cc:492] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:38.345263  1777 ts_tablet_manager.cc:1403] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.343s	user 0.011s	sys 0.006s
I20260709 14:50:38.347388  1697 leader_election.cc:304] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:38.348235  2026 raft_consensus.cc:2804] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.349800  2026 raft_consensus.cc:493] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.350975  2026 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.348168  2133 raft_consensus.cc:2804] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.348596  1697 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:38.352008  2133 raft_consensus.cc:697] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:38.352938  1962 raft_consensus.cc:2749] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.352921  2133 consensus_queue.cc:237] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.360679  2026 raft_consensus.cc:515] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.370724  2026 leader_election.cc:290] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.378201  1774 raft_consensus.cc:359] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.382723  1774 raft_consensus.cc:385] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.392619  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:38.383148  1774 raft_consensus.cc:740] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.396694  1774 consensus_queue.cc:260] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.417936  1774 ts_tablet_manager.cc:1434] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.000s
I20260709 14:50:38.395248  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.436560  1774 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.435000  1652 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.456470  1777 raft_consensus.cc:359] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.457262  1777 raft_consensus.cc:385] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.457530  1777 raft_consensus.cc:740] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.458268  1777 consensus_queue.cc:260] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.460068  1777 ts_tablet_manager.cc:1434] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.115s	user 0.008s	sys 0.000s
I20260709 14:50:38.460836  1577 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.461573  1652 raft_consensus.cc:2468] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.479760  1777 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:38.487756  1732 raft_consensus.cc:3060] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.488425  1577 raft_consensus.cc:2468] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.495190  1774 tablet_bootstrap.cc:654] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.526803  1777 tablet_bootstrap.cc:654] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.528467  1697 leader_election.cc:304] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.532932  2204 raft_consensus.cc:2804] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.533533  2204 raft_consensus.cc:697] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:38.534461  2204 consensus_queue.cc:237] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.546307  2177 raft_consensus.cc:493] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.547001  2177 raft_consensus.cc:515] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.546051  1479 catalog_manager.cc:5697] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:38.550181  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:38.550807  2177 leader_election.cc:290] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
W20260709 14:50:38.554952  1623 leader_election.cc:343] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.560962  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
W20260709 14:50:38.564383  1624 leader_election.cc:343] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:38.565878  1624 leader_election.cc:304] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294
I20260709 14:50:38.568179  2177 raft_consensus.cc:2749] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.580168  1774 tablet_bootstrap.cc:492] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.580807  1774 ts_tablet_manager.cc:1403] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.145s	user 0.022s	sys 0.011s
I20260709 14:50:38.583844  1774 raft_consensus.cc:359] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.584762  1774 raft_consensus.cc:385] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.585094  1774 raft_consensus.cc:740] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.585789  1774 consensus_queue.cc:260] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.589275  1774 ts_tablet_manager.cc:1434] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 14:50:38.590394  1774 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.592941  1962 consensus_queue.cc:1048] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:38.595634  1777 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:38.596190  1777 ts_tablet_manager.cc:1403] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.005s
I20260709 14:50:38.599632  1777 raft_consensus.cc:359] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.600368  1777 raft_consensus.cc:385] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.600802  1777 raft_consensus.cc:740] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.601141  1774 tablet_bootstrap.cc:654] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.601594  1777 consensus_queue.cc:260] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.600270  1479 catalog_manager.cc:5697] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:38.610625  1777 ts_tablet_manager.cc:1434] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 14:50:38.612658  1777 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:38.632108  1777 tablet_bootstrap.cc:654] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.645282  1725 raft_consensus.cc:3060] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.646554  1774 tablet_bootstrap.cc:492] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.647176  1774 ts_tablet_manager.cc:1403] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.015s
I20260709 14:50:38.649588  1777 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:38.650123  1777 ts_tablet_manager.cc:1403] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.004s
I20260709 14:50:38.650033  1774 raft_consensus.cc:359] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.650818  1774 raft_consensus.cc:385] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.651160  1774 raft_consensus.cc:740] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.651844  1774 consensus_queue.cc:260] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.653596  1777 raft_consensus.cc:359] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.654244  1774 ts_tablet_manager.cc:1434] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:50:38.654350  1777 raft_consensus.cc:385] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.654951  1777 raft_consensus.cc:740] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.655514  1774 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.656093  1777 consensus_queue.cc:260] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.658427  1777 ts_tablet_manager.cc:1434] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 14:50:38.659056  2195 consensus_queue.cc:1048] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.661527  1777 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:38.668717  1777 tablet_bootstrap.cc:654] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.669251  1774 tablet_bootstrap.cc:654] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.711364  1777 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:38.711905  1777 ts_tablet_manager.cc:1403] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.004s
I20260709 14:50:38.719728  1777 raft_consensus.cc:359] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.720530  1777 raft_consensus.cc:385] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.720835  1777 raft_consensus.cc:740] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.721540  1777 consensus_queue.cc:260] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.724664  1774 tablet_bootstrap.cc:492] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.729007  1774 ts_tablet_manager.cc:1403] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.074s	user 0.024s	sys 0.017s
I20260709 14:50:38.730235  1777 ts_tablet_manager.cc:1434] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 14:50:38.732117  1774 raft_consensus.cc:359] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.732846  1774 raft_consensus.cc:385] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.733111  1774 raft_consensus.cc:740] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.733794  1774 consensus_queue.cc:260] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.734516  1777 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: Bootstrap starting.
I20260709 14:50:38.736975  1774 ts_tablet_manager.cc:1434] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 14:50:38.740392  1774 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.748461  1774 tablet_bootstrap.cc:654] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.755709  1777 tablet_bootstrap.cc:654] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.780835  1777 tablet_bootstrap.cc:492] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: No bootstrap required, opened a new log
I20260709 14:50:38.781409  1777 ts_tablet_manager.cc:1403] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.010s
I20260709 14:50:38.786700  1777 raft_consensus.cc:359] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.787539  1777 raft_consensus.cc:385] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.787830  1777 raft_consensus.cc:740] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.788638  1774 tablet_bootstrap.cc:492] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.788638  1777 consensus_queue.cc:260] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.789535  1774 ts_tablet_manager.cc:1403] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.017s
I20260709 14:50:38.791040  1777 ts_tablet_manager.cc:1434] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:50:38.792533  1774 raft_consensus.cc:359] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.793203  1774 raft_consensus.cc:385] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.793485  1774 raft_consensus.cc:740] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.794505  1774 consensus_queue.cc:260] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.798026  1774 ts_tablet_manager.cc:1434] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:50:38.799592  2195 raft_consensus.cc:493] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.800477  1774 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: Bootstrap starting.
I20260709 14:50:38.806425  2227 raft_consensus.cc:493] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.807020  2227 raft_consensus.cc:515] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.806814  2195 raft_consensus.cc:515] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.808461  2056 consensus_queue.cc:1048] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.810027  2227 leader_election.cc:290] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.811791  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:38.815410  1774 tablet_bootstrap.cc:654] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:38.817898  1654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.818686  1654 raft_consensus.cc:2468] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.820674  1548 leader_election.cc:304] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.822198  2056 raft_consensus.cc:2804] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.822650  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:38.823635  1725 raft_consensus.cc:2468] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.827199  1826 raft_consensus.cc:493] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.828001  1826 raft_consensus.cc:515] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.822654  2056 raft_consensus.cc:493] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.834594  2056 raft_consensus.cc:3060] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.837333  1774 tablet_bootstrap.cc:492] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: No bootstrap required, opened a new log
I20260709 14:50:38.837929  1774 ts_tablet_manager.cc:1403] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.009s
I20260709 14:50:38.840958  1826 leader_election.cc:290] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.843366  1774 raft_consensus.cc:359] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.848134  1774 raft_consensus.cc:385] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:38.848450  1774 raft_consensus.cc:740] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Initialized, Role: FOLLOWER
I20260709 14:50:38.844053  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:38.849243  1774 consensus_queue.cc:260] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.851502  1774 ts_tablet_manager.cc:1434] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 14:50:38.852006  1732 raft_consensus.cc:2468] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.854781  2201 consensus_queue.cc:1048] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:38.844223  2195 leader_election.cc:290] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.845479  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806dd19b6e18429c8743b9aceda41aa5" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.845453  1654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.858321  1654 raft_consensus.cc:2468] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.859488  1549 leader_election.cc:304] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.864718  2227 raft_consensus.cc:2804] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.864709  1697 leader_election.cc:304] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:38.865386  2227 raft_consensus.cc:493] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.865800  2227 raft_consensus.cc:3060] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.866231  2195 raft_consensus.cc:2749] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:38.868912  2056 raft_consensus.cc:515] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.877465  2056 leader_election.cc:290] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.877863  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.878782  1652 raft_consensus.cc:3060] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.880378  2227 raft_consensus.cc:515] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.884784  1732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b689c1f7b39446488c097ffeb75f608c" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:38.885915  2227 leader_election.cc:290] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.886161  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:38.885679  1732 raft_consensus.cc:3060] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.886863  1725 raft_consensus.cc:3060] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.891561  1658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35708989e3c345b3a0bad8e5ecebc998" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.893075  1658 raft_consensus.cc:3060] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.893177  2195 raft_consensus.cc:493] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.893671  2195 raft_consensus.cc:515] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.899111  1732 raft_consensus.cc:2468] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:38.901918  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:38.902763  1580 raft_consensus.cc:2468] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:38.904810  2195 leader_election.cc:290] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.910204  1652 raft_consensus.cc:2468] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:38.906935  1549 leader_election.cc:304] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.912420  2227 raft_consensus.cc:2804] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.913028  2227 raft_consensus.cc:697] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:38.913928  2227 consensus_queue.cc:237] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.919657  1697 leader_election.cc:304] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.921213  2194 raft_consensus.cc:2804] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.921762  2194 raft_consensus.cc:493] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.922235  2194 raft_consensus.cc:3060] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.930693  2194 raft_consensus.cc:515] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.933288  2194 leader_election.cc:290] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:38.939091  1580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:38.939997  1580 raft_consensus.cc:3060] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.940557  1654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.941252  1654 raft_consensus.cc:2468] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 0.
I20260709 14:50:38.945178  1725 raft_consensus.cc:2468] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:38.945849  1654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc051130230a45eba628e68c39f0cda3" candidate_uuid: "36d78753ff96438e9cc2808f05853294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:38.946461  1654 raft_consensus.cc:3060] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.946614  1549 leader_election.cc:304] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 36d78753ff96438e9cc2808f05853294; no voters: 
I20260709 14:50:38.948410  2056 raft_consensus.cc:2804] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.949255  2227 raft_consensus.cc:493] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.948900  2056 raft_consensus.cc:697] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Leader. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:38.949766  2227 raft_consensus.cc:515] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.907078  1658 raft_consensus.cc:2468] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 1.
I20260709 14:50:38.951354  2056 consensus_queue.cc:237] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.955839  2092 raft_consensus.cc:493] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:38.960206  2092 raft_consensus.cc:515] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.962637  2092 leader_election.cc:290] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.963635  1479 catalog_manager.cc:5697] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:38.968380  2227 leader_election.cc:290] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:38.971305  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:38.971767  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3" is_pre_election: true
I20260709 14:50:38.975071  1654 raft_consensus.cc:2468] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.976816  1652 raft_consensus.cc:2468] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.977758  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294" is_pre_election: true
I20260709 14:50:38.978873  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe" is_pre_election: true
I20260709 14:50:38.979597  1577 raft_consensus.cc:2468] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 0.
I20260709 14:50:38.980965  1623 leader_election.cc:304] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.982594  2248 raft_consensus.cc:2804] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.983229  2248 raft_consensus.cc:493] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.983623  2248 raft_consensus.cc:3060] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.986114  1697 leader_election.cc:304] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.986167  1548 leader_election.cc:304] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:38.987067  2194 raft_consensus.cc:2804] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:38.988103  2056 raft_consensus.cc:2804] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:38.988546  2056 raft_consensus.cc:493] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:38.988749  2194 raft_consensus.cc:697] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Leader. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:38.989033  2056 raft_consensus.cc:3060] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:38.979636  1725 raft_consensus.cc:2468] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e506fd82284f2592f156640c0edafe in term 0.
I20260709 14:50:38.989921  2194 consensus_queue.cc:237] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:38.993041  1580 raft_consensus.cc:2468] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d78753ff96438e9cc2808f05853294 in term 1.
I20260709 14:50:38.998593  2248 raft_consensus.cc:515] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:39.002136  2248 leader_election.cc:290] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:39.002353  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:39.003077  1727 raft_consensus.cc:3060] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:39.006440  2056 raft_consensus.cc:515] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:39.009641  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:39.010676  1577 raft_consensus.cc:2393] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45d5272855ec440cbd07d4d3995c68d3 in current term 1: Already voted for candidate 17e506fd82284f2592f156640c0edafe in this term.
I20260709 14:50:39.011933  1652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
I20260709 14:50:39.012878  1652 raft_consensus.cc:2393] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:39.013091  1727 raft_consensus.cc:2468] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 1.
I20260709 14:50:39.014079  2056 leader_election.cc:290] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Requested vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:39.015236  1479 catalog_manager.cc:5697] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e506fd82284f2592f156640c0edafe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "17e506fd82284f2592f156640c0edafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:39.016033  1624 leader_election.cc:304] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 17e506fd82284f2592f156640c0edafe
I20260709 14:50:39.016033  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e44a51657244bdac12c9e76cc53017" candidate_uuid: "17e506fd82284f2592f156640c0edafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:39.017100  1725 raft_consensus.cc:2393] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17e506fd82284f2592f156640c0edafe in current term 1: Already voted for candidate 45d5272855ec440cbd07d4d3995c68d3 in this term.
I20260709 14:50:39.018471  1549 leader_election.cc:304] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17e506fd82284f2592f156640c0edafe; no voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:39.020313  2056 raft_consensus.cc:2749] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:50:39.019655  1483 catalog_manager.cc:5697] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d78753ff96438e9cc2808f05853294 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "36d78753ff96438e9cc2808f05853294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:39.022835  2247 raft_consensus.cc:2804] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:39.023459  2247 raft_consensus.cc:697] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:39.024389  2247 consensus_queue.cc:237] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:39.037010  2194 consensus_queue.cc:1048] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:39.070989  1727 raft_consensus.cc:3060] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:39.065783  1483 catalog_manager.cc:5697] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:39.085891  2194 consensus_queue.cc:1048] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:39.346278  1826 consensus_queue.cc:1048] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:39.457545  2227 consensus_queue.cc:1048] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:50:39.513942  2227 consensus_queue.cc:1048] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:39.556073  2227 consensus_queue.cc:1048] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:50:39.611212  2200 consensus_queue.cc:1048] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:39.628773  2247 consensus_queue.cc:1048] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 14:50:39.645606  2195 consensus_queue.cc:1048] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:39.656055  2247 consensus_queue.cc:1048] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:40.742763   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:40.769326  2266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:40.775305  2269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:40.780865  2267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:40.781208   431 server_base.cc:1061] running on GCE node
I20260709 14:50:40.785650   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:40.786087   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:40.786424   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608640786392 us; error 0 us; skew 500 ppm
I20260709 14:50:40.787721   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:40.792675   431 webserver.cc:533] Webserver started at http://127.0.107.196:46093/ using document root <none> and password file <none>
I20260709 14:50:40.793815   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:40.794152   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:40.794625   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:40.796413   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-3-root/instance:
uuid: "764cd4e5270e45729c440d9e5b1dc714"
format_stamp: "Formatted at 2026-07-09 14:50:40 on dist-test-slave-wg8n"
I20260709 14:50:40.802392   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 14:50:40.807212  2274 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:40.809736   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 14:50:40.810209   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-3-root
uuid: "764cd4e5270e45729c440d9e5b1dc714"
format_stamp: "Formatted at 2026-07-09 14:50:40 on dist-test-slave-wg8n"
I20260709 14:50:40.810590   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783608598766170-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:40.833027   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:40.835122   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:40.837060   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:40.839978   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:40.840366   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:40.840708   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:40.841009   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:40.935329   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:40499
I20260709 14:50:40.935478  2337 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:40499 every 8 connection(s)
I20260709 14:50:40.986574   431 auto_leader_rebalancer-test.cc:446] add a tserver: 764cd4e5270e45729c440d9e5b1dc714
I20260709 14:50:41.002075  2338 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38079
I20260709 14:50:41.002642  2338 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:41.003650  2338 heartbeater.cc:507] Master 127.0.107.254:38079 requested a full tablet report, sending...
I20260709 14:50:41.007777  1479 ts_manager.cc:194] Registered new tserver with Master: 764cd4e5270e45729c440d9e5b1dc714 (127.0.107.196:40499)
I20260709 14:50:41.009930  1479 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57276
I20260709 14:50:42.013394  2338 heartbeater.cc:499] Master 127.0.107.254:38079 was elected leader, sending a full tablet report...
W20260709 14:50:45.839574  1550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.002s	sys 0.007s
W20260709 14:50:45.860548  1550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.003s	sys 0.007s
I20260709 14:50:46.843495   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:50:46.852190  1755 debug-util.cc:398] Leaking SignalData structure 0x7b080019b280 after lost signal to thread 434
W20260709 14:50:46.853513  1755 debug-util.cc:398] Leaking SignalData structure 0x7b080016db00 after lost signal to thread 1517
W20260709 14:50:46.854969  1755 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7da0 after lost signal to thread 1609
W20260709 14:50:46.856333  1755 debug-util.cc:398] Leaking SignalData structure 0x7b08002d8260 after lost signal to thread 1684
W20260709 14:50:46.859128  1755 debug-util.cc:398] Leaking SignalData structure 0x7b08002d88a0 after lost signal to thread 1758
W20260709 14:50:46.860466  1755 debug-util.cc:398] Leaking SignalData structure 0x7b0800272e80 after lost signal to thread 2337
I20260709 14:50:46.990151  1583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50"
dest_uuid: "17e506fd82284f2592f156640c0edafe"
mode: GRACEFUL
new_leader_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:54030
I20260709 14:50:46.991811  1583 raft_consensus.cc:606] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Received request to transfer leadership to TS 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:47.000135  1583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9203095ce82145248458d98a71f8f07f"
dest_uuid: "17e506fd82284f2592f156640c0edafe"
mode: GRACEFUL
new_leader_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:54030
I20260709 14:50:47.000705  1583 raft_consensus.cc:606] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Received request to transfer leadership to TS 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:47.019811  1727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca"
dest_uuid: "36d78753ff96438e9cc2808f05853294"
mode: GRACEFUL
new_leader_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:50862
I20260709 14:50:47.020555  1727 raft_consensus.cc:606] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Received request to transfer leadership to TS 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:47.026626  1727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22f5227c90341148049d7ab114e5272"
dest_uuid: "36d78753ff96438e9cc2808f05853294"
mode: GRACEFUL
new_leader_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:50862
I20260709 14:50:47.027503  1727 raft_consensus.cc:606] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Received request to transfer leadership to TS 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:47.028822   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 14:50:47.030642   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:50:47.199157  2366 raft_consensus.cc:993] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: : Instructing follower 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.199800  2356 raft_consensus.cc:1081] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Signalling peer 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.201882  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f22f5227c90341148049d7ab114e5272"
dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:38626
I20260709 14:50:47.202504  1651 raft_consensus.cc:493] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:47.202894  1651 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:47.209432  1651 raft_consensus.cc:515] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.212261  1651 leader_election.cc:290] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 3 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:47.213240  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:47.213366  1727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22f5227c90341148049d7ab114e5272" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:47.214187  1583 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:47.214207  1727 raft_consensus.cc:3055] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:50:47.214656  1727 raft_consensus.cc:740] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:47.215588  1727 consensus_queue.cc:260] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.216998  1727 raft_consensus.cc:3060] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:50:47.222390  1583 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 3.
I20260709 14:50:47.224304  1623 leader_election.cc:304] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:47.225517  2376 raft_consensus.cc:2804] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:50:47.228008  2376 raft_consensus.cc:697] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 3 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:47.229919  1727 raft_consensus.cc:2468] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 3.
I20260709 14:50:47.229094  2376 consensus_queue.cc:237] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.242082  2359 raft_consensus.cc:993] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: : Instructing follower 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.245443  2355 raft_consensus.cc:1081] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Signalling peer 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.247766  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9203095ce82145248458d98a71f8f07f"
dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:38616
I20260709 14:50:47.248854  1651 raft_consensus.cc:493] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:47.250051  1651 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.255151  1479 catalog_manager.cc:5697] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 2 to 3, leader changed from 36d78753ff96438e9cc2808f05853294 (127.0.107.195) to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:47.259438  1651 raft_consensus.cc:515] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.263540  1583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:47.264415  1651 leader_election.cc:290] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:47.265829  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9203095ce82145248458d98a71f8f07f" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:47.266248  1583 raft_consensus.cc:3055] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:47.266553  1725 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.266659  1583 raft_consensus.cc:740] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:47.268242  1583 consensus_queue.cc:260] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.270035  1583 raft_consensus.cc:3060] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.275942  1725 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.278050  1624 leader_election.cc:304] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d78753ff96438e9cc2808f05853294, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:47.279075  1583 raft_consensus.cc:2468] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.281898  2380 raft_consensus.cc:2804] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:47.282506  2380 raft_consensus.cc:697] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:47.283496  2380 consensus_queue.cc:237] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.332257  1479 catalog_manager.cc:5697] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 1 to 2, leader changed from 17e506fd82284f2592f156640c0edafe (127.0.107.193) to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:47.527973  2359 raft_consensus.cc:993] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: : Instructing follower 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.528682  2359 raft_consensus.cc:1081] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Signalling peer 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.531637  1658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50"
dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:38616
I20260709 14:50:47.532183  1658 raft_consensus.cc:493] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:47.532550  1658 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.540241  1658 raft_consensus.cc:515] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.542382  1658 leader_election.cc:290] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:47.544014  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:47.544700  1577 raft_consensus.cc:3055] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:47.544999  1577 raft_consensus.cc:740] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17e506fd82284f2592f156640c0edafe, State: Running, Role: LEADER
I20260709 14:50:47.546021  1577 consensus_queue.cc:260] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.547180  1577 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.555614  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6621a92ee54b33a369dab7c279fc50" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:47.557883  1725 raft_consensus.cc:3060] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.563318  1577 raft_consensus.cc:2468] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.564882  2366 raft_consensus.cc:993] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: : Instructing follower 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.565442  2366 raft_consensus.cc:1081] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Signalling peer 45d5272855ec440cbd07d4d3995c68d3 to start an election
I20260709 14:50:47.566279  1623 leader_election.cc:304] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:47.567422  2380 raft_consensus.cc:2804] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:47.572031  2380 raft_consensus.cc:697] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:47.573318  2380 consensus_queue.cc:237] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.574170  1725 raft_consensus.cc:2468] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.579730  1651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca"
dest_uuid: "45d5272855ec440cbd07d4d3995c68d3"
 from {username='slave'} at 127.0.0.1:38626
I20260709 14:50:47.580403  1651 raft_consensus.cc:493] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:50:47.580768  1651 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.589808  1651 raft_consensus.cc:515] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.594436  1651 leader_election.cc:290] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Requested vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 36d78753ff96438e9cc2808f05853294 (127.0.107.195:38515)
I20260709 14:50:47.594244  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17e506fd82284f2592f156640c0edafe"
I20260709 14:50:47.595191  1577 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.596153  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb4e4a3db5f46deb623118788ef2fca" candidate_uuid: "45d5272855ec440cbd07d4d3995c68d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36d78753ff96438e9cc2808f05853294"
I20260709 14:50:47.596801  1725 raft_consensus.cc:3055] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:50:47.597131  1725 raft_consensus.cc:740] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 36d78753ff96438e9cc2808f05853294, State: Running, Role: LEADER
I20260709 14:50:47.597998  1725 consensus_queue.cc:260] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.599166  1725 raft_consensus.cc:3060] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:50:47.609051  1577 raft_consensus.cc:2468] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.610376  1623 leader_election.cc:304] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3; no voters: 
I20260709 14:50:47.611441  2375 raft_consensus.cc:2804] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:50:47.612038  2375 raft_consensus.cc:697] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Becoming Leader. State: Replica: 45d5272855ec440cbd07d4d3995c68d3, State: Running, Role: LEADER
I20260709 14:50:47.612983  2375 consensus_queue.cc:237] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:47.613873  1725 raft_consensus.cc:2468] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d5272855ec440cbd07d4d3995c68d3 in term 2.
I20260709 14:50:47.622200  1479 catalog_manager.cc:5697] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 1 to 2, leader changed from 17e506fd82284f2592f156640c0edafe (127.0.107.193) to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:47.640894  1479 catalog_manager.cc:5697] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 reported cstate change: term changed from 1 to 2, leader changed from 36d78753ff96438e9cc2808f05853294 (127.0.107.195) to 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "45d5272855ec440cbd07d4d3995c68d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:47.694473  1727 raft_consensus.cc:1275] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:47.696123  2375 consensus_queue.cc:1048] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:47.735541  1583 raft_consensus.cc:1275] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:47.741580  2375 consensus_queue.cc:1048] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:47.828076  1583 raft_consensus.cc:1275] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 3 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:47.829543  2375 consensus_queue.cc:1048] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:50:47.843429  1727 raft_consensus.cc:1275] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 3 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:50:47.847404  2395 consensus_queue.cc:1048] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:47.980372  1727 raft_consensus.cc:1275] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:47.982398  2380 consensus_queue.cc:1048] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:48.033097   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:50:48.073009  1583 raft_consensus.cc:1275] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:48.077078  2375 consensus_queue.cc:1048] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:48.099891  1727 raft_consensus.cc:1275] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Refusing update from remote peer 45d5272855ec440cbd07d4d3995c68d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:50:48.102396  2380 consensus_queue.cc:1048] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:50:48.113049   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 14:50:48.121933  1697 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 14:50:48.128590  1697 consensus_peers.cc:597] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.139639  1697 consensus_peers.cc:597] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.166249  1623 consensus_peers.cc:597] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.176759  1623 consensus_peers.cc:597] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.199193  1697 consensus_peers.cc:597] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.208521  1697 consensus_peers.cc:597] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.212738  1623 consensus_peers.cc:597] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.214218  1623 consensus_peers.cc:597] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.228039  1623 consensus_peers.cc:597] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.232832  1697 consensus_peers.cc:597] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.242596   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:48.243805   431 tablet_replica.cc:333] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.244689   431 raft_consensus.cc:2243] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.245362   431 raft_consensus.cc:2272] T 07e44a51657244bdac12c9e76cc53017 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.247758   431 tablet_replica.cc:333] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.248593   431 raft_consensus.cc:2243] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.249104   431 raft_consensus.cc:2272] T bc051130230a45eba628e68c39f0cda3 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.251322   431 tablet_replica.cc:333] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.252049   431 raft_consensus.cc:2243] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.252954   431 raft_consensus.cc:2272] T 35708989e3c345b3a0bad8e5ecebc998 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.255219   431 tablet_replica.cc:333] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.256151   431 raft_consensus.cc:2243] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.257133   431 raft_consensus.cc:2272] T e4fc279a847e48d5a240ecf6cc61c035 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.260350   431 tablet_replica.cc:333] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.261763   431 raft_consensus.cc:2243] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.263741   431 raft_consensus.cc:2272] T 806dd19b6e18429c8743b9aceda41aa5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.266999   431 tablet_replica.cc:333] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.267969   431 raft_consensus.cc:2243] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.269197   431 raft_consensus.cc:2272] T d08c454fd6fc4270ad8e5ca2542757b0 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.271569   431 tablet_replica.cc:333] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.272320   431 raft_consensus.cc:2243] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.273274   431 raft_consensus.cc:2272] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.275439   431 tablet_replica.cc:333] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.276117   431 raft_consensus.cc:2243] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.277001   431 raft_consensus.cc:2272] T c7f577518d474c0782bcd1edaefde746 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.279176   431 tablet_replica.cc:333] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.279875   431 raft_consensus.cc:2243] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.280344   431 raft_consensus.cc:2272] T ebb4e4a3db5f46deb623118788ef2fca P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.282568   431 tablet_replica.cc:333] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.283536   431 raft_consensus.cc:2243] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.284116   431 raft_consensus.cc:2272] T b1c0c3916ca4477fb2e82c720b4c1e87 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.287081   431 tablet_replica.cc:333] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.287990   431 raft_consensus.cc:2243] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.288607   431 raft_consensus.cc:2272] T 6015635506e44421bfc21fc36bdb0191 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.291676   431 tablet_replica.cc:333] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
W20260709 14:50:48.291715  1697 consensus_peers.cc:597] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.292757   431 raft_consensus.cc:2243] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.293474   431 raft_consensus.cc:2272] T 915bcb7025e246cf9db5e084d3115085 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.296597   431 tablet_replica.cc:333] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.297580   431 raft_consensus.cc:2243] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.298849   431 raft_consensus.cc:2272] T 575fecfb4a7e4812b9b8f6fb093936b5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.302026   431 tablet_replica.cc:333] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.302989   431 raft_consensus.cc:2243] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.304075   431 raft_consensus.cc:2272] T ab75a712ecfe4deeb97bc690ad53e39c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.307191   431 tablet_replica.cc:333] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.308174   431 raft_consensus.cc:2243] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.308774   431 raft_consensus.cc:2272] T 66b38ba9a15445b787fe3e5738e58a0f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.311977   431 tablet_replica.cc:333] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.313009   431 raft_consensus.cc:2243] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.313568   431 raft_consensus.cc:2272] T b0e50386aa8642078375fbb0a9fb3f82 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.316346   431 tablet_replica.cc:333] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.317281   431 raft_consensus.cc:2243] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.318457   431 raft_consensus.cc:2272] T 610d6977f8694bfe8fcccaa5d37d222a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.321214   431 tablet_replica.cc:333] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.322501   431 raft_consensus.cc:2243] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:50:48.322407  1623 consensus_peers.cc:597] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.324458   431 raft_consensus.cc:2272] T 4130f75a588248fba88156660a11b6e8 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.327962   431 tablet_replica.cc:333] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.328896   431 raft_consensus.cc:2243] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 14:50:48.329504  1623 consensus_peers.cc:597] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.329543   431 raft_consensus.cc:2272] T f22f5227c90341148049d7ab114e5272 P 17e506fd82284f2592f156640c0edafe [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.331058  1697 consensus_peers.cc:597] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.335036   431 tablet_replica.cc:333] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.335826   431 raft_consensus.cc:2243] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.336467   431 raft_consensus.cc:2272] T bc44ef62c96a40118ff58c994395ed5a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.339694  1697 consensus_peers.cc:597] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.340327   431 tablet_replica.cc:333] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
W20260709 14:50:48.340354  1697 consensus_peers.cc:597] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.341245   431 raft_consensus.cc:2243] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.342185   431 raft_consensus.cc:2272] T 5242c2facbbb46338963db64018b812e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.344856  1697 consensus_peers.cc:597] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.347018   431 tablet_replica.cc:333] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.348184   431 raft_consensus.cc:2243] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.348824   431 raft_consensus.cc:2272] T 84f4ff68bffd45df8bbdafb01119e3aa P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.353739   431 tablet_replica.cc:333] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.355875   431 raft_consensus.cc:2243] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.356530   431 raft_consensus.cc:2272] T b0a4bcec2d91452aa995a62be8b6c26b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.358563  1623 consensus_peers.cc:597] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.359645   431 tablet_replica.cc:333] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.360646   431 raft_consensus.cc:2243] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.361299   431 raft_consensus.cc:2272] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.361559  1623 consensus_peers.cc:597] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.364552   431 tablet_replica.cc:333] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.365646   431 raft_consensus.cc:2243] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.366364   431 raft_consensus.cc:2272] T 59f4b54cf7034f25bdd1182b79e561d7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.364120  1697 consensus_peers.cc:597] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.367551  1697 consensus_peers.cc:597] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.367885  1623 consensus_peers.cc:597] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.370571   431 tablet_replica.cc:333] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.371428   431 raft_consensus.cc:2243] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.372022   431 raft_consensus.cc:2272] T a043e41dae8347b3988abf806a866b74 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.375154   431 tablet_replica.cc:333] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.376143   431 raft_consensus.cc:2243] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.377241   431 raft_consensus.cc:2272] T 7ce2a67caa54429fb9510b8ee027fba5 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.380419   431 tablet_replica.cc:333] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.381285   431 raft_consensus.cc:2243] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.381910   431 raft_consensus.cc:2272] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.384842   431 tablet_replica.cc:333] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.385859   431 raft_consensus.cc:2243] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.387686   431 raft_consensus.cc:2272] T 1f6621a92ee54b33a369dab7c279fc50 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.391794   431 tablet_replica.cc:333] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.392907   431 raft_consensus.cc:2243] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.393576   431 raft_consensus.cc:2272] T 73244150135b4bb5aeeba3dc4b94fab7 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.395428  1697 consensus_peers.cc:597] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.397112   431 tablet_replica.cc:333] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.397892   431 raft_consensus.cc:2243] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.398818   431 raft_consensus.cc:2272] T c5deb3961d984f8481c1ee3e463f8a3b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.401405   431 tablet_replica.cc:333] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.402261   431 raft_consensus.cc:2243] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.402709   431 raft_consensus.cc:2272] T 9d9d4f77ab264824824b9de6802cec54 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.406219   431 tablet_replica.cc:333] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.406956   431 raft_consensus.cc:2243] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.407939   431 raft_consensus.cc:2272] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.410946   431 tablet_replica.cc:333] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.411918   431 raft_consensus.cc:2243] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.412472   431 raft_consensus.cc:2272] T fcd1126d930d488dab086eb5d690385a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.414011  1623 consensus_peers.cc:597] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.415563   431 tablet_replica.cc:333] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.416522   431 raft_consensus.cc:2243] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.417562   431 raft_consensus.cc:2272] T 08aaabb4b40d415dacd518d43a09869a P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.420756   431 tablet_replica.cc:333] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
W20260709 14:50:48.421262  1623 consensus_peers.cc:597] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.421499   431 raft_consensus.cc:2243] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.422164   431 raft_consensus.cc:2272] T 9fe3cb63a07e4ce6b38156b136569cca P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.425773   431 tablet_replica.cc:333] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.426689   431 raft_consensus.cc:2243] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.427312   431 raft_consensus.cc:2272] T ce56315bb31441e1bbe7db96a8ce4410 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.428412  1697 consensus_peers.cc:597] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.429939   431 tablet_replica.cc:333] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.430600   431 raft_consensus.cc:2243] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.431104   431 raft_consensus.cc:2272] T 6915b13ffb704edcac0fd629381d253e P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.433097   431 tablet_replica.cc:333] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.433755   431 raft_consensus.cc:2243] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.434689   431 raft_consensus.cc:2272] T 2550480abe0c4d12b8c11bf82ec5039b P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.436784   431 tablet_replica.cc:333] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.437399   431 raft_consensus.cc:2243] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.437940   431 raft_consensus.cc:2272] T 422982f0dc6b4d3ca2fdaf28d0411514 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.440215   431 tablet_replica.cc:333] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.440843   431 raft_consensus.cc:2243] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.441313   431 raft_consensus.cc:2272] T f5b7cdd0e0a24e54b17023ff094ef854 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.443387   431 tablet_replica.cc:333] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.444049   431 raft_consensus.cc:2243] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.444561   431 raft_consensus.cc:2272] T 0ea7a74d9aa14ef38365e872e3efb562 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.447306   431 tablet_replica.cc:333] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.448077   431 raft_consensus.cc:2243] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.448678   431 raft_consensus.cc:2272] T b4987548a10d4e82b06659b89058e65c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.451212   431 tablet_replica.cc:333] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.452076   431 raft_consensus.cc:2243] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.452639   431 raft_consensus.cc:2272] T d0ccd774316a4d69b235f2eddf58ddcc P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.455279   431 tablet_replica.cc:333] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.456110   431 raft_consensus.cc:2243] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.456689   431 raft_consensus.cc:2272] T 1e64847f9b324f7faefa87f278434672 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.460223   431 tablet_replica.cc:333] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.461141   431 raft_consensus.cc:2243] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.462342   431 raft_consensus.cc:2272] T c69d6d1223974de892e79da6d6c27d86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.465332   431 tablet_replica.cc:333] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.466265   431 raft_consensus.cc:2243] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.466925   431 raft_consensus.cc:2272] T 4d58c741dd1442bf9096d719c5d54058 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.469771   431 tablet_replica.cc:333] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.470635   431 raft_consensus.cc:2243] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.471448   431 raft_consensus.cc:2272] T 72c265ae69744159a2cbd5fea8af0d07 P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.472855  1623 consensus_peers.cc:597] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.471969  1697 consensus_peers.cc:597] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.474807   431 tablet_replica.cc:333] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.475826   431 raft_consensus.cc:2243] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.476430   431 raft_consensus.cc:2272] T 3244754255c4494a97a20d6ba372923c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.479377   431 tablet_replica.cc:333] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.480204   431 raft_consensus.cc:2243] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.480708   431 raft_consensus.cc:2272] T 9203095ce82145248458d98a71f8f07f P 17e506fd82284f2592f156640c0edafe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.483532   431 tablet_replica.cc:333] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.484192   431 raft_consensus.cc:2243] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.485409   431 raft_consensus.cc:2272] T b844b9b6088d424d8cd83fa504010c9c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.488878   431 tablet_replica.cc:333] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.489487   431 raft_consensus.cc:2243] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.490489   431 raft_consensus.cc:2272] T 550babf7e3344307bf7f14bae6eb5b86 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.493613   431 tablet_replica.cc:333] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.494481   431 raft_consensus.cc:2243] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.495747   431 raft_consensus.cc:2272] T c784884b0ef3459b95c5a9028bed9e61 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.498932   431 tablet_replica.cc:333] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.499763   431 raft_consensus.cc:2243] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.501070   431 raft_consensus.cc:2272] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.501070  1697 consensus_peers.cc:597] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.504669   431 tablet_replica.cc:333] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.505604   431 raft_consensus.cc:2243] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.506845   431 raft_consensus.cc:2272] T 438a8ba60c454c089cab93412c5c93a1 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.509866   431 tablet_replica.cc:333] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.510742   431 raft_consensus.cc:2243] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.511396   431 raft_consensus.cc:2272] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.514192   431 tablet_replica.cc:333] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.514998   431 raft_consensus.cc:2243] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:50:48.515609  1697 consensus_peers.cc:597] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.516108   431 raft_consensus.cc:2272] T b689c1f7b39446488c097ffeb75f608c P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.519018   431 tablet_replica.cc:333] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.519809   431 raft_consensus.cc:2243] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.520360   431 raft_consensus.cc:2272] T b259d0baa0c4493eba83ff1de96cf93f P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.523335   431 tablet_replica.cc:333] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe: stopping tablet replica
I20260709 14:50:48.524070   431 raft_consensus.cc:2243] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.524627   431 raft_consensus.cc:2272] T a3c555043e8447c4baadb1583c60a7e6 P 17e506fd82284f2592f156640c0edafe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.543640  1623 consensus_peers.cc:597] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.552157  1623 consensus_peers.cc:597] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.575768  1623 consensus_peers.cc:597] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.597309  1697 consensus_peers.cc:597] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.605633  1623 consensus_peers.cc:597] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.606470  1623 consensus_peers.cc:597] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.638455  1623 consensus_peers.cc:597] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.644090  1697 consensus_peers.cc:597] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.681944   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:50:48.696533  1697 consensus_peers.cc:597] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 -> Peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Couldn't send request to peer 17e506fd82284f2592f156640c0edafe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.846555   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 14:50:48.887461  1697 consensus_peers.cc:597] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.920524  1697 consensus_peers.cc:597] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:48.962430  1697 consensus_peers.cc:597] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:48.964287   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:48.967967   431 tablet_replica.cc:333] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:48.969735   431 raft_consensus.cc:2243] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.970436   431 raft_consensus.cc:2272] T 35708989e3c345b3a0bad8e5ecebc998 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.973870   431 tablet_replica.cc:333] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:48.975057   431 raft_consensus.cc:2243] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.975721   431 raft_consensus.cc:2272] T bc051130230a45eba628e68c39f0cda3 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.978727   431 tablet_replica.cc:333] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:48.979844   431 raft_consensus.cc:2243] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:48.981071   431 raft_consensus.cc:2272] T 07e44a51657244bdac12c9e76cc53017 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.984218   431 tablet_replica.cc:333] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:48.985411   431 raft_consensus.cc:2243] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.986132   431 raft_consensus.cc:2272] T d08c454fd6fc4270ad8e5ca2542757b0 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:48.990283   431 tablet_replica.cc:333] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:48.991493   431 raft_consensus.cc:2243] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:48.992156   431 raft_consensus.cc:2272] T e4fc279a847e48d5a240ecf6cc61c035 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:48.993559  1697 consensus_peers.cc:597] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.001549   431 tablet_replica.cc:333] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.002787   431 raft_consensus.cc:2243] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.003793   431 raft_consensus.cc:2272] T 806dd19b6e18429c8743b9aceda41aa5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.005807   431 tablet_replica.cc:333] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.006481   431 raft_consensus.cc:2243] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.007014   431 raft_consensus.cc:2272] T c7f577518d474c0782bcd1edaefde746 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.009922   431 tablet_replica.cc:333] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.010865   431 raft_consensus.cc:2243] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.011416   431 raft_consensus.cc:2272] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.014142   431 tablet_replica.cc:333] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.015065   431 raft_consensus.cc:2243] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.015602   431 raft_consensus.cc:2272] T b0e50386aa8642078375fbb0a9fb3f82 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.018304   431 tablet_replica.cc:333] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.019253   431 raft_consensus.cc:2243] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.019829   431 raft_consensus.cc:2272] T 610d6977f8694bfe8fcccaa5d37d222a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.022574   431 tablet_replica.cc:333] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.023243   431 raft_consensus.cc:2243] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.024171   431 raft_consensus.cc:2272] T ab75a712ecfe4deeb97bc690ad53e39c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.026304   431 tablet_replica.cc:333] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.026990   431 raft_consensus.cc:2243] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.027837   431 raft_consensus.cc:2272] T 66b38ba9a15445b787fe3e5738e58a0f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.029978   431 tablet_replica.cc:333] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.030694   431 raft_consensus.cc:2243] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.031258   431 raft_consensus.cc:2272] T 915bcb7025e246cf9db5e084d3115085 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.033833   431 tablet_replica.cc:333] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.034641   431 raft_consensus.cc:2243] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.035236   431 raft_consensus.cc:2272] T 575fecfb4a7e4812b9b8f6fb093936b5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.037969   431 tablet_replica.cc:333] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.038772   431 raft_consensus.cc:2243] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.039867   431 raft_consensus.cc:2272] T f22f5227c90341148049d7ab114e5272 P 45d5272855ec440cbd07d4d3995c68d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.042654   431 tablet_replica.cc:333] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.043466   431 raft_consensus.cc:2243] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.044615   431 raft_consensus.cc:2272] T ebb4e4a3db5f46deb623118788ef2fca P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.046039  1697 consensus_peers.cc:597] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.047771   431 tablet_replica.cc:333] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.048724   431 raft_consensus.cc:2243] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.049705   431 raft_consensus.cc:2272] T 6015635506e44421bfc21fc36bdb0191 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.051146  1697 consensus_peers.cc:597] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.052676   431 tablet_replica.cc:333] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.053468   431 raft_consensus.cc:2243] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.053962   431 raft_consensus.cc:2272] T b1c0c3916ca4477fb2e82c720b4c1e87 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.056294   431 tablet_replica.cc:333] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.057148   431 raft_consensus.cc:2243] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.057592   431 raft_consensus.cc:2272] T 4130f75a588248fba88156660a11b6e8 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.059664   431 tablet_replica.cc:333] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.060333   431 raft_consensus.cc:2243] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.061223   431 raft_consensus.cc:2272] T 1f6621a92ee54b33a369dab7c279fc50 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.063484   431 tablet_replica.cc:333] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.064348   431 raft_consensus.cc:2243] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.065172   431 raft_consensus.cc:2272] T 9fe3cb63a07e4ce6b38156b136569cca P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.067649   431 tablet_replica.cc:333] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.068269   431 raft_consensus.cc:2243] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.068732   431 raft_consensus.cc:2272] T ce56315bb31441e1bbe7db96a8ce4410 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.071029   431 tablet_replica.cc:333] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.071640   431 raft_consensus.cc:2243] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.072134   431 raft_consensus.cc:2272] T bc44ef62c96a40118ff58c994395ed5a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.074150   431 tablet_replica.cc:333] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.074975   431 raft_consensus.cc:2243] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.075565   431 raft_consensus.cc:2272] T 5242c2facbbb46338963db64018b812e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.078193   431 tablet_replica.cc:333] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.079020   431 raft_consensus.cc:2243] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.079535   431 raft_consensus.cc:2272] T 84f4ff68bffd45df8bbdafb01119e3aa P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.081985   431 tablet_replica.cc:333] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.082901   431 raft_consensus.cc:2243] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.083477   431 raft_consensus.cc:2272] T b0a4bcec2d91452aa995a62be8b6c26b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.086186   431 tablet_replica.cc:333] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.087253   431 raft_consensus.cc:2243] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.088572   431 raft_consensus.cc:2272] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.091600   431 tablet_replica.cc:333] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.092679   431 raft_consensus.cc:2243] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.093840   431 raft_consensus.cc:2272] T 73244150135b4bb5aeeba3dc4b94fab7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.096340   431 tablet_replica.cc:333] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.096980   431 raft_consensus.cc:2243] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.097553   431 raft_consensus.cc:2272] T c5deb3961d984f8481c1ee3e463f8a3b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.099630   431 tablet_replica.cc:333] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.100263   431 raft_consensus.cc:2243] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.100696   431 raft_consensus.cc:2272] T 6915b13ffb704edcac0fd629381d253e P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.102981   431 tablet_replica.cc:333] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.103837   431 raft_consensus.cc:2243] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.104321   431 raft_consensus.cc:2272] T 2550480abe0c4d12b8c11bf82ec5039b P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.106940   431 tablet_replica.cc:333] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
W20260709 14:50:49.107179  1697 consensus_peers.cc:597] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.107682   431 raft_consensus.cc:2243] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.108251   431 raft_consensus.cc:2272] T 422982f0dc6b4d3ca2fdaf28d0411514 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.110431   431 tablet_replica.cc:333] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.111192   431 raft_consensus.cc:2243] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.111685   431 raft_consensus.cc:2272] T 7ce2a67caa54429fb9510b8ee027fba5 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.114198   431 tablet_replica.cc:333] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.114969   431 raft_consensus.cc:2243] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.115449   431 raft_consensus.cc:2272] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.117775   431 tablet_replica.cc:333] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.118649   431 raft_consensus.cc:2243] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.119328   431 raft_consensus.cc:2272] T 08aaabb4b40d415dacd518d43a09869a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.122092   431 tablet_replica.cc:333] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.123116   431 raft_consensus.cc:2243] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.124254   431 raft_consensus.cc:2272] T f5b7cdd0e0a24e54b17023ff094ef854 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.127200   431 tablet_replica.cc:333] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.128208   431 raft_consensus.cc:2243] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.128875   431 raft_consensus.cc:2272] T 0ea7a74d9aa14ef38365e872e3efb562 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.131690   431 tablet_replica.cc:333] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.132627   431 raft_consensus.cc:2243] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.134182   431 raft_consensus.cc:2272] T d0ccd774316a4d69b235f2eddf58ddcc P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.137066   431 tablet_replica.cc:333] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.138044   431 raft_consensus.cc:2243] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.139221   431 raft_consensus.cc:2272] T b4987548a10d4e82b06659b89058e65c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.142122   431 tablet_replica.cc:333] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.143227   431 raft_consensus.cc:2243] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 14:50:49.141419  1697 consensus_peers.cc:597] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.143863   431 raft_consensus.cc:2272] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.144570  1697 consensus_peers.cc:597] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:49.145351  1697 consensus_peers.cc:597] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.146956   431 tablet_replica.cc:333] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.148130   431 raft_consensus.cc:2243] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.148828   431 raft_consensus.cc:2272] T 438a8ba60c454c089cab93412c5c93a1 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.150489  1697 consensus_peers.cc:597] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:49.151400  1697 consensus_peers.cc:597] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.152509   431 tablet_replica.cc:333] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.153517   431 raft_consensus.cc:2243] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.154222   431 raft_consensus.cc:2272] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.157472   431 tablet_replica.cc:333] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.158420   431 raft_consensus.cc:2243] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.159137   431 raft_consensus.cc:2272] T c784884b0ef3459b95c5a9028bed9e61 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.162281   431 tablet_replica.cc:333] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.163288   431 raft_consensus.cc:2243] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 14:50:49.163373  1697 consensus_peers.cc:597] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.164059   431 raft_consensus.cc:2272] T 550babf7e3344307bf7f14bae6eb5b86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.167223   431 tablet_replica.cc:333] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.168259   431 raft_consensus.cc:2243] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.169025   431 raft_consensus.cc:2272] T 72c265ae69744159a2cbd5fea8af0d07 P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.172494   431 tablet_replica.cc:333] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.173727   431 raft_consensus.cc:2243] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.174431   431 raft_consensus.cc:2272] T 4d58c741dd1442bf9096d719c5d54058 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.177500   431 tablet_replica.cc:333] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.178399   431 raft_consensus.cc:2243] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.179544   431 raft_consensus.cc:2272] T 1e64847f9b324f7faefa87f278434672 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.182381   431 tablet_replica.cc:333] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.183323   431 raft_consensus.cc:2243] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.184013   431 raft_consensus.cc:2272] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.185835  1697 consensus_peers.cc:597] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.187083   431 tablet_replica.cc:333] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.188043   431 raft_consensus.cc:2243] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.189137   431 raft_consensus.cc:2272] T 9d9d4f77ab264824824b9de6802cec54 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.191880   431 tablet_replica.cc:333] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.192760   431 raft_consensus.cc:2243] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.193377   431 raft_consensus.cc:2272] T c69d6d1223974de892e79da6d6c27d86 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.194494  2371 raft_consensus.cc:493] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.195145  2371 raft_consensus.cc:515] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.196359   431 tablet_replica.cc:333] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.197299  2371 leader_election.cc:290] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.197264   431 raft_consensus.cc:2243] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.200201   431 raft_consensus.cc:2272] T 9203095ce82145248458d98a71f8f07f P 45d5272855ec440cbd07d4d3995c68d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.203318   431 tablet_replica.cc:333] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
W20260709 14:50:49.203753  1697 leader_election.cc:336] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.204330   431 raft_consensus.cc:2243] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 14:50:49.204536  1697 leader_election.cc:336] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:50:49.205020   431 raft_consensus.cc:2272] T b844b9b6088d424d8cd83fa504010c9c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.205030  1697 leader_election.cc:304] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.205965  2371 raft_consensus.cc:2749] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.208186   431 tablet_replica.cc:333] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.209203   431 raft_consensus.cc:2243] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.209867   431 raft_consensus.cc:2272] T a043e41dae8347b3988abf806a866b74 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.212970   431 tablet_replica.cc:333] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.213864   431 raft_consensus.cc:2243] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.214468   431 raft_consensus.cc:2272] T 59f4b54cf7034f25bdd1182b79e561d7 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.217619   431 tablet_replica.cc:333] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.218524   431 raft_consensus.cc:2243] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.219190   431 raft_consensus.cc:2272] T 3244754255c4494a97a20d6ba372923c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.222234   431 tablet_replica.cc:333] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.223083   431 raft_consensus.cc:2243] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.224141   431 raft_consensus.cc:2272] T fcd1126d930d488dab086eb5d690385a P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.227062   431 tablet_replica.cc:333] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.227880   431 raft_consensus.cc:2243] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.228471   431 raft_consensus.cc:2272] T b689c1f7b39446488c097ffeb75f608c P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.229878  1697 consensus_peers.cc:597] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.231310   431 tablet_replica.cc:333] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.232206   431 raft_consensus.cc:2243] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.232801   431 raft_consensus.cc:2272] T b259d0baa0c4493eba83ff1de96cf93f P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.235719   431 tablet_replica.cc:333] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3: stopping tablet replica
I20260709 14:50:49.236512   431 raft_consensus.cc:2243] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.237764   431 raft_consensus.cc:2272] T a3c555043e8447c4baadb1583c60a7e6 P 45d5272855ec440cbd07d4d3995c68d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.241556  1697 consensus_peers.cc:597] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.267459  2366 raft_consensus.cc:493] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.268077  2366 raft_consensus.cc:515] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:49.268671  1697 consensus_peers.cc:597] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.270521  2366 leader_election.cc:290] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.278566  1697 leader_election.cc:336] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.279376  1697 leader_election.cc:336] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:50:49.279726  1697 leader_election.cc:304] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.280412  2366 raft_consensus.cc:2749] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:49.303313  1697 consensus_peers.cc:597] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:50:49.304124  1697 consensus_peers.cc:597] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.350342   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 14:50:49.372385  1697 consensus_peers.cc:597] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 -> Peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Couldn't send request to peer 45d5272855ec440cbd07d4d3995c68d3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:49.377866  2403 raft_consensus.cc:493] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.378639  2403 raft_consensus.cc:515] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.381358  2403 leader_election.cc:290] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.383149  2366 raft_consensus.cc:493] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.383774  2366 raft_consensus.cc:515] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.386334  2366 leader_election.cc:290] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.405347  1697 leader_election.cc:336] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.406085  1697 leader_election.cc:336] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.407544  1697 leader_election.cc:336] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.408051  1697 leader_election.cc:304] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.408998  2366 raft_consensus.cc:2749] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:49.409168  1697 leader_election.cc:336] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.409706  1697 leader_election.cc:304] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.410538  2366 raft_consensus.cc:2749] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.435078  2366 raft_consensus.cc:493] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.435753  2366 raft_consensus.cc:515] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.439396  2366 leader_election.cc:290] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.444594  1697 leader_election.cc:336] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.445369  2356 raft_consensus.cc:493] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.446149  2356 raft_consensus.cc:515] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.449028  2356 leader_election.cc:290] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.449731  1697 leader_election.cc:336] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.450249  1697 leader_election.cc:304] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.451421  2356 raft_consensus.cc:2749] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:49.494678  1697 leader_election.cc:336] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.497542  2366 raft_consensus.cc:493] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.499210  2366 raft_consensus.cc:515] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
W20260709 14:50:49.500669  1697 leader_election.cc:336] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.501428  1697 leader_election.cc:304] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.503602  2356 raft_consensus.cc:2749] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.503983  2366 leader_election.cc:290] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.527311  1697 leader_election.cc:336] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
W20260709 14:50:49.528437  1697 leader_election.cc:336] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.528995  1697 leader_election.cc:304] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.530249  2409 raft_consensus.cc:2749] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.531335  2403 raft_consensus.cc:493] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.532117  2403 raft_consensus.cc:515] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.534914  2403 leader_election.cc:290] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.535662  2403 raft_consensus.cc:493] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.536470  2403 raft_consensus.cc:515] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.539156  2403 leader_election.cc:290] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.545593  2403 raft_consensus.cc:493] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.546504  2403 raft_consensus.cc:515] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.548774  2403 leader_election.cc:290] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
W20260709 14:50:49.552750  1697 leader_election.cc:336] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.553378  1697 leader_election.cc:336] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.554335  1697 leader_election.cc:336] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.554848  1697 leader_election.cc:304] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.555718  2403 raft_consensus.cc:2749] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:49.555970  1697 leader_election.cc:336] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.556456  1697 leader_election.cc:304] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.557289  2403 raft_consensus.cc:2749] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:49.560896  1697 leader_election.cc:336] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.561863  1697 leader_election.cc:336] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.562260  1697 leader_election.cc:304] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.563093  2403 raft_consensus.cc:2749] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.577126   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 14:50:49.672256  2409 raft_consensus.cc:493] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.672853  2409 raft_consensus.cc:515] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.675027  2409 leader_election.cc:290] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.683069   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 14:50:49.683858  1697 leader_election.cc:336] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
W20260709 14:50:49.684979  1697 leader_election.cc:336] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.685443  1697 leader_election.cc:304] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.686401  2409 raft_consensus.cc:2749] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.686630   431 tablet_replica.cc:333] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.688798  2409 raft_consensus.cc:493] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.689324   431 raft_consensus.cc:2243] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.689479  2409 raft_consensus.cc:515] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.690268   431 raft_consensus.cc:2272] T 07e44a51657244bdac12c9e76cc53017 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.694348  2409 leader_election.cc:290] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.694800   431 tablet_replica.cc:333] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.695416  2356 raft_consensus.cc:493] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.696496  2356 raft_consensus.cc:515] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.697611   431 raft_consensus.cc:2243] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.698649   431 raft_consensus.cc:2272] T 806dd19b6e18429c8743b9aceda41aa5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.699411  2356 leader_election.cc:290] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.700100  2409 raft_consensus.cc:493] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.700817  2409 raft_consensus.cc:515] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } } peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } }
I20260709 14:50:49.703615  2409 leader_election.cc:290] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999), 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817)
I20260709 14:50:49.704581   431 tablet_replica.cc:333] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
W20260709 14:50:49.706207  1697 leader_election.cc:336] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.707289   431 raft_consensus.cc:2243] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.709190   431 raft_consensus.cc:2272] T 35708989e3c345b3a0bad8e5ecebc998 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.710193  1697 leader_election.cc:336] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.710798  1697 leader_election.cc:304] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.711855  2409 raft_consensus.cc:2749] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.713131   431 tablet_replica.cc:333] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
W20260709 14:50:49.714916  1697 leader_election.cc:336] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.715212   431 raft_consensus.cc:2243] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.717100   431 raft_consensus.cc:2272] T bc051130230a45eba628e68c39f0cda3 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.718139  1697 leader_election.cc:336] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.718672  1697 leader_election.cc:304] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.719792  2371 raft_consensus.cc:2749] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.721427   431 tablet_replica.cc:333] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
W20260709 14:50:49.721773  1697 leader_election.cc:336] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.722733   431 raft_consensus.cc:2243] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.723781   431 raft_consensus.cc:2272] T d08c454fd6fc4270ad8e5ca2542757b0 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.727051  1697 leader_election.cc:336] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.727689   431 tablet_replica.cc:333] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.727591  1697 leader_election.cc:304] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.728456  2409 raft_consensus.cc:2749] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.729257   431 raft_consensus.cc:2243] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.730515   431 raft_consensus.cc:2272] T e4fc279a847e48d5a240ecf6cc61c035 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.734328   431 tablet_replica.cc:333] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.735442   431 raft_consensus.cc:2243] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.736037   431 raft_consensus.cc:2272] T 630ab4f3cb154a0bb2fa8c41fbe95434 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.739037   431 tablet_replica.cc:333] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.740434   431 raft_consensus.cc:2243] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.742208   431 raft_consensus.cc:2272] T b0e50386aa8642078375fbb0a9fb3f82 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.746035   431 tablet_replica.cc:333] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.747324   431 raft_consensus.cc:2243] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.748132   431 raft_consensus.cc:2272] T 610d6977f8694bfe8fcccaa5d37d222a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.752008   431 tablet_replica.cc:333] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.752948   431 raft_consensus.cc:2243] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.753974   431 raft_consensus.cc:2272] T 915bcb7025e246cf9db5e084d3115085 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.756333   431 tablet_replica.cc:333] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.757182   431 raft_consensus.cc:2243] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.757836   431 raft_consensus.cc:2272] T 575fecfb4a7e4812b9b8f6fb093936b5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.760214   431 tablet_replica.cc:333] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.761044   431 raft_consensus.cc:2243] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.761657   431 raft_consensus.cc:2272] T ab75a712ecfe4deeb97bc690ad53e39c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.764137   431 tablet_replica.cc:333] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.764921   431 raft_consensus.cc:2243] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.765525   431 raft_consensus.cc:2272] T 66b38ba9a15445b787fe3e5738e58a0f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.767953   431 tablet_replica.cc:333] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.768671   431 raft_consensus.cc:2243] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.769375   431 raft_consensus.cc:2272] T 4130f75a588248fba88156660a11b6e8 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.771754   431 tablet_replica.cc:333] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.772523   431 raft_consensus.cc:2243] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.773180   431 raft_consensus.cc:2272] T ebb4e4a3db5f46deb623118788ef2fca P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.776324   431 tablet_replica.cc:333] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.777107   431 raft_consensus.cc:2243] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.778095   431 raft_consensus.cc:2272] T 6015635506e44421bfc21fc36bdb0191 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.781248   431 tablet_replica.cc:333] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.782135   431 raft_consensus.cc:2243] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.783650   431 raft_consensus.cc:2272] T b1c0c3916ca4477fb2e82c720b4c1e87 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.786060   431 tablet_replica.cc:333] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.786785   431 raft_consensus.cc:2243] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.787458   431 raft_consensus.cc:2272] T f22f5227c90341148049d7ab114e5272 P 36d78753ff96438e9cc2808f05853294 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.789858   431 tablet_replica.cc:333] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.790516   431 raft_consensus.cc:2243] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.791579   431 raft_consensus.cc:2272] T bc44ef62c96a40118ff58c994395ed5a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.794162   431 tablet_replica.cc:333] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.794970   431 raft_consensus.cc:2243] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.795459   431 raft_consensus.cc:2272] T 5242c2facbbb46338963db64018b812e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.797536   431 tablet_replica.cc:333] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.798357   431 raft_consensus.cc:2243] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.799463   431 raft_consensus.cc:2272] T 84f4ff68bffd45df8bbdafb01119e3aa P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.801677   431 tablet_replica.cc:333] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.802459   431 raft_consensus.cc:2243] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.803161   431 raft_consensus.cc:2272] T 73244150135b4bb5aeeba3dc4b94fab7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.805765   431 tablet_replica.cc:333] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.806437   431 raft_consensus.cc:2243] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.807029   431 raft_consensus.cc:2272] T c5deb3961d984f8481c1ee3e463f8a3b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.809780   431 tablet_replica.cc:333] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.810853   431 raft_consensus.cc:2243] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.811582   431 raft_consensus.cc:2272] T 1f6621a92ee54b33a369dab7c279fc50 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.814574   431 tablet_replica.cc:333] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.815516   431 raft_consensus.cc:2243] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.816659   431 raft_consensus.cc:2272] T 59f4b54cf7034f25bdd1182b79e561d7 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.819902   431 tablet_replica.cc:333] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.820675   431 raft_consensus.cc:2243] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.821859   431 raft_consensus.cc:2272] T a043e41dae8347b3988abf806a866b74 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.824165   431 tablet_replica.cc:333] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.824855   431 raft_consensus.cc:2243] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.825717   431 raft_consensus.cc:2272] T 7ce2a67caa54429fb9510b8ee027fba5 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.828188   431 tablet_replica.cc:333] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.828926   431 raft_consensus.cc:2243] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.830123   431 raft_consensus.cc:2272] T e93fe5d42f4a4b38a4d1adc56b5fcabb P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.832907  2409 raft_consensus.cc:493] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.833382   431 tablet_replica.cc:333] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.833726  2409 raft_consensus.cc:515] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.834411   431 raft_consensus.cc:2243] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.835156   431 raft_consensus.cc:2272] T 08aaabb4b40d415dacd518d43a09869a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.838752   431 tablet_replica.cc:333] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.839193  2409 leader_election.cc:290] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.840140   431 raft_consensus.cc:2243] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.840960   431 raft_consensus.cc:2272] T 9bd3d3a0c81c4e6e83d700d44aa3d2fc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.844074   431 tablet_replica.cc:333] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.844991   431 raft_consensus.cc:2243] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.846084   431 raft_consensus.cc:2272] T b0a4bcec2d91452aa995a62be8b6c26b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.845914  1697 leader_election.cc:336] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
W20260709 14:50:49.846817  1697 leader_election.cc:336] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.847289  1697 leader_election.cc:304] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.848153  2409 raft_consensus.cc:2749] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.849092   431 tablet_replica.cc:333] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.849838   431 raft_consensus.cc:2243] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.850502   431 raft_consensus.cc:2272] T f5b7cdd0e0a24e54b17023ff094ef854 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.852748   431 tablet_replica.cc:333] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.853456   431 raft_consensus.cc:2243] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.854060   431 raft_consensus.cc:2272] T b4987548a10d4e82b06659b89058e65c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.855819  2409 raft_consensus.cc:493] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.856473  2409 raft_consensus.cc:515] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.857220   431 tablet_replica.cc:333] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.858219   431 raft_consensus.cc:2243] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.859021  2409 leader_election.cc:290] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.859563   431 raft_consensus.cc:2272] T 0ea7a74d9aa14ef38365e872e3efb562 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.863270   431 tablet_replica.cc:333] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.864246   431 raft_consensus.cc:2243] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.864930   431 raft_consensus.cc:2272] T d0ccd774316a4d69b235f2eddf58ddcc P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.865492  1697 leader_election.cc:336] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.867925   431 tablet_replica.cc:333] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
W20260709 14:50:49.868189  1697 leader_election.cc:336] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.868853   431 raft_consensus.cc:2243] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.868906  1697 leader_election.cc:304] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.869581   431 raft_consensus.cc:2272] T 1e64847f9b324f7faefa87f278434672 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.869957  2409 raft_consensus.cc:2749] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.872622   431 tablet_replica.cc:333] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.873412   431 raft_consensus.cc:2243] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.874119   431 raft_consensus.cc:2272] T 1298fa6acd614e99b7cdb2fc4adbf5b9 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.877668   431 tablet_replica.cc:333] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.878449   431 raft_consensus.cc:2243] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.879256   431 raft_consensus.cc:2272] T 9d9d4f77ab264824824b9de6802cec54 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.879048  2409 raft_consensus.cc:493] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.879868  2409 raft_consensus.cc:515] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.881850   431 tablet_replica.cc:333] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.882503  2409 leader_election.cc:290] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.882936   431 raft_consensus.cc:2243] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.883635   431 raft_consensus.cc:2272] T c69d6d1223974de892e79da6d6c27d86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.886641   431 tablet_replica.cc:333] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.887173  2409 raft_consensus.cc:493] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 17e506fd82284f2592f156640c0edafe)
I20260709 14:50:49.887529   431 raft_consensus.cc:2243] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.887759  2409 raft_consensus.cc:515] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e506fd82284f2592f156640c0edafe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43817 } } peers { permanent_uuid: "45d5272855ec440cbd07d4d3995c68d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44999 } } peers { permanent_uuid: "36d78753ff96438e9cc2808f05853294" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38515 } }
I20260709 14:50:49.888257   431 raft_consensus.cc:2272] T 9203095ce82145248458d98a71f8f07f P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.890110  2409 leader_election.cc:290] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817), 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999)
I20260709 14:50:49.891541   431 tablet_replica.cc:333] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
W20260709 14:50:49.893148  1697 leader_election.cc:336] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.892530   431 raft_consensus.cc:2243] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.894013   431 raft_consensus.cc:2272] T b844b9b6088d424d8cd83fa504010c9c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.894292  1697 leader_election.cc:336] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.894773  1697 leader_election.cc:304] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.895627  2409 raft_consensus.cc:2749] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.897202   431 tablet_replica.cc:333] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.898063   431 raft_consensus.cc:2243] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 14:50:49.898595  1697 leader_election.cc:336] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 17e506fd82284f2592f156640c0edafe (127.0.107.193:43817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43817: connect: Connection refused (error 111)
I20260709 14:50:49.898731   431 raft_consensus.cc:2272] T c7f577518d474c0782bcd1edaefde746 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:50:49.899361  1697 leader_election.cc:336] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45d5272855ec440cbd07d4d3995c68d3 (127.0.107.194:44999): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44999: connect: Connection refused (error 111)
I20260709 14:50:49.899758  1697 leader_election.cc:304] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36d78753ff96438e9cc2808f05853294; no voters: 17e506fd82284f2592f156640c0edafe, 45d5272855ec440cbd07d4d3995c68d3
I20260709 14:50:49.900548  2409 raft_consensus.cc:2749] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:49.901723   431 tablet_replica.cc:333] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.902392   431 raft_consensus.cc:2243] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.903456   431 raft_consensus.cc:2272] T 6915b13ffb704edcac0fd629381d253e P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.905508   431 tablet_replica.cc:333] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.906147   431 raft_consensus.cc:2243] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.906682   431 raft_consensus.cc:2272] T 2550480abe0c4d12b8c11bf82ec5039b P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.908883   431 tablet_replica.cc:333] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.909549   431 raft_consensus.cc:2243] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.910050   431 raft_consensus.cc:2272] T b689c1f7b39446488c097ffeb75f608c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.912283   431 tablet_replica.cc:333] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.912921   431 raft_consensus.cc:2243] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.913882   431 raft_consensus.cc:2272] T b259d0baa0c4493eba83ff1de96cf93f P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.915973   431 tablet_replica.cc:333] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.916527   431 raft_consensus.cc:2243] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.917001   431 raft_consensus.cc:2272] T a3c555043e8447c4baadb1583c60a7e6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.919245   431 tablet_replica.cc:333] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.920013   431 raft_consensus.cc:2243] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.920552   431 raft_consensus.cc:2272] T 438a8ba60c454c089cab93412c5c93a1 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.922726   431 tablet_replica.cc:333] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.923362   431 raft_consensus.cc:2243] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.923894   431 raft_consensus.cc:2272] T fcd1126d930d488dab086eb5d690385a P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.925899   431 tablet_replica.cc:333] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.926478   431 raft_consensus.cc:2243] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.927055   431 raft_consensus.cc:2272] T 550babf7e3344307bf7f14bae6eb5b86 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.929275   431 tablet_replica.cc:333] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.929889   431 raft_consensus.cc:2243] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.930390   431 raft_consensus.cc:2272] T c784884b0ef3459b95c5a9028bed9e61 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.933001   431 tablet_replica.cc:333] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.933892   431 raft_consensus.cc:2243] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.934432   431 raft_consensus.cc:2272] T 5af1fd4b621c4a69afab73a4d89fa1a6 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.937422   431 tablet_replica.cc:333] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.938007   431 raft_consensus.cc:2243] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.939152   431 raft_consensus.cc:2272] T 9c7fe23a70e9483db6bb31a0a6ab1096 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.941355   431 tablet_replica.cc:333] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.941886   431 raft_consensus.cc:2243] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.942744   431 raft_consensus.cc:2272] T 72c265ae69744159a2cbd5fea8af0d07 P 36d78753ff96438e9cc2808f05853294 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.944777   431 tablet_replica.cc:333] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.945322   431 raft_consensus.cc:2243] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.946252   431 raft_consensus.cc:2272] T 4d58c741dd1442bf9096d719c5d54058 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.948395   431 tablet_replica.cc:333] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.949968   431 raft_consensus.cc:2243] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:50:49.950527   431 raft_consensus.cc:2272] T 9fe3cb63a07e4ce6b38156b136569cca P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.953207   431 tablet_replica.cc:333] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.953896   431 raft_consensus.cc:2243] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.955094   431 raft_consensus.cc:2272] T ce56315bb31441e1bbe7db96a8ce4410 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.957516   431 tablet_replica.cc:333] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.958101   431 raft_consensus.cc:2243] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.959169   431 raft_consensus.cc:2272] T 422982f0dc6b4d3ca2fdaf28d0411514 P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:49.961480   431 tablet_replica.cc:333] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294: stopping tablet replica
I20260709 14:50:49.962083   431 raft_consensus.cc:2243] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:49.963140   431 raft_consensus.cc:2272] T 3244754255c4494a97a20d6ba372923c P 36d78753ff96438e9cc2808f05853294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:50.935750   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:50:51.027817   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 14:50:51.064881   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:50:51.081858   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 14:50:51.092590   431 master.cc:562] Master@127.0.107.254:38079 shutting down...
I20260709 14:50:51.114001   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:50:51.115015   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:50:51.115422   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 36846e51a4be4e68ba398d50874f8d9e: stopping tablet replica
I20260709 14:50:51.137297   431 master.cc:584] Master@127.0.107.254:38079 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (25521 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 14:50:51.245379   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39575
I20260709 14:50:51.246706   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:51.254000  2411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.254637  2410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.254849  2413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.255714   431 server_base.cc:1061] running on GCE node
I20260709 14:50:51.257247   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:51.257489   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:51.257687   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608651257664 us; error 0 us; skew 500 ppm
I20260709 14:50:51.258795   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:51.261781   431 webserver.cc:533] Webserver started at http://127.0.107.254:44857/ using document root <none> and password file <none>
I20260709 14:50:51.262384   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:51.262588   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:51.262979   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:51.264747   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "3f6dfe8cfa964684b547b44a4b308ca2"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.270040   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.003s
I20260709 14:50:51.273993  2418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.275072   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:50:51.275365   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "3f6dfe8cfa964684b547b44a4b308ca2"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.275640   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:51.294520   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:51.296002   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:51.340821   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39575
I20260709 14:50:51.340950  2469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39575 every 8 connection(s)
I20260709 14:50:51.345714  2470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:51.358302  2470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Bootstrap starting.
I20260709 14:50:51.364594  2470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:51.369875  2470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: No bootstrap required, opened a new log
I20260709 14:50:51.372751  2470 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER }
I20260709 14:50:51.373236  2470 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:51.373543  2470 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6dfe8cfa964684b547b44a4b308ca2, State: Initialized, Role: FOLLOWER
I20260709 14:50:51.374333  2470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER }
I20260709 14:50:51.374917  2470 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:50:51.375149  2470 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:50:51.375456  2470 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:51.381322  2470 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER }
I20260709 14:50:51.382033  2470 leader_election.cc:304] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f6dfe8cfa964684b547b44a4b308ca2; no voters: 
I20260709 14:50:51.383411  2470 leader_election.cc:290] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:50:51.383957  2473 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:51.385502  2473 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 1 LEADER]: Becoming Leader. State: Replica: 3f6dfe8cfa964684b547b44a4b308ca2, State: Running, Role: LEADER
I20260709 14:50:51.386330  2473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER }
I20260709 14:50:51.386678  2470 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:50:51.394513  2475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f6dfe8cfa964684b547b44a4b308ca2. Latest consensus state: current_term: 1 leader_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER } }
I20260709 14:50:51.394248  2474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6dfe8cfa964684b547b44a4b308ca2" member_type: VOTER } }
I20260709 14:50:51.395200  2475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:51.395292  2474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [sys.catalog]: This master's current role is: LEADER
I20260709 14:50:51.397343  2481 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:50:51.404418  2481 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:50:51.405964   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:50:51.414994  2481 catalog_manager.cc:1383] Generated new cluster ID: 4645a43248864efc8d98a0f580c60171
I20260709 14:50:51.415367  2481 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:50:51.431311  2481 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:50:51.432842  2481 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:50:51.448786  2481 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Generated new TSK 0
I20260709 14:50:51.449505  2481 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:50:51.473124   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:51.480545  2491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.481845  2492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.483311  2494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.483778   431 server_base.cc:1061] running on GCE node
I20260709 14:50:51.484916   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:51.485111   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:51.485350   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608651485302 us; error 0 us; skew 500 ppm
I20260709 14:50:51.485991   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:51.489162   431 webserver.cc:533] Webserver started at http://127.0.107.193:33325/ using document root <none> and password file <none>
I20260709 14:50:51.489766   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:51.489989   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:51.490322   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:51.492075   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "c1b0e692e4294ee3b8694bbada174ba1"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.498066   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 14:50:51.502614  2499 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.503705   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:50:51.504123   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "c1b0e692e4294ee3b8694bbada174ba1"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.504464   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:51.526737   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:51.528234   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:51.529945   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:51.532738   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:51.533037   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.533320   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:51.533529   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.585048   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:37831
I20260709 14:50:51.585175  2561 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:37831 every 8 connection(s)
I20260709 14:50:51.591282   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:51.600553  2566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.601310  2567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.605618   431 server_base.cc:1061] running on GCE node
W20260709 14:50:51.605646  2569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.607245   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:51.607534   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:51.607769   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608651607743 us; error 0 us; skew 500 ppm
I20260709 14:50:51.608573   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:51.609498  2562 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39575
I20260709 14:50:51.610020  2562 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:51.611037  2562 heartbeater.cc:507] Master 127.0.107.254:39575 requested a full tablet report, sending...
I20260709 14:50:51.611861   431 webserver.cc:533] Webserver started at http://127.0.107.194:36047/ using document root <none> and password file <none>
I20260709 14:50:51.612555   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:51.612794   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:51.613090   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:51.613945  2435 ts_manager.cc:194] Registered new tserver with Master: c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:51.614491   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "56620e612f0549e6abf320aa80398e74"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.616286  2435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50320
I20260709 14:50:51.620255   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 14:50:51.624754  2574 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.625672   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:50:51.626046   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "56620e612f0549e6abf320aa80398e74"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.626406   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:51.654502   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:51.655999   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:51.657794   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:51.660857   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:51.661146   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.661448   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:51.661644   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.710644   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40091
I20260709 14:50:51.710772  2636 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40091 every 8 connection(s)
I20260709 14:50:51.716410   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:51.724982  2640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.728181  2641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.730485   431 server_base.cc:1061] running on GCE node
W20260709 14:50:51.730593  2643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.731056  2637 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39575
I20260709 14:50:51.731550  2637 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:51.731916   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:51.732185   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:51.732370   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608651732351 us; error 0 us; skew 500 ppm
I20260709 14:50:51.732492  2637 heartbeater.cc:507] Master 127.0.107.254:39575 requested a full tablet report, sending...
I20260709 14:50:51.733021   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:51.735040  2435 ts_manager.cc:194] Registered new tserver with Master: 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:51.736210   431 webserver.cc:533] Webserver started at http://127.0.107.195:45535/ using document root <none> and password file <none>
I20260709 14:50:51.736986   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:51.737264   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:51.737175  2435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50322
I20260709 14:50:51.737794   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:51.739377   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.744575   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:50:51.748556  2648 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.749467   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 14:50:51.749817   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.750149   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:51.765445   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:51.766973   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:51.768656   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:51.771204   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:51.771466   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.771728   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:51.771924   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.824328   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:41829
I20260709 14:50:51.824450  2710 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:41829 every 8 connection(s)
I20260709 14:50:51.830976   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:50:51.840233  2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.840248  2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:50:51.843703  2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:50:51.844669  2711 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39575
I20260709 14:50:51.845188  2711 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:51.845510   431 server_base.cc:1061] running on GCE node
I20260709 14:50:51.846124  2711 heartbeater.cc:507] Master 127.0.107.254:39575 requested a full tablet report, sending...
I20260709 14:50:51.846565   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:50:51.846824   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:50:51.847052   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608651847033 us; error 0 us; skew 500 ppm
I20260709 14:50:51.847888   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:50:51.848666  2435 ts_manager.cc:194] Registered new tserver with Master: 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:51.850513  2435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50338
I20260709 14:50:51.851373   431 webserver.cc:533] Webserver started at http://127.0.107.196:42523/ using document root <none> and password file <none>
I20260709 14:50:51.852166   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:50:51.852429   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:50:51.852749   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:50:51.853914   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-3-root/instance:
uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.859261   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 14:50:51.864362  2722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.865494   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 14:50:51.865928   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-3-root
uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:50:51.866307   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:50:51.918790   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:50:51.920428   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:50:51.921964   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:50:51.924378   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:50:51.924603   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.924952   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:50:51.925134   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:50:51.979998   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:46277
I20260709 14:50:51.980125  2784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:46277 every 8 connection(s)
I20260709 14:50:51.999979  2785 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39575
I20260709 14:50:52.000530  2785 heartbeater.cc:461] Registering TS with master...
I20260709 14:50:52.001398  2785 heartbeater.cc:507] Master 127.0.107.254:39575 requested a full tablet report, sending...
I20260709 14:50:52.003860  2435 ts_manager.cc:194] Registered new tserver with Master: 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:52.004624   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.02111703s
I20260709 14:50:52.005784  2435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50348
I20260709 14:50:52.036074  2435 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50358:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 14:50:52.299834  2750 tablet_service.cc:1511] Processing CreateTablet for tablet acb30692225e4daebaa7e02e9cb291b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 14:50:52.301563  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb30692225e4daebaa7e02e9cb291b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.304891  2526 tablet_service.cc:1511] Processing CreateTablet for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:52.306486  2526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.303098  2527 tablet_service.cc:1511] Processing CreateTablet for tablet acb30692225e4daebaa7e02e9cb291b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 14:50:52.311406  2527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb30692225e4daebaa7e02e9cb291b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.317538  2749 tablet_service.cc:1511] Processing CreateTablet for tablet ff9fe3d5f686477aa2bc32f628dcfb08 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:52.319787  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9fe3d5f686477aa2bc32f628dcfb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.329052  2525 tablet_service.cc:1511] Processing CreateTablet for tablet ff9fe3d5f686477aa2bc32f628dcfb08 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:52.332396  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9fe3d5f686477aa2bc32f628dcfb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.339313  2524 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb844e4a36641d79afdab73422e06f1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:52.342507  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb844e4a36641d79afdab73422e06f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.352232  2805 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:52.353145  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 498c3de2b5414c968242f7f8fc2ca7a1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:52.355674  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498c3de2b5414c968242f7f8fc2ca7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.367228  2600 tablet_service.cc:1511] Processing CreateTablet for tablet 498c3de2b5414c968242f7f8fc2ca7a1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:52.368005  2602 tablet_service.cc:1511] Processing CreateTablet for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:52.366348  2601 tablet_service.cc:1511] Processing CreateTablet for tablet ff9fe3d5f686477aa2bc32f628dcfb08 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 14:50:52.371480  2599 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb844e4a36641d79afdab73422e06f1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:52.373080  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498c3de2b5414c968242f7f8fc2ca7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.374500  2805 tablet_bootstrap.cc:654] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.377952  2601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9fe3d5f686477aa2bc32f628dcfb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.378808  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb844e4a36641d79afdab73422e06f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.379824  2748 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f08b71bff4091b9f08d398c3f894d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:52.380962  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.381341  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f08b71bff4091b9f08d398c3f894d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.387009  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f08b71bff4091b9f08d398c3f894d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:52.388954  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f08b71bff4091b9f08d398c3f894d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.400259  2676 tablet_service.cc:1511] Processing CreateTablet for tablet acb30692225e4daebaa7e02e9cb291b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION VALUES < 36398027
W20260709 14:50:52.401862  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:52.402251  2676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb30692225e4daebaa7e02e9cb291b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.404166  2673 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb844e4a36641d79afdab73422e06f1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 14:50:52.405812  2673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb844e4a36641d79afdab73422e06f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.402648  2674 tablet_service.cc:1511] Processing CreateTablet for tablet 498c3de2b5414c968242f7f8fc2ca7a1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 14:50:52.400497  2675 tablet_service.cc:1511] Processing CreateTablet for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 14:50:52.408564  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498c3de2b5414c968242f7f8fc2ca7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.416949  2672 tablet_service.cc:1511] Processing CreateTablet for tablet a594c31352514fb685540e1082f8ee7d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:52.418627  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a594c31352514fb685540e1082f8ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.419756  2675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c0ea3d097a4b27b9367e3b84b6d5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.410526  2598 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f08b71bff4091b9f08d398c3f894d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 14:50:52.422535  2598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f08b71bff4091b9f08d398c3f894d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.429994  2806 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:52.439159  2747 tablet_service.cc:1511] Processing CreateTablet for tablet a594c31352514fb685540e1082f8ee7d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:52.441037  2747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a594c31352514fb685540e1082f8ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.458731  2523 tablet_service.cc:1511] Processing CreateTablet for tablet a594c31352514fb685540e1082f8ee7d (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 14:50:52.458909  2597 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab522322e3448f18a90387481cab3ff (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:52.460580  2597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab522322e3448f18a90387481cab3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.469877  2806 tablet_bootstrap.cc:654] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.471379  2671 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab522322e3448f18a90387481cab3ff (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:52.480204  2671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab522322e3448f18a90387481cab3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.475823  2748 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab522322e3448f18a90387481cab3ff (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 14:50:52.487982  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab522322e3448f18a90387481cab3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.488687  2525 tablet_service.cc:1511] Processing CreateTablet for tablet 7584380efde04ee289b42acaea2e4742 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:52.491644  2807 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:52.460573  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a594c31352514fb685540e1082f8ee7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.492573  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7584380efde04ee289b42acaea2e4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.514003  2600 tablet_service.cc:1511] Processing CreateTablet for tablet 7584380efde04ee289b42acaea2e4742 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:52.515611  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7584380efde04ee289b42acaea2e4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.538380  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1073216da14f9c91bcc3f31f935620 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:52.540036  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1073216da14f9c91bcc3f31f935620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.540777  2527 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1073216da14f9c91bcc3f31f935620 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:52.542192  2527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1073216da14f9c91bcc3f31f935620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.533600  2670 tablet_service.cc:1511] Processing CreateTablet for tablet 7584380efde04ee289b42acaea2e4742 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 14:50:52.544338  2670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7584380efde04ee289b42acaea2e4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.556485  2669 tablet_service.cc:1511] Processing CreateTablet for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:52.558079  2669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.560232  2807 tablet_bootstrap.cc:654] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.554553  2599 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1073216da14f9c91bcc3f31f935620 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 14:50:52.564006  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1073216da14f9c91bcc3f31f935620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.569160  2805 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:52.579699  2805 ts_tablet_manager.cc:1403] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.233s	user 0.020s	sys 0.011s
I20260709 14:50:52.581558  2747 tablet_service.cc:1511] Processing CreateTablet for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:52.583222  2747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.597739  2596 tablet_service.cc:1511] Processing CreateTablet for tablet 6cf25c436e92407284806ff0a6ae57b0 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:52.604357  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 6cf25c436e92407284806ff0a6ae57b0 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:52.606105  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cf25c436e92407284806ff0a6ae57b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.609490  2673 tablet_service.cc:1511] Processing CreateTablet for tablet 6cf25c436e92407284806ff0a6ae57b0 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 14:50:52.609516  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 14:50:52.612852  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c52e7cc9c9f460c9c052fe21cfb57d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.615525  2596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cf25c436e92407284806ff0a6ae57b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.619441  2673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cf25c436e92407284806ff0a6ae57b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.619976  2562 heartbeater.cc:499] Master 127.0.107.254:39575 was elected leader, sending a full tablet report...
I20260709 14:50:52.583395  2805 raft_consensus.cc:359] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.635519  2805 raft_consensus.cc:385] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:52.647529  2805 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:52.671993  2805 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.722909  2811 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:52.739426  2805 ts_tablet_manager.cc:1434] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.159s	user 0.011s	sys 0.010s
I20260709 14:50:52.740899  2637 heartbeater.cc:499] Master 127.0.107.254:39575 was elected leader, sending a full tablet report...
I20260709 14:50:52.746996  2805 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:52.747762  2785 heartbeater.cc:499] Master 127.0.107.254:39575 was elected leader, sending a full tablet report...
I20260709 14:50:52.751319  2811 tablet_bootstrap.cc:654] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.755527  2805 tablet_bootstrap.cc:654] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.767205  2806 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:52.767935  2806 ts_tablet_manager.cc:1403] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.338s	user 0.014s	sys 0.009s
I20260709 14:50:52.756279  2807 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:52.777213  2807 ts_tablet_manager.cc:1403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.286s	user 0.032s	sys 0.024s
I20260709 14:50:52.812590  2806 raft_consensus.cc:359] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.813520  2806 raft_consensus.cc:385] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:52.813830  2806 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:52.817106  2524 tablet_service.cc:1511] Processing CreateTablet for tablet c1bae386d4ba412d86a21df33a9fa819 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:52.818727  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bae386d4ba412d86a21df33a9fa819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.831820  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:52.833578  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.838096  2672 tablet_service.cc:1511] Processing CreateTablet for tablet c1bae386d4ba412d86a21df33a9fa819 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:52.839727  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bae386d4ba412d86a21df33a9fa819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.814651  2806 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.849515  2527 tablet_service.cc:1511] Processing CreateTablet for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:52.851253  2527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.853963  2711 heartbeater.cc:499] Master 127.0.107.254:39575 was elected leader, sending a full tablet report...
I20260709 14:50:52.849583  2807 raft_consensus.cc:359] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:52.864780  2807 raft_consensus.cc:385] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:52.865329  2807 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:52.873733  2805 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:52.874409  2805 ts_tablet_manager.cc:1403] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.128s	user 0.030s	sys 0.025s
I20260709 14:50:52.883455  2805 raft_consensus.cc:359] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:52.883306  2747 tablet_service.cc:1511] Processing CreateTablet for tablet 6baf9a66ae7e41df9d5d2079bebb7381 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:52.884577  2805 raft_consensus.cc:385] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:52.885047  2805 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:52.885584  2747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baf9a66ae7e41df9d5d2079bebb7381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.886142  2805 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:52.897269  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 6baf9a66ae7e41df9d5d2079bebb7381 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:52.908990  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baf9a66ae7e41df9d5d2079bebb7381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.866401  2807 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:52.922232  2811 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:52.923039  2811 ts_tablet_manager.cc:1403] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.201s	user 0.033s	sys 0.017s
I20260709 14:50:52.926134  2811 raft_consensus.cc:359] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.937021  2811 raft_consensus.cc:385] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:52.937448  2811 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:52.944427  2524 tablet_service.cc:1511] Processing CreateTablet for tablet 4837b30553bb4eac8840db77c7ffca53 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:52.946185  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4837b30553bb4eac8840db77c7ffca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.938447  2811 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:52.968005  2811 ts_tablet_manager.cc:1434] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.003s
I20260709 14:50:52.969339  2811 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:52.983783  2527 tablet_service.cc:1511] Processing CreateTablet for tablet 1f87ae033ca94d58bdc3a087236657b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:52.986395  2527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f87ae033ca94d58bdc3a087236657b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.987252  2673 tablet_service.cc:1511] Processing CreateTablet for tablet 1591def49c3c47edac03ade007ebc91a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:52.988842  2673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1591def49c3c47edac03ade007ebc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:52.989171  2811 tablet_bootstrap.cc:654] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:52.991796  2524 tablet_service.cc:1511] Processing CreateTablet for tablet 343553f3139e4da4ba16879fae174c5a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:52.993379  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343553f3139e4da4ba16879fae174c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.000488  2523 tablet_service.cc:1511] Processing CreateTablet for tablet e9427dea4aea4238bc56533da5d2720e (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:53.002218  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9427dea4aea4238bc56533da5d2720e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.010056  2806 ts_tablet_manager.cc:1434] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.242s	user 0.026s	sys 0.016s
I20260709 14:50:53.016000  2806 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:53.025344  2668 tablet_service.cc:1511] Processing CreateTablet for tablet b188cd7e4b5e4d9fb22a3645bdff7da1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:53.056481  2668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b188cd7e4b5e4d9fb22a3645bdff7da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.028226  2669 tablet_service.cc:1511] Processing CreateTablet for tablet 343553f3139e4da4ba16879fae174c5a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:53.059994  2669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343553f3139e4da4ba16879fae174c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.061520  2522 tablet_service.cc:1511] Processing CreateTablet for tablet 750a16eb87de49b4a212064463a9d3a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:53.030055  2746 tablet_service.cc:1511] Processing CreateTablet for tablet 343553f3139e4da4ba16879fae174c5a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 14:50:53.063200  2522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750a16eb87de49b4a212064463a9d3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.032035  2521 tablet_service.cc:1511] Processing CreateTablet for tablet 100d7ba791034203ab40e147a55e2183 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:53.064386  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d7ba791034203ab40e147a55e2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.034487  2667 tablet_service.cc:1511] Processing CreateTablet for tablet 100d7ba791034203ab40e147a55e2183 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:53.067840  2667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d7ba791034203ab40e147a55e2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.037400  2595 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea5acf8ab374c8fa3009782429fd3ed (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:53.071841  2595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea5acf8ab374c8fa3009782429fd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.037768  2594 tablet_service.cc:1511] Processing CreateTablet for tablet e9427dea4aea4238bc56533da5d2720e (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:53.075799  2594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9427dea4aea4238bc56533da5d2720e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.039994  2601 tablet_service.cc:1511] Processing CreateTablet for tablet 4837b30553bb4eac8840db77c7ffca53 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:53.077806  2601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4837b30553bb4eac8840db77c7ffca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.077878  2520 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac95ecab0524f35a894d205dac97a44 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:53.041771  2602 tablet_service.cc:1511] Processing CreateTablet for tablet 1f87ae033ca94d58bdc3a087236657b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:53.079630  2520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac95ecab0524f35a894d205dac97a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.079769  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f87ae033ca94d58bdc3a087236657b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.079418  2671 tablet_service.cc:1511] Processing CreateTablet for tablet 750a16eb87de49b4a212064463a9d3a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:53.080924  2671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750a16eb87de49b4a212064463a9d3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.083508  2527 tablet_service.cc:1511] Processing CreateTablet for tablet da98c08aee5d4d86bce741e08142c1e3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:53.086362  2527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da98c08aee5d4d86bce741e08142c1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.044551  2598 tablet_service.cc:1511] Processing CreateTablet for tablet 1591def49c3c47edac03ade007ebc91a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:53.091223  2598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1591def49c3c47edac03ade007ebc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.092106  2742 tablet_service.cc:1511] Processing CreateTablet for tablet b188cd7e4b5e4d9fb22a3645bdff7da1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:53.095151  2748 tablet_service.cc:1511] Processing CreateTablet for tablet 1591def49c3c47edac03ade007ebc91a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 14:50:53.096810  2672 tablet_service.cc:1511] Processing CreateTablet for tablet e91eefb5789d41f0b4cd60e9f83fef3f (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:53.047190  2674 tablet_service.cc:1511] Processing CreateTablet for tablet e9427dea4aea4238bc56533da5d2720e (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 14:50:53.049131  2599 tablet_service.cc:1511] Processing CreateTablet for tablet c1bae386d4ba412d86a21df33a9fa819 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 14:50:53.098955  2743 tablet_service.cc:1511] Processing CreateTablet for tablet 750a16eb87de49b4a212064463a9d3a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 14:50:53.100343  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bae386d4ba412d86a21df33a9fa819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.100852  2815 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.101435  2815 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.102522  2818 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.103096  2818 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:53.104194  2675 tablet_service.cc:1511] Processing CreateTablet for tablet 4837b30553bb4eac8840db77c7ffca53 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 14:50:53.105782  2744 tablet_service.cc:1511] Processing CreateTablet for tablet 9a8bd118baa7481399ce15d422463083 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:53.098969  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91eefb5789d41f0b4cd60e9f83fef3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.116988  2815 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:53.119333  2675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4837b30553bb4eac8840db77c7ffca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.119830  2596 tablet_service.cc:1511] Processing CreateTablet for tablet b188cd7e4b5e4d9fb22a3645bdff7da1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 14:50:53.121201  2596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b188cd7e4b5e4d9fb22a3645bdff7da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.051034  2750 tablet_service.cc:1511] Processing CreateTablet for tablet 1f87ae033ca94d58bdc3a087236657b7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 14:50:53.052840  2523 tablet_service.cc:1511] Processing CreateTablet for tablet e91eefb5789d41f0b4cd60e9f83fef3f (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:53.123870  2812 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.114192  2670 tablet_service.cc:1511] Processing CreateTablet for tablet 6baf9a66ae7e41df9d5d2079bebb7381 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 14:50:53.053046  2745 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea5acf8ab374c8fa3009782429fd3ed (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:53.124446  2812 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.125154  2670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baf9a66ae7e41df9d5d2079bebb7381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.038554  2597 tablet_service.cc:1511] Processing CreateTablet for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 14:50:53.126751  2597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c97f578e0e4d3f9c754fb31c0b4c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.012924  2526 tablet_service.cc:1511] Processing CreateTablet for tablet 9a8bd118baa7481399ce15d422463083 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:53.085973  2741 tablet_service.cc:1511] Processing CreateTablet for tablet a57a97528f31400dbc0900e76292b021 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:53.131393  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9427dea4aea4238bc56533da5d2720e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.133451  2676 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea5acf8ab374c8fa3009782429fd3ed (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 14:50:53.135974  2676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea5acf8ab374c8fa3009782429fd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.134519  2600 tablet_service.cc:1511] Processing CreateTablet for tablet 100d7ba791034203ab40e147a55e2183 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 14:50:53.138073  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100d7ba791034203ab40e147a55e2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.149148  2746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343553f3139e4da4ba16879fae174c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.150935  2742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b188cd7e4b5e4d9fb22a3645bdff7da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.152611  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1591def49c3c47edac03ade007ebc91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.154913  2743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750a16eb87de49b4a212064463a9d3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.158819  2750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f87ae033ca94d58bdc3a087236657b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.159545  2816 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.160346  2816 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.161813  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a8bd118baa7481399ce15d422463083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.168126  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91eefb5789d41f0b4cd60e9f83fef3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.170740  2745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea5acf8ab374c8fa3009782429fd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.175669  2741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57a97528f31400dbc0900e76292b021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.173544  2526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a8bd118baa7481399ce15d422463083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.043440  2524 tablet_service.cc:1511] Processing CreateTablet for tablet a57a97528f31400dbc0900e76292b021 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:53.186324  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57a97528f31400dbc0900e76292b021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.188068  2593 tablet_service.cc:1511] Processing CreateTablet for tablet 9a8bd118baa7481399ce15d422463083 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 14:50:53.189780  2593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a8bd118baa7481399ce15d422463083. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:53.194334  2422 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71029us
I20260709 14:50:53.198096  2519 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9cf6350d3b499497e3ace2d1dea988 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:53.200009  2519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9cf6350d3b499497e3ace2d1dea988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.202086  2816 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:53.226646  2805 ts_tablet_manager.cc:1434] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.352s	user 0.011s	sys 0.003s
I20260709 14:50:53.246425  2822 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.300356  2818 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:53.303376  2805 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:53.280864  2822 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:53.318382  2812 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:50:53.333693  2525 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.795s	user 0.014s	sys 0.008s
I20260709 14:50:53.347121  2807 ts_tablet_manager.cc:1434] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.564s	user 0.030s	sys 0.026s
I20260709 14:50:53.361184  2811 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
W20260709 14:50:53.358634  2807 ts_tablet_manager.cc:1467] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Tablet startup took 580ms
I20260709 14:50:53.368690  2822 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:53.398985  2811 ts_tablet_manager.cc:1403] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.430s	user 0.048s	sys 0.036s
W20260709 14:50:53.397468  2807 ts_tablet_manager.cc:1469] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Trace:
0709 14:50:52.365236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:52.365867 (+   631us) service_pool.cc:224] Handling call
0709 14:50:52.368348 (+  2481us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:52.368480 (+   132us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:52.409021 (+ 40541us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:52.418494 (+  9473us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:52.490949 (+ 72455us) inbound_call.cc:177] Queueing success response
0709 14:50:52.491098 (+   149us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 14:50:52.780774 (+289676us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 14:50:52.846930 (+ 66156us) tablet_replica.cc:261] Starting instrumentation
0709 14:50:52.847634 (+   704us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":117271,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":197,"tablet-open.queue_time_us":72114,"thread_start_us":194790,"threads_started":2}
I20260709 14:50:53.425453  2807 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:53.425135  2525 tablet_service.cc:1511] Processing CreateTablet for tablet 80e0e0c146c54411901808c4bbf22280 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:53.426795  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:53.429015  2652 leader_election.cc:343] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.372372  2518 tablet_service.cc:1511] Processing CreateTablet for tablet a2e7eae198c34a6fafb697d50f269192 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:53.426699  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e0e0c146c54411901808c4bbf22280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.455250  2518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2e7eae198c34a6fafb697d50f269192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.422478  2811 raft_consensus.cc:359] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.466622  2811 raft_consensus.cc:385] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:53.467063  2811 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:53.525310  2675 tablet_service.cc:1511] Processing CreateTablet for tablet c39462fb47b74715a7e5498d49eaab87 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:53.495939  2811 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:50:53.542943  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:53.568073  2811 ts_tablet_manager.cc:1434] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.149s	user 0.003s	sys 0.006s
I20260709 14:50:53.543064  2806 tablet_bootstrap.cc:654] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:53.577625  2805 tablet_bootstrap.cc:654] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:53.585045  2811 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
W20260709 14:50:53.587395  2419 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57794.3us
I20260709 14:50:53.589464  2807 tablet_bootstrap.cc:654] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:53.593204  2741 tablet_service.cc:1511] Processing CreateTablet for tablet e91eefb5789d41f0b4cd60e9f83fef3f (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 14:50:53.593508  2521 tablet_service.cc:1511] Processing CreateTablet for tablet dee6ef23c038421d972947971af9f0cc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:53.601035  2675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c39462fb47b74715a7e5498d49eaab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.594726  2741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91eefb5789d41f0b4cd60e9f83fef3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.594726  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee6ef23c038421d972947971af9f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.620000  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac95ecab0524f35a894d205dac97a44 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:53.622783  2600 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac95ecab0524f35a894d205dac97a44 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 14:50:53.624415  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac95ecab0524f35a894d205dac97a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.633504  2674 tablet_service.cc:1511] Processing CreateTablet for tablet da98c08aee5d4d86bce741e08142c1e3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:53.639861  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:50:53.642989  2577 leader_election.cc:343] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.645628  2594 tablet_service.cc:1511] Processing CreateTablet for tablet a2e7eae198c34a6fafb697d50f269192 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:53.651288  2595 tablet_service.cc:1511] Processing CreateTablet for tablet a57a97528f31400dbc0900e76292b021 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 14:50:53.654886  2597 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2b04c469b2404c908d67b076cf2ba4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:53.675310  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac95ecab0524f35a894d205dac97a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.675570  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da98c08aee5d4d86bce741e08142c1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.690872  2746 tablet_service.cc:1511] Processing CreateTablet for tablet 80e0e0c146c54411901808c4bbf22280 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:53.692533  2746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e0e0c146c54411901808c4bbf22280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.692368  2672 tablet_service.cc:1511] Processing CreateTablet for tablet a2e7eae198c34a6fafb697d50f269192 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 14:50:53.691005  2671 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2b04c469b2404c908d67b076cf2ba4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:53.700167  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:53.701053  2686 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:53.701906  2668 tablet_service.cc:1511] Processing CreateTablet for tablet dee6ef23c038421d972947971af9f0cc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:53.662457  2748 tablet_service.cc:1511] Processing CreateTablet for tablet c39462fb47b74715a7e5498d49eaab87 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:53.705839  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c39462fb47b74715a7e5498d49eaab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.707381  2500 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:53.707610  2599 tablet_service.cc:1511] Processing CreateTablet for tablet da98c08aee5d4d86bce741e08142c1e3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 14:50:53.688885  2669 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9cf6350d3b499497e3ace2d1dea988 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:53.709483  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:50:53.711840  2575 leader_election.cc:343] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.711903  2598 tablet_service.cc:1511] Processing CreateTablet for tablet c39462fb47b74715a7e5498d49eaab87 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 14:50:53.712414  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:53.713914  2818 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:53.702227  2600 tablet_service.cc:1511] Processing CreateTablet for tablet 92641606b7ec403d843b2ee872402de3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:53.715113  2669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9cf6350d3b499497e3ace2d1dea988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.724128  2594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2e7eae198c34a6fafb697d50f269192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.668586  2602 tablet_service.cc:1511] Processing CreateTablet for tablet 80e0e0c146c54411901808c4bbf22280 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 14:50:53.731488  2743 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9cf6350d3b499497e3ace2d1dea988 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 14:50:53.733075  2743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9cf6350d3b499497e3ace2d1dea988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.738379  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:53.738919  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e0e0c146c54411901808c4bbf22280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.740283  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92641606b7ec403d843b2ee872402de3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:53.743085  2503 leader_election.cc:343] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.746311  2525 tablet_service.cc:1511] Processing CreateTablet for tablet eeb3e91a07fc4383bd55f034f23c3cea (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:53.681380  2747 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2b04c469b2404c908d67b076cf2ba4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 14:50:53.753297  2670 tablet_service.cc:1511] Processing CreateTablet for tablet 92641606b7ec403d843b2ee872402de3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:53.756907  2747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b04c469b2404c908d67b076cf2ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.754729  2598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c39462fb47b74715a7e5498d49eaab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.787020  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb3e91a07fc4383bd55f034f23c3cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.790030  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:50:53.792026  2723 leader_election.cc:343] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.790099  2811 tablet_bootstrap.cc:654] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:53.793148  2745 tablet_service.cc:1511] Processing CreateTablet for tablet dee6ef23c038421d972947971af9f0cc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 14:50:53.796015  2597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b04c469b2404c908d67b076cf2ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.794768  2745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee6ef23c038421d972947971af9f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.815598  2432 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 2) took 1058 ms. Trace:
I20260709 14:50:53.812903  2671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2b04c469b2404c908d67b076cf2ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.816937  2595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57a97528f31400dbc0900e76292b021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.828569  2668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee6ef23c038421d972947971af9f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.841444  2839 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:53.823166  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:53.816217  2432 rpcz_store.cc:276] 0709 14:50:52.757207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:52.757405 (+   198us) service_pool.cc:224] Handling call
0709 14:50:53.815348 (+1057943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20023,"reported_tablets":8}
I20260709 14:50:53.856822  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da98c08aee5d4d86bce741e08142c1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.857764  2593 tablet_service.cc:1511] Processing CreateTablet for tablet eeb3e91a07fc4383bd55f034f23c3cea (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:53.860116  2593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb3e91a07fc4383bd55f034f23c3cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.868744  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2e7eae198c34a6fafb697d50f269192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.873008  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 92641606b7ec403d843b2ee872402de3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 14:50:53.879634  2524 tablet_service.cc:1511] Processing CreateTablet for tablet a1826b730f9f4ae4b0e05570574563c3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:53.882652  2670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92641606b7ec403d843b2ee872402de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.883980  2667 tablet_service.cc:1511] Processing CreateTablet for tablet eeb3e91a07fc4383bd55f034f23c3cea (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 14:50:53.885641  2667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb3e91a07fc4383bd55f034f23c3cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.891364  2816 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:53.892138  2816 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.896165  2816 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:53.874713  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92641606b7ec403d843b2ee872402de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.891259  2807 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:53.898056  2807 ts_tablet_manager.cc:1403] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.473s	user 0.017s	sys 0.011s
I20260709 14:50:53.855707  2839 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:53.916558  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1826b730f9f4ae4b0e05570574563c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:53.917900  2726 leader_election.cc:343] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:53.924080  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08002a32a0 after lost signal to thread 434
W20260709 14:50:53.929451  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800199e00 after lost signal to thread 2469
I20260709 14:50:53.932142  2676 tablet_service.cc:1511] Processing CreateTablet for tablet f32f10706d04497089f6159669dc04a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:53.915171  2839 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:53.933884  2676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32f10706d04497089f6159669dc04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.918736  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:53.946090  2839 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.941974  2805 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:53.955067  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:50:53.957247  2503 leader_election.cc:343] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.915405  2807 raft_consensus.cc:359] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:53.965698  2744 tablet_service.cc:1511] Processing CreateTablet for tablet f32f10706d04497089f6159669dc04a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:53.967700  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32f10706d04497089f6159669dc04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.969022  2748 tablet_service.cc:1511] Processing CreateTablet for tablet a1826b730f9f4ae4b0e05570574563c3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:53.970659  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1826b730f9f4ae4b0e05570574563c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.976554  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:53.978580  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:50:53.981016  2724 leader_election.cc:343] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:53.981604  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:53.987632  2602 tablet_service.cc:1511] Processing CreateTablet for tablet a1826b730f9f4ae4b0e05570574563c3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 14:50:53.988024  2599 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2c3e021741a28db500c36ebdc494 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:53.988372  2520 tablet_service.cc:1511] Processing CreateTablet for tablet f32f10706d04497089f6159669dc04a4 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 14:50:53.989482  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1826b730f9f4ae4b0e05570574563c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.989792  2520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32f10706d04497089f6159669dc04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.990171  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2c3e021741a28db500c36ebdc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.991384  2675 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2c3e021741a28db500c36ebdc494 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:53.992846  2675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2c3e021741a28db500c36ebdc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:53.963016  2805 ts_tablet_manager.cc:1403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.676s	user 0.049s	sys 0.076s
I20260709 14:50:54.001992  2805 raft_consensus.cc:359] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:53.961781  2839 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:54.002810  2805 raft_consensus.cc:385] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.026859  2805 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.003664  2806 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
W20260709 14:50:54.021780  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba8c0 after lost signal to thread 2802
I20260709 14:50:54.041199  2806 ts_tablet_manager.cc:1403] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 1.025s	user 0.016s	sys 0.034s
I20260709 14:50:54.044768  2811 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:54.045526  2811 ts_tablet_manager.cc:1403] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.461s	user 0.014s	sys 0.024s
I20260709 14:50:54.046344  2595 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55856 (request call id 19) took 1002 ms. Trace:
I20260709 14:50:54.050614  2595 rpcz_store.cc:276] 0709 14:50:53.043685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:53.586766 (+543081us) service_pool.cc:224] Handling call
0709 14:50:53.706609 (+119843us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17367a831 00007fa173657ac3 00007fa173606400 00007fa16ca8baf4 00007fa16ca8ba82 00007fa16ca8b9b1
0709 14:50:53.706807 (+   198us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:53.706961 (+   154us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:53.982040 (+275079us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:54.014484 (+ 32444us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:54.046261 (+ 31777us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38566784}
W20260709 14:50:54.049994  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 342884us
I20260709 14:50:54.014269  2807 raft_consensus.cc:385] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.054536  2805 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.054873  2807 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.069379  2811 raft_consensus.cc:359] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.080782  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:54.081050  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:54.113672  2686 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.111310  2598 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55856 (request call id 20) took 1056 ms. Trace:
I20260709 14:50:54.116919  2598 rpcz_store.cc:276] 0709 14:50:53.054615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:53.603185 (+548570us) service_pool.cc:224] Handling call
0709 14:50:53.753973 (+150788us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17367a831 00007fa173657ac3 00007fa173606400 00007fa16ca8baf4 00007fa16ca8ba82 00007fa16ca8b9b1
0709 14:50:53.754141 (+   168us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:53.754250 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:54.063330 (+309080us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:54.076951 (+ 13621us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:54.110529 (+ 33578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12107136}
I20260709 14:50:54.111229  2811 raft_consensus.cc:385] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.119201  2811 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.120246  2807 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.121199  2521 tablet_service.cc:1511] Processing CreateTablet for tablet 31e4166c53c7459b808e9e6841d90c56 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 14:50:54.120246  2811 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.122699  2612 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:54.124012  2598 tablet_service.cc:1511] Processing CreateTablet for tablet 31e4166c53c7459b808e9e6841d90c56 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 14:50:54.125514  2598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e4166c53c7459b808e9e6841d90c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.131542  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e4166c53c7459b808e9e6841d90c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.132714  2811 ts_tablet_manager.cc:1434] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.003s
W20260709 14:50:54.103425  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 122648us
I20260709 14:50:54.147410  2670 tablet_service.cc:1511] Processing CreateTablet for tablet 31e4166c53c7459b808e9e6841d90c56 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 14:50:54.146504  2806 raft_consensus.cc:359] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:54.149091  2670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e4166c53c7459b808e9e6841d90c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.151057  2742 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2c3e021741a28db500c36ebdc494 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 14:50:54.152915  2811 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:54.148715  2741 tablet_service.cc:1511] Processing CreateTablet for tablet 0d27004b985a45779b0f86fcb7273dbc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:54.128017  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:54.155226  2806 raft_consensus.cc:385] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.158190  2806 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.158422  2536 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:50:54.159649  2724 leader_election.cc:343] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.159308  2806 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:54.160210  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:54.166270  2811 tablet_bootstrap.cc:654] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.172824  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:54.174531  2807 ts_tablet_manager.cc:1434] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.276s	user 0.010s	sys 0.000s
I20260709 14:50:54.174877  2816 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:54.175379  2816 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:54.175745  2816 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.176275  2807 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:54.175827  2518 tablet_service.cc:1511] Processing CreateTablet for tablet 0d27004b985a45779b0f86fcb7273dbc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:54.153570  2843 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:50:54.183300  2650 leader_election.cc:343] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.185585  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:54.183804  2518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d27004b985a45779b0f86fcb7273dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.188071  2674 tablet_service.cc:1511] Processing CreateTablet for tablet b5cca18a56d54a9aae32ff20c48ef0f8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:54.194809  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5cca18a56d54a9aae32ff20c48ef0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.196851  2843 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:54.180593  2742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2c3e021741a28db500c36ebdc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.200888  2595 tablet_service.cc:1511] Processing CreateTablet for tablet 0d27004b985a45779b0f86fcb7273dbc (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 14:50:54.201289  2600 tablet_service.cc:1511] Processing CreateTablet for tablet e5f95289d39644bead88ff8640742871 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:54.202800  2672 tablet_service.cc:1511] Processing CreateTablet for tablet e5f95289d39644bead88ff8640742871 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:54.204219  2524 tablet_service.cc:1511] Processing CreateTablet for tablet b5cca18a56d54a9aae32ff20c48ef0f8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:54.204839  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f95289d39644bead88ff8640742871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.205621  2524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5cca18a56d54a9aae32ff20c48ef0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.211606  2741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d27004b985a45779b0f86fcb7273dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.223038  2595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d27004b985a45779b0f86fcb7273dbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.223706  2669 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45608 (request call id 22) took 1108 ms. Trace:
I20260709 14:50:54.224179  2669 rpcz_store.cc:276] 0709 14:50:53.115304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:50:53.687613 (+572309us) service_pool.cc:224] Handling call
0709 14:50:53.710278 (+ 22665us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 14:50:53.710406 (+   128us) ts_tablet_manager.cc:682] Creating new metadata...
0709 14:50:53.981015 (+270609us) tablet_metadata.cc:665] Metadata flushed
0709 14:50:54.155260 (+174245us) tablet_replica.cc:206] Creating consensus instance
0709 14:50:54.223305 (+ 68045us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b52230e 00007fa174463bf9 00007fa173594e51 00007fa17365bb1b 00007fa17365770f 00007fa173657ecd 00007fa173606400 00007fa16ca8baf4 00007fa16ca8ba82
0709 14:50:54.223631 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5366,"spinlock_wait_cycles":5869824}
I20260709 14:50:54.231740  2686 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:54.233474  2500 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:54.234970  2839 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:54.237111  2806 ts_tablet_manager.cc:1434] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.195s	user 0.010s	sys 0.004s
I20260709 14:50:54.249137  2748 tablet_service.cc:1511] Processing CreateTablet for tablet b5cca18a56d54a9aae32ff20c48ef0f8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 14:50:54.250761  2748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5cca18a56d54a9aae32ff20c48ef0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.251775  2746 tablet_service.cc:1511] Processing CreateTablet for tablet e5f95289d39644bead88ff8640742871 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 14:50:54.251580  2816 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.253319  2746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f95289d39644bead88ff8640742871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.285748  2600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f95289d39644bead88ff8640742871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.260110  2805 ts_tablet_manager.cc:1434] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.262s	user 0.008s	sys 0.016s
I20260709 14:50:54.285132  2806 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:54.256501  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:54.254736  2816 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:50:54.306115  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 120625us
I20260709 14:50:54.318357  2744 tablet_service.cc:1511] Processing CreateTablet for tablet 8b56ad927e054a56a77c811620dda543 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 14:50:54.329752  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b56ad927e054a56a77c811620dda543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.276459  2597 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc2eae2c0540e4bc1c7141e81bf63c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:54.336256  2597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc2eae2c0540e4bc1c7141e81bf63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.336683  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:50:54.305265  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 145647us
I20260709 14:50:54.339704  2839 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:54.339772  2612 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.273684  2525 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc2eae2c0540e4bc1c7141e81bf63c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:54.278784  2676 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc2eae2c0540e4bc1c7141e81bf63c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 14:50:54.357169  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc2eae2c0540e4bc1c7141e81bf63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.358503  2811 tablet_bootstrap.cc:492] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:54.359167  2811 ts_tablet_manager.cc:1403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.207s	user 0.012s	sys 0.012s
I20260709 14:50:54.358361  2593 tablet_service.cc:1511] Processing CreateTablet for tablet 8b56ad927e054a56a77c811620dda543 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 14:50:54.360531  2593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b56ad927e054a56a77c811620dda543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.365456  2846 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:50:54.348930  2650 leader_election.cc:343] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.356279  2805 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:54.372510  2839 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.373536  2807 tablet_bootstrap.cc:654] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.379273  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 8e31634415994959847625e41eecf9b3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:54.379930  2741 tablet_service.cc:1511] Processing CreateTablet for tablet 8e31634415994959847625e41eecf9b3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:54.380985  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e31634415994959847625e41eecf9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.381434  2741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e31634415994959847625e41eecf9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.382373  2676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc2eae2c0540e4bc1c7141e81bf63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.384572  2667 tablet_service.cc:1511] Processing CreateTablet for tablet 8e31634415994959847625e41eecf9b3 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 14:50:54.386734  2845 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:54.387390  2845 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.397141  2521 tablet_service.cc:1511] Processing CreateTablet for tablet 8b56ad927e054a56a77c811620dda543 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 14:50:54.398617  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:54.400683  2845 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:54.402658  2594 tablet_service.cc:1511] Processing CreateTablet for tablet 6340ce5f0bc04ded8681c8573a0910c1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 14:50:54.408360  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61369.9us
I20260709 14:50:54.410698  2611 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:54.411242  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:50:54.432370  2723 leader_election.cc:343] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.421737  2667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e31634415994959847625e41eecf9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.428226  2594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6340ce5f0bc04ded8681c8573a0910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.429675  2674 tablet_service.cc:1511] Processing CreateTablet for tablet 6340ce5f0bc04ded8681c8573a0910c1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 14:50:54.426434  2612 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:54.438961  2749 tablet_service.cc:1511] Processing CreateTablet for tablet 6340ce5f0bc04ded8681c8573a0910c1 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 14:50:54.436558  2846 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:54.462486  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:54.464250  2816 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:54.451965  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:54.468178  2845 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:54.468629  2845 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:54.468984  2845 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.466393  2846 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.470579  2432 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:54.474368  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b56ad927e054a56a77c811620dda543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.485749  2806 tablet_bootstrap.cc:654] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.496959  2805 tablet_bootstrap.cc:654] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.498713  2602 tablet_service.cc:1511] Processing CreateTablet for tablet 1c902c27b99b4cc0a5186bc8117246a7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:54.500339  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c902c27b99b4cc0a5186bc8117246a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.500530  2520 tablet_service.cc:1511] Processing CreateTablet for tablet 1c902c27b99b4cc0a5186bc8117246a7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:54.501940  2520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c902c27b99b4cc0a5186bc8117246a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.512053  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6340ce5f0bc04ded8681c8573a0910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.515812  2672 tablet_service.cc:1511] Processing CreateTablet for tablet 1c902c27b99b4cc0a5186bc8117246a7 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 14:50:54.516637  2744 tablet_service.cc:1511] Processing CreateTablet for tablet 75e6d6e528724dcba0bbaf89427c37a9 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:54.517299  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c902c27b99b4cc0a5186bc8117246a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.523332  2811 raft_consensus.cc:359] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.524343  2811 raft_consensus.cc:385] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.524720  2811 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.525759  2811 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.511555  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6340ce5f0bc04ded8681c8573a0910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.528832  2807 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:54.529376  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e6d6e528724dcba0bbaf89427c37a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.529475  2807 ts_tablet_manager.cc:1403] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.354s	user 0.028s	sys 0.030s
I20260709 14:50:54.532941  2807 raft_consensus.cc:359] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.533751  2807 raft_consensus.cc:385] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.534096  2807 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.554679  2846 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:54.556576  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:54.558853  2652 leader_election.cc:343] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.559350  2518 tablet_service.cc:1511] Processing CreateTablet for tablet 75e6d6e528724dcba0bbaf89427c37a9 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:54.560904  2518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e6d6e528724dcba0bbaf89427c37a9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:50:54.565183  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:54.513173  2816 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:54.576095  2599 tablet_service.cc:1511] Processing CreateTablet for tablet 75e6d6e528724dcba0bbaf89427c37a9 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 14:50:54.578655  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e6d6e528724dcba0bbaf89427c37a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.588716  2670 tablet_service.cc:1511] Processing CreateTablet for tablet 406cc4479d48441398435d7dc1850d64 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:54.593331  2670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406cc4479d48441398435d7dc1850d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.595527  2816 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.594630  2742 tablet_service.cc:1511] Processing CreateTablet for tablet 406cc4479d48441398435d7dc1850d64 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:54.599200  2742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406cc4479d48441398435d7dc1850d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.625643  2811 ts_tablet_manager.cc:1434] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.266s	user 0.023s	sys 0.033s
I20260709 14:50:54.628724  2811 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:54.631436  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 406cc4479d48441398435d7dc1850d64 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 14:50:54.634572  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406cc4479d48441398435d7dc1850d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.569176  2807 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.640648  2839 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:54.641435  2839 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:54.663679  2806 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:54.664368  2806 ts_tablet_manager.cc:1403] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.380s	user 0.012s	sys 0.016s
I20260709 14:50:54.667070  2845 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.667845  2806 raft_consensus.cc:359] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:54.668606  2806 raft_consensus.cc:385] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.668931  2806 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.683344  2806 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:54.686237  2806 ts_tablet_manager.cc:1434] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 14:50:54.688161  2593 tablet_service.cc:1511] Processing CreateTablet for tablet 194ae28527c14d78817238308fb284c6 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:54.708510  2676 tablet_service.cc:1511] Processing CreateTablet for tablet 194ae28527c14d78817238308fb284c6 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:54.717145  2593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194ae28527c14d78817238308fb284c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.718115  2676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194ae28527c14d78817238308fb284c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.723138  2807 ts_tablet_manager.cc:1434] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.193s	user 0.010s	sys 0.008s
I20260709 14:50:54.724500  2807 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:54.738855  2525 tablet_service.cc:1511] Processing CreateTablet for tablet f01d4b3ec58f4ca3b994c5e66179174a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:54.740491  2525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01d4b3ec58f4ca3b994c5e66179174a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.738106  2435 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:54.734699  2806 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:54.744588  2594 tablet_service.cc:1511] Processing CreateTablet for tablet f01d4b3ec58f4ca3b994c5e66179174a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:54.746068  2594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01d4b3ec58f4ca3b994c5e66179174a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.752646  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:54.755468  2503 leader_election.cc:343] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.755618  2746 tablet_service.cc:1511] Processing CreateTablet for tablet 194ae28527c14d78817238308fb284c6 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 14:50:54.756162  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:54.757411  2746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194ae28527c14d78817238308fb284c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.728742  2816 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:54.765072  2651 leader_election.cc:343] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.765993  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:54.767462  2860 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:54.768201  2860 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:54.768416  2853 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:54.759085  2839 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:54.783490  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:54.787602  2686 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:54.783182  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:54.790951  2523 tablet_service.cc:1511] Processing CreateTablet for tablet 947b4a5caeb243ef87d3a535eddda080 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:54.792605  2523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 947b4a5caeb243ef87d3a535eddda080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.775209  2860 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:54.800657  2602 tablet_service.cc:1511] Processing CreateTablet for tablet 947b4a5caeb243ef87d3a535eddda080 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:54.802784  2744 tablet_service.cc:1511] Processing CreateTablet for tablet 947b4a5caeb243ef87d3a535eddda080 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 14:50:54.813040  2669 tablet_service.cc:1511] Processing CreateTablet for tablet f01d4b3ec58f4ca3b994c5e66179174a (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 14:50:54.774003  2816 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:54.806483  2612 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.831707  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 947b4a5caeb243ef87d3a535eddda080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.820472  2805 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:54.806334  2839 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:54.833879  2805 ts_tablet_manager.cc:1403] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.483s	user 0.022s	sys 0.007s
I20260709 14:50:54.833959  2839 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:54.807713  2845 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:54.811321  2806 tablet_bootstrap.cc:654] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.826579  2674 tablet_service.cc:1511] Processing CreateTablet for tablet b47a7b94ee76444db9c73a90f520af7c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:54.827503  2669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01d4b3ec58f4ca3b994c5e66179174a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.827283  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:54.828150  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 947b4a5caeb243ef87d3a535eddda080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.869867  2674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47a7b94ee76444db9c73a90f520af7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.873117  2816 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:54.872999  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:54.872999  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:54.875327  2760 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:50:54.876243  2807 tablet_bootstrap.cc:654] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.877746  2611 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.877619  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:54.866416  2686 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.902297  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:54.904320  2839 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:54.905109  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:54.905490  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:54.905964  2760 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
W20260709 14:50:54.907786  2503 leader_election.cc:343] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:54.907965  2502 leader_election.cc:343] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:54.908663  2502 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:54.882606  2811 tablet_bootstrap.cc:654] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:54.915872  2502 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:54.915035  2839 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:54.925076  2847 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:54.928045  2521 tablet_service.cc:1511] Processing CreateTablet for tablet b47a7b94ee76444db9c73a90f520af7c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:54.922348  2686 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:54.915666  2805 raft_consensus.cc:359] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:54.944240  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47a7b94ee76444db9c73a90f520af7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:54.937425  2816 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:54.952035  2749 tablet_service.cc:1511] Processing CreateTablet for tablet b47a7b94ee76444db9c73a90f520af7c (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 14:50:54.946008  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:54.941219  2847 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:54.938795  2805 raft_consensus.cc:385] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:54.969769  2845 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:54.970029  2847 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:54.970074  2805 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:54.955288  2597 tablet_service.cc:1511] Processing CreateTablet for tablet f231033e78bf42ad8d6dbdf1f54d21be (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:54.976568  2667 tablet_service.cc:1511] Processing CreateTablet for tablet f231033e78bf42ad8d6dbdf1f54d21be (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:54.977310  2807 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:54.984818  2805 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:54.994293  2741 tablet_service.cc:1511] Processing CreateTablet for tablet f231033e78bf42ad8d6dbdf1f54d21be (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 14:50:55.001056  2597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f231033e78bf42ad8d6dbdf1f54d21be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.007133  2749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47a7b94ee76444db9c73a90f520af7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.012921  2806 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.013615  2806 ts_tablet_manager.cc:1403] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.298s	user 0.018s	sys 0.021s
I20260709 14:50:54.962173  2868 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 14:50:54.989495  2807 ts_tablet_manager.cc:1403] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.265s	user 0.017s	sys 0.012s
I20260709 14:50:55.036772  2520 tablet_service.cc:1511] Processing CreateTablet for tablet 963a4155717b4274b0121341056b7cb8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:55.038563  2520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963a4155717b4274b0121341056b7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.022881  2667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f231033e78bf42ad8d6dbdf1f54d21be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.061365  2741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f231033e78bf42ad8d6dbdf1f54d21be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.074816  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:55.076741  2599 tablet_service.cc:1511] Processing CreateTablet for tablet 963a4155717b4274b0121341056b7cb8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:55.078110  2672 tablet_service.cc:1511] Processing CreateTablet for tablet 963a4155717b4274b0121341056b7cb8 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 14:50:55.078727  2599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963a4155717b4274b0121341056b7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.079613  2672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963a4155717b4274b0121341056b7cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.079372  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:55.006500  2612 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:55.081317  2860 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:50:55.089059  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:55.057981  2807 raft_consensus.cc:359] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.093254  2807 raft_consensus.cc:385] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.093688  2807 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.102514  2847 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.107851  2807 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.113653  2742 tablet_service.cc:1511] Processing CreateTablet for tablet dd47a5b34fb04ee887e263844b637438 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:55.115561  2742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd47a5b34fb04ee887e263844b637438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.128578  2845 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:50:55.129522  2503 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73799.8us
I20260709 14:50:55.143689  2668 tablet_service.cc:1511] Processing CreateTablet for tablet c051e44d64d14770b36c5012be981196 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:55.145323  2668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c051e44d64d14770b36c5012be981196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.145534  2845 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.147017  2868 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:55.148339  2806 raft_consensus.cc:359] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.149116  2806 raft_consensus.cc:385] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.149426  2806 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.150152  2847 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:55.132686  2805 ts_tablet_manager.cc:1434] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.298s	user 0.009s	sys 0.012s
I20260709 14:50:55.134563  2807 ts_tablet_manager.cc:1434] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.002s
I20260709 14:50:55.139009  2602 tablet_service.cc:1511] Processing CreateTablet for tablet dd47a5b34fb04ee887e263844b637438 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:55.151953  2602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd47a5b34fb04ee887e263844b637438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.152801  2806 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.155821  2806 ts_tablet_manager.cc:1434] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.142s	user 0.008s	sys 0.001s
I20260709 14:50:55.156859  2806 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:55.159286  2816 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:55.139904  2521 tablet_service.cc:1511] Processing CreateTablet for tablet dd47a5b34fb04ee887e263844b637438 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 14:50:55.149434  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:55.160326  2760 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.160943  2521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd47a5b34fb04ee887e263844b637438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.163914  2807 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:55.167447  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:50:55.169070  2503 leader_election.cc:343] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.140774  2811 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:55.176851  2811 ts_tablet_manager.cc:1403] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.548s	user 0.066s	sys 0.077s
I20260709 14:50:55.190354  2744 tablet_service.cc:1511] Processing CreateTablet for tablet c051e44d64d14770b36c5012be981196 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:55.192018  2744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c051e44d64d14770b36c5012be981196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.206959  2805 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:55.235100  2847 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.236027  2847 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.240141  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.240938  2759 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:55.241328  2518 tablet_service.cc:1511] Processing CreateTablet for tablet c051e44d64d14770b36c5012be981196 (DEFAULT_TABLE table=test-workload [id=b6196ef5fd834ecd907c5a75e1b9c496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 14:50:55.242343  2502 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:55.243191  2860 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.243340  2868 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:55.243764  2868 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:55.243456  2811 raft_consensus.cc:359] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.243734  2860 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.244387  2811 raft_consensus.cc:385] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.244813  2811 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.245818  2811 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.246685  2760 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:55.247890  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.290362  2759 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:55.292034  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:55.293301  2861 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:55.293777  2861 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:55.294144  2861 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.249027  2806 tablet_bootstrap.cc:654] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.249748  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:55.310743  2536 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:55.250667  2807 tablet_bootstrap.cc:654] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.250502  2860 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.244246  2868 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.251441  2518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c051e44d64d14770b36c5012be981196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:50:55.272149  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:55.315479  2686 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:55.317381  2502 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:55.272282  2847 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:55.287803  2805 tablet_bootstrap.cc:654] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.319177  2847 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.336743  2847 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:55.338981  2847 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.305531  2431 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:55.371517  2861 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.396792  2811 ts_tablet_manager.cc:1434] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.209s	user 0.005s	sys 0.005s
I20260709 14:50:55.413206  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:55.414351  2760 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.419529  2868 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.430212  2811 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:55.434206  2806 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.434734  2435 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:55.437000  2806 ts_tablet_manager.cc:1403] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.280s	user 0.018s	sys 0.021s
I20260709 14:50:55.436302  2805 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:55.439296  2805 ts_tablet_manager.cc:1403] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.233s	user 0.050s	sys 0.036s
I20260709 14:50:55.443162  2805 raft_consensus.cc:359] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.447047  2805 raft_consensus.cc:385] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.447525  2805 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.448628  2805 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.451618  2805 ts_tablet_manager.cc:1434] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 14:50:55.452977  2805 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:55.460697  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:55.461604  2805 tablet_bootstrap.cc:654] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.461642  2536 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.467519  2811 tablet_bootstrap.cc:654] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.445429  2868 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:55.445825  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:55.446352  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:55.445814  2806 raft_consensus.cc:359] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.471268  2806 raft_consensus.cc:385] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.471994  2686 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.473002  2806 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.470458  2759 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.474185  2806 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.469568  2861 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.487046  2536 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:55.488870  2807 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:55.489118  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:55.489512  2807 ts_tablet_manager.cc:1403] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.326s	user 0.013s	sys 0.038s
I20260709 14:50:55.490581  2861 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.491282  2861 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:55.492481  2861 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.492978  2807 raft_consensus.cc:359] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.493798  2807 raft_consensus.cc:385] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.494107  2807 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.495051  2807 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.496373  2806 ts_tablet_manager.cc:1434] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 14:50:55.497614  2806 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:55.498109  2807 ts_tablet_manager.cc:1434] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:50:55.499277  2807 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:55.506397  2760 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:55.524545  2759 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:55.515028  2686 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:55.526229  2502 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:55.526854  2807 tablet_bootstrap.cc:654] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.529111  2868 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.529732  2868 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:55.530889  2868 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.549389  2431 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:55.551505  2811 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:55.552266  2811 ts_tablet_manager.cc:1403] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.130s	user 0.026s	sys 0.009s
I20260709 14:50:55.557145  2805 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:55.557825  2805 ts_tablet_manager.cc:1403] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.105s	user 0.020s	sys 0.030s
I20260709 14:50:55.560243  2811 raft_consensus.cc:359] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.561317  2811 raft_consensus.cc:385] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.561206  2805 raft_consensus.cc:359] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.554596  2807 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:55.562143  2805 raft_consensus.cc:385] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.562486  2805 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.562573  2807 ts_tablet_manager.cc:1403] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.007s
I20260709 14:50:55.563985  2805 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.552553  2806 tablet_bootstrap.cc:654] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.566258  2861 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.567063  2861 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.568674  2845 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.569280  2845 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.571007  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:55.572567  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:55.573181  2861 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.573101  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:55.574090  2536 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:55.561823  2811 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.574383  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.575305  2759 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:55.575701  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:55.577425  2845 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.575850  2811 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:50:55.580295  2723 leader_election.cc:343] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.580431  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:55.577093  2871 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:55.581614  2871 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:55.581967  2871 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.584344  2861 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.587270  2845 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:55.581002  2435 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:50:55.593202  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:55.577028  2807 raft_consensus.cc:359] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.595966  2871 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.604538  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:50:55.607014  2723 leader_election.cc:343] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.607568  2811 ts_tablet_manager.cc:1434] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 14:50:55.609170  2811 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:55.613025  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:55.613895  2536 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.618893  2871 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.596493  2807 raft_consensus.cc:385] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.628647  2806 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.629319  2806 ts_tablet_manager.cc:1403] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.132s	user 0.025s	sys 0.017s
I20260709 14:50:55.631522  2536 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:55.632805  2806 raft_consensus.cc:359] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.633129  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:55.634023  2806 raft_consensus.cc:385] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.634817  2806 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.634996  2871 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.635545  2871 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:55.635689  2805 ts_tablet_manager.cc:1434] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 14:50:55.636492  2871 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.624670  2807 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.638077  2807 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.635735  2806 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.640892  2807 ts_tablet_manager.cc:1434] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.077s	user 0.004s	sys 0.004s
I20260709 14:50:55.637049  2805 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:55.641804  2806 ts_tablet_manager.cc:1434] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 14:50:55.642156  2807 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:55.643049  2806 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:55.646740  2816 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.647475  2816 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.649438  2816 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:55.650401  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.651639  2806 tablet_bootstrap.cc:654] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:55.652252  2651 leader_election.cc:343] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.631554  2861 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:55.660391  2811 tablet_bootstrap.cc:654] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.667577  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:50:55.670790  2650 leader_election.cc:343] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.671459  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:55.671936  2806 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.672528  2806 ts_tablet_manager.cc:1403] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.008s
I20260709 14:50:55.673152  2816 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:55.675935  2806 raft_consensus.cc:359] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.676744  2806 raft_consensus.cc:385] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.676870  2868 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.677201  2806 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.677407  2868 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.678048  2806 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.680339  2806 ts_tablet_manager.cc:1434] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 14:50:55.681196  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:55.682170  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.682780  2759 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:55.683883  2868 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:50:55.683826  2503 leader_election.cc:343] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.689944  2806 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:55.701198  2502 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:55.702024  2807 tablet_bootstrap.cc:654] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.702514  2868 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:55.704452  2868 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:55.706737  2868 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.721503  2872 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:55.722018  2868 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.730217  2805 tablet_bootstrap.cc:654] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.733388  2847 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:55.741762  2868 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:55.741676  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:55.742659  2760 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.743239  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:55.744412  2811 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:55.745018  2811 ts_tablet_manager.cc:1403] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.001s
W20260709 14:50:55.745426  2503 leader_election.cc:343] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.749084  2806 tablet_bootstrap.cc:654] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.763659  2760 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:55.766744  2502 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:55.768744  2868 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.769359  2868 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:55.769398  2435 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:55.770344  2868 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.792770  2811 raft_consensus.cc:359] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.793996  2811 raft_consensus.cc:385] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.807421  2806 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.808261  2806 ts_tablet_manager.cc:1403] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.123s	user 0.011s	sys 0.018s
I20260709 14:50:55.811569  2805 tablet_bootstrap.cc:492] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:55.811662  2806 raft_consensus.cc:359] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.812585  2806 raft_consensus.cc:385] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.812835  2806 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.794384  2811 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.813578  2806 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.814440  2811 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.812242  2805 ts_tablet_manager.cc:1403] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.175s	user 0.018s	sys 0.004s
I20260709 14:50:55.817576  2811 ts_tablet_manager.cc:1434] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.001s
I20260709 14:50:55.821110  2806 ts_tablet_manager.cc:1434] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:50:55.822603  2806 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:55.823746  2805 raft_consensus.cc:359] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.824582  2805 raft_consensus.cc:385] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.824903  2805 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.827057  2878 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:55.830256  2806 tablet_bootstrap.cc:654] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.836359  2805 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.837802  2807 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:55.838441  2807 ts_tablet_manager.cc:1403] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.197s	user 0.009s	sys 0.008s
I20260709 14:50:55.842279  2807 raft_consensus.cc:359] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.843185  2807 raft_consensus.cc:385] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.843514  2807 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.844162  2805 ts_tablet_manager.cc:1434] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 14:50:55.845918  2805 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:55.844710  2811 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:55.847576  2433 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:55.860944  2868 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.861655  2868 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.863708  2807 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:55.866565  2807 ts_tablet_manager.cc:1434] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.003s
I20260709 14:50:55.867288  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:55.876012  2807 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
W20260709 14:50:55.876493  2502 leader_election.cc:343] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.867894  2868 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:55.876247  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:55.879132  2686 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:55.880879  2500 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:55.882733  2868 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:55.883203  2868 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:55.883591  2868 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.880084  2811 tablet_bootstrap.cc:654] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.897015  2805 tablet_bootstrap.cc:654] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.893355  2868 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.905604  2847 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:55.906252  2847 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.909329  2807 tablet_bootstrap.cc:654] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.913084  2847 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:55.913887  2806 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:55.910781  2861 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:55.910589  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:55.918658  2806 ts_tablet_manager.cc:1403] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.049s
I20260709 14:50:55.926241  2806 raft_consensus.cc:359] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.926391  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:55.927994  2806 raft_consensus.cc:385] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.930177  2806 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.935168  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:55.935982  2685 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:55.937417  2806 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:55.941511  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:55.942312  2868 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:55.951639  2500 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:55.952610  2806 ts_tablet_manager.cc:1434] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 14:50:55.953385  2883 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:55.953814  2806 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
W20260709 14:50:55.953922  2502 leader_election.cc:343] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:55.960680  2806 tablet_bootstrap.cc:654] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:55.965565  2685 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:55.968773  2500 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:55.970415  2868 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:55.973855  2868 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:55.975035  2868 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:55.984242  2811 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:55.985309  2811 ts_tablet_manager.cc:1403] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.167s	user 0.045s	sys 0.014s
I20260709 14:50:55.988348  2807 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:55.988971  2807 ts_tablet_manager.cc:1403] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.113s	user 0.007s	sys 0.014s
I20260709 14:50:55.991274  2811 raft_consensus.cc:359] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:55.992350  2811 raft_consensus.cc:385] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:55.992817  2811 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:55.993891  2811 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.001364  2805 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.001982  2805 ts_tablet_manager.cc:1403] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.156s	user 0.030s	sys 0.003s
I20260709 14:50:56.004628  2807 raft_consensus.cc:359] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.004957  2805 raft_consensus.cc:359] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.005759  2807 raft_consensus.cc:385] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.005849  2805 raft_consensus.cc:385] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.006234  2805 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.007122  2805 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.007217  2861 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.009871  2805 ts_tablet_manager.cc:1434] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 14:50:56.011070  2861 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.011669  2861 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.014966  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.015470  2861 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:56.015847  2759 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:56.011068  2805 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.015867  2807 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.017951  2807 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.019485  2811 ts_tablet_manager.cc:1434] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 14:50:56.020334  2807 ts_tablet_manager.cc:1434] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 14:50:56.025090  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.025802  2861 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.026346  2861 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.031973  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.032231  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.036046  2811 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.035912  2847 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.037323  2431 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:50:56.039222  2576 leader_election.cc:343] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.041589  2917 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.044628  2807 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.045854  2805 tablet_bootstrap.cc:654] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.058300  2806 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.058934  2806 ts_tablet_manager.cc:1403] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.008s
I20260709 14:50:56.061348  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.064749  2806 raft_consensus.cc:359] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.065553  2806 raft_consensus.cc:385] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.065830  2806 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.057168  2861 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:56.066540  2806 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.073199  2806 ts_tablet_manager.cc:1434] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
W20260709 14:50:56.073972  2575 leader_election.cc:343] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.074563  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.075874  2917 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.076689  2917 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.077286  2536 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.078606  2806 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.082659  2917 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.083413  2917 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.086496  2807 tablet_bootstrap.cc:654] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.087553  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.088915  2917 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.089278  2760 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:56.091881  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:56.094790  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.095723  2685 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:56.097259  2861 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:56.097802  2861 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:56.098197  2861 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.099845  2882 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.099586  2847 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.100448  2882 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.104365  2811 tablet_bootstrap.cc:654] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.105125  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.103488  2882 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:56.120620  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.121862  2537 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:50:56.123533  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.124696  2882 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.128139  2806 tablet_bootstrap.cc:654] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.142091  2872 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.144428  2861 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.151933  2805 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.156809  2805 ts_tablet_manager.cc:1403] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.146s	user 0.020s	sys 0.014s
I20260709 14:50:56.170007  2807 tablet_bootstrap.cc:492] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.170982  2807 ts_tablet_manager.cc:1403] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.006s
I20260709 14:50:56.180294  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:56.181099  2685 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.182260  2811 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.182963  2811 ts_tablet_manager.cc:1403] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.147s	user 0.022s	sys 0.003s
I20260709 14:50:56.186544  2811 raft_consensus.cc:359] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.187453  2811 raft_consensus.cc:385] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.187649  2811 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.190963  2807 raft_consensus.cc:359] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.193073  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:56.200373  2759 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.202068  2806 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.202490  2839 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.202665  2806 ts_tablet_manager.cc:1403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.004s
I20260709 14:50:56.194767  2861 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.188709  2811 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.207486  2882 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.205579  2805 raft_consensus.cc:359] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.209517  2805 raft_consensus.cc:385] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.209827  2805 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.212054  2805 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.213250  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:56.205862  2839 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.214447  2805 ts_tablet_manager.cc:1434] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.002s
I20260709 14:50:56.215701  2872 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:56.215631  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:56.217212  2861 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:56.217075  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.218773  2861 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:56.212954  2807 raft_consensus.cc:385] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 14:50:56.218994  2503 leader_election.cc:343] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.222808  2861 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.219626  2807 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.220445  2811 ts_tablet_manager.cc:1434] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 14:50:56.225852  2805 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.228961  2811 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.228441  2807 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.227802  2806 raft_consensus.cc:359] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.231021  2806 raft_consensus.cc:385] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.231452  2806 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.232756  2807 ts_tablet_manager.cc:1434] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.061s	user 0.010s	sys 0.003s
I20260709 14:50:56.236481  2806 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.225100  2839 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:56.245904  2806 ts_tablet_manager.cc:1434] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 14:50:56.247339  2806 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.254719  2878 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.260278  2807 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.260058  2921 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.261339  2878 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.262239  2921 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.264730  2921 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.265714  2806 tablet_bootstrap.cc:654] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.265923  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.266330  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.269539  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.270560  2682 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:56.267771  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.271713  2759 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:56.256597  2685 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
W20260709 14:50:56.275401  2502 leader_election.cc:343] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:56.276232  2500 leader_election.cc:343] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.276778  2500 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:56.278559  2845 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.279685  2845 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.278573  2868 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.282008  2845 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:56.284298  2805 tablet_bootstrap.cc:654] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.275440  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.289234  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.290306  2682 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:56.273416  2878 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.273147  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:56.300071  2811 tablet_bootstrap.cc:654] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:56.301705  2503 leader_election.cc:343] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.302268  2503 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:56.296461  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.303282  2807 tablet_bootstrap.cc:654] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.311962  2839 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.310205  2921 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:56.315333  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.316155  2921 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:56.316560  2921 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.317206  2845 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.327080  2435 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:56.335139  2847 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.343694  2806 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.348704  2806 ts_tablet_manager.cc:1403] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.023s
I20260709 14:50:56.353283  2811 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.354140  2811 ts_tablet_manager.cc:1403] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.128s	user 0.014s	sys 0.002s
I20260709 14:50:56.339439  2847 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.357366  2847 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.339366  2878 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 14:50:56.358664  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.360519  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:50:56.360978  2500 leader_election.cc:343] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:56.362993  2502 leader_election.cc:343] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.363525  2502 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:56.364841  2878 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.365706  2806 raft_consensus.cc:359] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.366761  2806 raft_consensus.cc:385] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.367213  2806 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.365937  2921 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.369040  2806 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.376449  2806 ts_tablet_manager.cc:1434] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 14:50:56.376319  2878 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.378711  2807 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.388623  2921 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.379927  2806 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.406332  2807 ts_tablet_manager.cc:1403] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.146s	user 0.017s	sys 0.014s
I20260709 14:50:56.379531  2845 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.383849  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:56.408298  2758 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.383800  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:56.411726  2685 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.388326  2811 raft_consensus.cc:359] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.413467  2811 raft_consensus.cc:385] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.413781  2811 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.414896  2806 tablet_bootstrap.cc:654] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.414739  2811 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.417897  2883 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.421751  2807 raft_consensus.cc:359] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.422596  2807 raft_consensus.cc:385] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.426074  2811 ts_tablet_manager.cc:1434] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.071s	user 0.013s	sys 0.000s
I20260709 14:50:56.450577  2811 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.381467  2805 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.458482  2811 tablet_bootstrap.cc:654] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.458660  2805 ts_tablet_manager.cc:1403] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.238s	user 0.015s	sys 0.015s
I20260709 14:50:56.459175  2807 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.461690  2878 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.473284  2685 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:56.475101  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:56.476089  2921 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:56.475564  2807 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.476662  2921 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:56.477614  2921 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.478312  2807 ts_tablet_manager.cc:1434] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 14:50:56.479647  2807 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.489439  2811 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.490682  2811 ts_tablet_manager.cc:1403] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.001s
I20260709 14:50:56.494771  2811 raft_consensus.cc:359] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.495929  2811 raft_consensus.cc:385] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.496239  2811 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.462728  2805 raft_consensus.cc:359] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.442013  2806 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.470973  2758 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:56.498142  2806 ts_tablet_manager.cc:1403] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.118s	user 0.025s	sys 0.009s
I20260709 14:50:56.500854  2816 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.501611  2805 raft_consensus.cc:385] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.502874  2805 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.504225  2807 tablet_bootstrap.cc:654] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.502117  2806 raft_consensus.cc:359] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.506326  2806 raft_consensus.cc:385] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.506726  2806 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.501459  2816 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.507583  2806 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.503840  2805 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.510380  2805 ts_tablet_manager.cc:1434] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 14:50:56.510691  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.511554  2805 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
W20260709 14:50:56.513770  2650 leader_election.cc:343] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.517007  2433 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:56.497016  2811 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.518992  2816 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.524377  2806 ts_tablet_manager.cc:1434] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 14:50:56.529551  2883 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.520403  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.526221  2811 ts_tablet_manager.cc:1434] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 14:50:56.534001  2612 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:56.532953  2883 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.534935  2811 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.535616  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.536562  2883 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:56.536911  2816 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:56.537299  2816 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:56.537664  2816 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.531860  2806 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.540453  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:56.542126  2503 leader_election.cc:343] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.548101  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.548956  2682 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:56.552143  2500 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:56.553226  2883 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:56.553676  2883 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:56.554072  2883 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.554128  2816 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.556871  2816 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.557217  2805 tablet_bootstrap.cc:654] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.560197  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:56.562134  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:56.562929  2612 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.565510  2806 tablet_bootstrap.cc:654] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.566434  2811 tablet_bootstrap.cc:654] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.581734  2883 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:50:56.574954  2650 leader_election.cc:343] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.587608  2807 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.588409  2807 ts_tablet_manager.cc:1403] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.109s	user 0.020s	sys 0.000s
I20260709 14:50:56.589221  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:56.590969  2883 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:56.592150  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:56.593184  2682 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.594509  2807 raft_consensus.cc:359] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:50:56.595036  2503 leader_election.cc:343] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.595392  2807 raft_consensus.cc:385] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.596298  2805 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.596843  2805 ts_tablet_manager.cc:1403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.003s
I20260709 14:50:56.595870  2807 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.604566  2807 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.606210  2805 raft_consensus.cc:359] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.607043  2805 raft_consensus.cc:385] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.607156  2807 ts_tablet_manager.cc:1434] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 14:50:56.607405  2805 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.608373  2807 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.608331  2805 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.614754  2682 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:56.616046  2807 tablet_bootstrap.cc:654] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.616338  2500 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:56.617790  2883 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:56.619994  2805 ts_tablet_manager.cc:1434] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 14:50:56.604429  2612 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:56.625759  2806 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.626497  2806 ts_tablet_manager.cc:1403] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.008s
I20260709 14:50:56.627688  2805 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.628573  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.619431  2883 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:56.631758  2883 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.632877  2816 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:56.635232  2816 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:56.636309  2923 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:56.636175  2816 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.637326  2805 tablet_bootstrap.cc:654] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:56.634608  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:56.679703  2923 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.680387  2923 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.687175  2806 raft_consensus.cc:359] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.687525  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:56.687899  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.685460  2923 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.688351  2806 raft_consensus.cc:385] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.690444  2806 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
W20260709 14:50:56.690038  2575 leader_election.cc:343] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:56.692204  2577 leader_election.cc:343] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.692781  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:56.693714  2923 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.695926  2806 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.697405  2923 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:56.701524  2807 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.702225  2807 ts_tablet_manager.cc:1403] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.094s	user 0.048s	sys 0.011s
I20260709 14:50:56.728193  2807 raft_consensus.cc:359] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.729048  2807 raft_consensus.cc:385] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.729149  2806 ts_tablet_manager.cc:1434] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.001s
I20260709 14:50:56.729380  2807 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.731047  2806 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.737481  2811 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.734258  2434 catalog_manager.cc:5697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:56.739557  2806 tablet_bootstrap.cc:654] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.730289  2807 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.737705  2805 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.754347  2805 ts_tablet_manager.cc:1403] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.023s
I20260709 14:50:56.755604  2807 ts_tablet_manager.cc:1434] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 14:50:56.756664  2807 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.757804  2805 raft_consensus.cc:359] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.758994  2805 raft_consensus.cc:385] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.759393  2805 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.760430  2805 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.763561  2878 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.764039  2807 tablet_bootstrap.cc:654] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.764143  2878 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.766562  2805 ts_tablet_manager.cc:1434] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:50:56.768368  2805 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.768229  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.766683  2878 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:50:56.770567  2502 leader_election.cc:343] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.771906  2811 ts_tablet_manager.cc:1403] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.237s	user 0.027s	sys 0.041s
I20260709 14:50:56.773295  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.789700  2805 tablet_bootstrap.cc:654] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
W20260709 14:50:56.776309  2503 leader_election.cc:343] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.800220  2503 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:56.775611  2811 raft_consensus.cc:359] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.801481  2811 raft_consensus.cc:385] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.801852  2811 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.802719  2811 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.805141  2811 ts_tablet_manager.cc:1434] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 14:50:56.805673  2878 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.816607  2811 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.815835  2434 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:56.817157  2805 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.817785  2805 ts_tablet_manager.cc:1403] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.005s
I20260709 14:50:56.820932  2805 raft_consensus.cc:359] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.810014  2807 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.823338  2805 raft_consensus.cc:385] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.823654  2807 ts_tablet_manager.cc:1403] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.067s	user 0.030s	sys 0.012s
I20260709 14:50:56.834740  2845 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.835441  2845 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.832563  2811 tablet_bootstrap.cc:654] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.836261  2806 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.836822  2806 ts_tablet_manager.cc:1403] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.106s	user 0.016s	sys 0.006s
I20260709 14:50:56.837906  2845 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.839184  2805 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.839306  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.840086  2805 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:50:56.840893  2726 leader_election.cc:343] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.842732  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.843353  2805 ts_tablet_manager.cc:1434] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 14:50:56.845126  2805 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.842571  2806 raft_consensus.cc:359] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.846233  2806 raft_consensus.cc:385] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.846508  2806 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.847399  2806 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:50:56.849123  2724 leader_election.cc:343] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.849704  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:56.851054  2845 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.851279  2806 ts_tablet_manager.cc:1434] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 14:50:56.852700  2806 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.854776  2805 tablet_bootstrap.cc:654] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.836300  2807 raft_consensus.cc:359] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.860152  2807 raft_consensus.cc:385] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.860548  2807 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.861814  2816 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.861373  2807 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.862468  2816 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.863721  2811 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.864329  2811 ts_tablet_manager.cc:1403] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.005s
I20260709 14:50:56.864982  2816 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.867805  2811 raft_consensus.cc:359] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.868165  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.868791  2811 raft_consensus.cc:385] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.869191  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:56.869436  2811 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.872810  2811 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:56.873410  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:56.874795  2945 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:56.875239  2945 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:56.875602  2945 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.878393  2811 ts_tablet_manager.cc:1434] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 14:50:56.879771  2811 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.883216  2807 ts_tablet_manager.cc:1434] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 14:50:56.886974  2807 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:56.870703  2806 tablet_bootstrap.cc:654] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.870107  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:56.898608  2537 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:56.901136  2807 tablet_bootstrap.cc:654] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.885587  2805 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:56.905133  2811 tablet_bootstrap.cc:654] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.905788  2805 ts_tablet_manager.cc:1403] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.023s
I20260709 14:50:56.909229  2805 raft_consensus.cc:359] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.910336  2805 raft_consensus.cc:385] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.910725  2805 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.911731  2805 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.914100  2806 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:56.915000  2806 ts_tablet_manager.cc:1403] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.004s
I20260709 14:50:56.905175  2945 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.921450  2945 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:56.922732  2806 raft_consensus.cc:359] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.923674  2806 raft_consensus.cc:385] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.923960  2806 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.924875  2807 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:56.924700  2883 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.925522  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:56.925402  2883 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.927000  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:56.931424  2537 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.929227  2811 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:56.933909  2811 ts_tablet_manager.cc:1403] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.000s
I20260709 14:50:56.934310  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:50:56.936546  2500 leader_election.cc:343] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.937215  2811 raft_consensus.cc:359] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.938027  2811 raft_consensus.cc:385] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.938316  2811 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.939095  2811 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:56.939814  2883 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:56.925523  2806 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.931501  2611 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:56.941203  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:56.942504  2537 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:56.930886  2807 ts_tablet_manager.cc:1403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.005s
I20260709 14:50:56.951845  2811 ts_tablet_manager.cc:1434] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 14:50:56.952517  2806 ts_tablet_manager.cc:1434] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.002s
I20260709 14:50:56.944839  2805 ts_tablet_manager.cc:1434] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 14:50:56.958247  2811 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:56.958750  2805 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:56.967056  2806 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:56.950395  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
W20260709 14:50:56.952019  2502 leader_election.cc:343] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:56.972949  2502 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:56.973629  2811 tablet_bootstrap.cc:654] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:56.981719  2878 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:56.986953  2807 raft_consensus.cc:359] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.988354  2807 raft_consensus.cc:385] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:56.988693  2807 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:56.989809  2845 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:56.991061  2845 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.991644  2807 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:56.946102  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:56.993454  2945 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:56.994175  2945 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:56.995441  2945 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:56.996723  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:56.997612  2612 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.006379  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:57.007706  2872 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.008299  2872 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.008710  2872 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.009450  2806 tablet_bootstrap.cc:654] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.014256  2805 tablet_bootstrap.cc:654] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.015877  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.013972  2807 ts_tablet_manager.cc:1434] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.004s
I20260709 14:50:57.016764  2537 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.018045  2807 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:57.020109  2861 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.004386  2845 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:57.023347  2806 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:57.025605  2806 ts_tablet_manager.cc:1403] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.000s
I20260709 14:50:57.029264  2806 raft_consensus.cc:359] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.030135  2806 raft_consensus.cc:385] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.030471  2806 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
W20260709 14:50:57.007611  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61649.6us
I20260709 14:50:57.037372  2806 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.039819  2806 ts_tablet_manager.cc:1434] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 14:50:57.041026  2806 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:57.046239  2816 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.047010  2816 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.050218  2816 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:57.042606  2807 tablet_bootstrap.cc:654] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.052297  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:57.054574  2652 leader_election.cc:343] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.057252  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.058214  2537 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:57.062615  2806 tablet_bootstrap.cc:654] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.063421  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:57.063563  2923 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.071599  2816 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.072116  2882 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.072795  2882 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.072227  2816 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.075270  2816 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.076373  2882 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:57.079463  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:50:57.082998  2651 leader_election.cc:343] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.084317  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.084208  2872 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:50:57.086541  2652 leader_election.cc:343] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.087174  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:57.089766  2806 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:57.090710  2811 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:57.090194  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:57.091519  2537 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.091509  2811 ts_tablet_manager.cc:1403] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.134s	user 0.041s	sys 0.032s
I20260709 14:50:57.090380  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:57.094182  2612 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.094990  2861 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.095559  2861 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.089906  2882 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.096745  2811 raft_consensus.cc:359] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.097484  2811 raft_consensus.cc:385] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.097785  2811 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.098505  2811 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.098805  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:50:57.100736  2575 leader_election.cc:343] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.101101  2811 ts_tablet_manager.cc:1434] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:50:57.102386  2811 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:57.104735  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.106146  2537 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:50:57.106648  2577 leader_election.cc:343] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.107188  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.107561  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:57.108167  2923 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.109604  2811 tablet_bootstrap.cc:654] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.109251  2845 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.115414  2845 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:57.116778  2845 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.093255  2872 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:57.121433  2883 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.125242  2861 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:57.128463  2806 ts_tablet_manager.cc:1403] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.005s
I20260709 14:50:57.133335  2537 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:57.143592  2503 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:57.148392  2805 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:57.149024  2805 ts_tablet_manager.cc:1403] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.191s	user 0.015s	sys 0.000s
I20260709 14:50:57.154392  2805 raft_consensus.cc:359] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.155267  2805 raft_consensus.cc:385] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.155391  2970 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.156862  2806 raft_consensus.cc:359] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.157739  2806 raft_consensus.cc:385] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.158015  2806 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.158885  2806 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.156939  2970 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.161839  2970 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.162928  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.163766  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:50:57.164610  2723 leader_election.cc:343] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.164723  2536 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.156432  2805 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.155211  2430 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:57.166903  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.168768  2970 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.138190  2816 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.174258  2806 ts_tablet_manager.cc:1434] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 14:50:57.177995  2878 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.178779  2878 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.179543  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:57.169206  2970 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.180297  2970 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.180758  2882 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:57.181099  2878 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.182363  2882 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.182909  2882 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.184834  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.185173  2882 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.186538  2811 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:57.187212  2811 ts_tablet_manager.cc:1403] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.085s	user 0.030s	sys 0.036s
W20260709 14:50:57.187358  2652 leader_election.cc:343] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:57.188921  2500 leader_election.cc:343] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.186472  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:57.192015  2535 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.193291  2806 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:57.166940  2805 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.192521  2816 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:57.194135  2845 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.195986  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.195725  2845 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.196746  2534 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:57.200587  2805 ts_tablet_manager.cc:1434] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.051s	user 0.011s	sys 0.000s
I20260709 14:50:57.202406  2805 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:57.190301  2811 raft_consensus.cc:359] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.203724  2811 raft_consensus.cc:385] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.204113  2811 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.204984  2811 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.209542  2861 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.211095  2861 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.212858  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.213514  2861 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.213706  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:57.214687  2972 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.216298  2503 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.216434  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:57.218015  2883 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.218441  2883 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.218811  2883 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.219205  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.219920  2807 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:57.220548  2807 ts_tablet_manager.cc:1403] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.203s	user 0.025s	sys 0.006s
W20260709 14:50:57.221247  2575 leader_election.cc:343] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.224398  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.227260  2972 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.232920  2972 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.233768  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.194226  2612 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:50:57.239369  2724 leader_election.cc:343] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.241132  2972 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.241755  2972 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.244020  2972 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.244987  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.245476  2816 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.245888  2811 ts_tablet_manager.cc:1434] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 14:50:57.245919  2816 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.246976  2811 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:57.247025  2816 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.245766  2611 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.249512  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:57.252096  2845 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.252802  2845 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.253191  2845 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.253533  2845 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.254361  2811 tablet_bootstrap.cc:654] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.255182  2807 raft_consensus.cc:359] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.256290  2807 raft_consensus.cc:385] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.256709  2807 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.254524  2805 tablet_bootstrap.cc:654] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.261655  2807 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.262423  2816 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.270776  2807 ts_tablet_manager.cc:1434] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 14:50:57.272259  2807 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:57.273438  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:57.274204  2534 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.290880  2535 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:57.293872  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:57.295346  2882 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.295912  2882 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:57.297168  2805 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:57.298624  2805 ts_tablet_manager.cc:1403] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.016s
I20260709 14:50:57.296885  2882 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.274699  2970 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.313596  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.314515  2685 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.315959  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.316244  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.316771  2758 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:57.317855  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:57.318550  2760 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.318971  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.284655  2816 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.319885  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:57.326481  2806 tablet_bootstrap.cc:654] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.330224  2807 tablet_bootstrap.cc:654] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.339483  2686 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:50:57.341492  2723 leader_election.cc:343] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.342973  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.346000  2978 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.346653  2978 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.347113  2978 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.348899  2970 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.356526  2976 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.358075  2760 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:57.359746  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:57.361472  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:57.362216  2533 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.363336  2811 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:57.363847  2811 ts_tablet_manager.cc:1403] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.117s	user 0.030s	sys 0.032s
I20260709 14:50:57.358683  2534 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:57.360860  2976 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.372079  2430 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:57.380216  2811 raft_consensus.cc:359] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.381960  2811 raft_consensus.cc:385] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.382483  2811 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.384403  2976 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:57.398847  2976 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.406744  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:57.414418  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:57.420213  2972 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.425173  2883 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.427168  2537 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:57.431391  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:57.432999  2923 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.441486  2883 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.444504  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:57.444783  2978 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.450179  2811 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.441640  2805 raft_consensus.cc:359] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.452270  2805 raft_consensus.cc:385] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.452597  2805 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.370196  2845 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.464838  2805 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.438697  2533 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:57.449410  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:57.484540  2811 ts_tablet_manager.cc:1434] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.120s	user 0.009s	sys 0.000s
I20260709 14:50:57.488536  2805 ts_tablet_manager.cc:1434] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.189s	user 0.006s	sys 0.002s
I20260709 14:50:57.453570  2686 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.492672  2805 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:57.485790  2972 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.514850  2972 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.519168  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.519997  2610 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:57.522521  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:57.522773  2816 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.523983  2972 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.524230  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:57.524689  2972 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.525213  2972 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:57.525229  2872 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.525959  2872 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.526366  2872 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.528371  2806 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:57.528970  2806 ts_tablet_manager.cc:1403] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.336s	user 0.053s	sys 0.031s
I20260709 14:50:57.531335  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:57.532094  2685 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.532205  2806 raft_consensus.cc:359] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.532996  2806 raft_consensus.cc:385] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.533255  2806 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.533985  2806 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.536780  2970 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.543690  2806 ts_tablet_manager.cc:1434] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 14:50:57.544418  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.547642  2806 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:57.552856  2883 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.561337  2883 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.561738  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:57.562412  2536 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.526204  2972 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:50:57.551326  2723 leader_election.cc:343] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.554157  2923 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:50:57.550048  2500 leader_election.cc:343] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.587890  2811 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:57.563128  2978 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.599004  2872 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.599828  2882 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.600442  2882 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.601629  2872 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:57.604305  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:57.588706  2816 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:57.602628  2882 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.613806  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.616811  2806 tablet_bootstrap.cc:654] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.607252  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.623876  2923 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.624504  2923 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.626268  2883 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:50:57.611639  2723 leader_election.cc:343] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.619699  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.639271  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.644215  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:57.626794  2923 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:50:57.654027  2500 leader_election.cc:343] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:57.655053  2650 leader_election.cc:343] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.657858  2807 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:57.658501  2807 ts_tablet_manager.cc:1403] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.386s	user 0.013s	sys 0.061s
I20260709 14:50:57.660077  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:57.660807  2610 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.627997  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:57.666640  2433 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:57.672327  2805 tablet_bootstrap.cc:654] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.683583  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:57.698037  2434 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:57.700573  2434 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:57.671417  2612 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:57.691669  2503 leader_election.cc:343] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.715341  2503 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.678628  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.718467  2868 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.672894  2845 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:50:57.679034  2651 leader_election.cc:343] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.724881  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:50:57.726099  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47145.1us
I20260709 14:50:57.726305  2976 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.727231  2535 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:50:57.730659  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:57.733203  2811 tablet_bootstrap.cc:654] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.733242  2923 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.674925  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:57.737977  2807 raft_consensus.cc:359] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.680896  2685 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:57.738739  2807 raft_consensus.cc:385] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.740201  2807 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.741238  2807 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.747890  2807 ts_tablet_manager.cc:1434] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.089s	user 0.011s	sys 0.000s
I20260709 14:50:57.746387  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:57.767109  2536 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:50:57.781966  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:57.778506  2610 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:57.763103  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:57.786305  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:50:57.790810  2723 leader_election.cc:343] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.801222  2991 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.803234  2503 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.804700  2868 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.805313  2868 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:57.806298  2868 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:57.808411  2991 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:57.809305  2972 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:57.809468  2991 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:57.813125  2805 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:57.813858  2805 ts_tablet_manager.cc:1403] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.324s	user 0.010s	sys 0.009s
I20260709 14:50:57.792578  2923 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:57.819947  2805 raft_consensus.cc:359] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.820786  2805 raft_consensus.cc:385] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.821115  2805 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.822402  2806 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:57.823045  2806 ts_tablet_manager.cc:1403] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.276s	user 0.024s	sys 0.016s
I20260709 14:50:57.822005  2805 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.827409  2806 raft_consensus.cc:359] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.828219  2806 raft_consensus.cc:385] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.828493  2806 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.829116  2805 ts_tablet_manager.cc:1434] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 14:50:57.829249  2806 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.830268  2805 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:57.830415  2807 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:57.832782  2806 ts_tablet_manager.cc:1434] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 14:50:57.834007  2806 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:57.826685  2811 tablet_bootstrap.cc:492] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:57.835548  2811 ts_tablet_manager.cc:1403] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.248s	user 0.020s	sys 0.013s
I20260709 14:50:57.852207  2805 tablet_bootstrap.cc:654] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.853850  2807 tablet_bootstrap.cc:654] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.854475  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.875237  2811 raft_consensus.cc:359] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.876363  2811 raft_consensus.cc:385] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.876739  2811 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.878144  2806 tablet_bootstrap.cc:654] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.883343  2972 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.884315  2972 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:57.885646  2972 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.898327  2811 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:50:57.899822  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45480.6us
I20260709 14:50:57.903074  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:57.906100  2811 ts_tablet_manager.cc:1434] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.070s	user 0.012s	sys 0.001s
I20260709 14:50:57.906720  2978 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:57.902387  2430 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:57.914320  2811 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:57.916464  2978 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:57.926071  2811 tablet_bootstrap.cc:654] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.918090  2978 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.929630  2805 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:57.930250  2805 ts_tablet_manager.cc:1403] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.100s	user 0.029s	sys 0.027s
I20260709 14:50:57.931140  2976 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.936025  2976 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.942030  2807 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:57.942623  2807 ts_tablet_manager.cc:1403] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.015s
I20260709 14:50:57.952159  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.938735  2976 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:57.953193  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:57.954747  2612 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:50:57.954661  2805 raft_consensus.cc:359] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.952883  2868 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.956049  2868 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.959376  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.960022  2861 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:57.960616  2861 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.961472  2807 raft_consensus.cc:359] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.961812  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:57.962232  2807 raft_consensus.cc:385] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.962474  2868 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:57.963419  2861 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:57.955605  2805 raft_consensus.cc:385] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.964660  2805 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.965533  2805 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:57.965814  2976 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:57.966670  2882 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.967242  2806 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:57.967806  2806 ts_tablet_manager.cc:1403] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.134s	user 0.036s	sys 0.010s
W20260709 14:50:57.969929  2502 leader_election.cc:343] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.971412  2805 ts_tablet_manager.cc:1434] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 14:50:57.971922  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:57.972532  2805 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:57.973477  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:57.974321  2537 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
W20260709 14:50:57.973491  2577 leader_election.cc:343] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:57.976025  2807 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.976603  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:57.978567  2861 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:57.979166  2861 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:57.979586  2861 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:57.976972  2807 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.981717  2806 raft_consensus.cc:359] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.982523  2806 raft_consensus.cc:385] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:57.982800  2806 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:57.983618  2806 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:57.986241  2806 ts_tablet_manager.cc:1434] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 14:50:57.987429  2806 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:57.992359  2807 ts_tablet_manager.cc:1434] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 14:50:57.993609  2807 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:57.994887  2806 tablet_bootstrap.cc:654] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:57.989951  2433 catalog_manager.cc:5697] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:57.997277  2433 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:58.008268  2805 tablet_bootstrap.cc:654] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.011297  2882 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:58.012660  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.014854  2882 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.015937  2811 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:58.016533  2811 ts_tablet_manager.cc:1403] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.010s
W20260709 14:50:58.019183  2503 leader_election.cc:343] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.019811  2503 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.021124  2868 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:58.024299  2945 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.032222  2811 raft_consensus.cc:359] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.033231  2807 tablet_bootstrap.cc:654] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.033450  2811 raft_consensus.cc:385] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.033924  2811 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.038357  2811 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.054265  2806 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:58.055230  2806 ts_tablet_manager.cc:1403] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.001s
I20260709 14:50:58.040534  2882 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 14:50:58.073645  2811 ts_tablet_manager.cc:1434] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.057s	user 0.012s	sys 0.000s
I20260709 14:50:58.075610  2811 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:58.061339  2868 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.084525  2868 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.110848  2806 raft_consensus.cc:359] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.119876  2806 raft_consensus.cc:385] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.120393  2806 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.122473  2806 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.125018  2806 ts_tablet_manager.cc:1434] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 14:50:58.126219  2806 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:58.129105  2868 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.132405  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.138378  2435 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:58.133955  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.137840  2811 tablet_bootstrap.cc:654] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.139183  2807 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:58.137876  2972 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.144381  2807 ts_tablet_manager.cc:1403] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.151s	user 0.015s	sys 0.008s
W20260709 14:50:58.142453  2503 leader_election.cc:343] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.147802  2978 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.140087  2861 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.151660  2861 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:50:58.155471  2502 leader_election.cc:343] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.155936  2806 tablet_bootstrap.cc:654] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.160604  2805 tablet_bootstrap.cc:492] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:58.161201  2805 ts_tablet_manager.cc:1403] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.189s	user 0.023s	sys 0.006s
I20260709 14:50:58.162636  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:50:58.166005  2577 leader_election.cc:343] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.162569  2882 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.168319  2882 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.170989  2502 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.171428  2882 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.172080  2868 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:58.172549  2861 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.173362  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.161967  2978 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.184933  2609 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:58.181828  2861 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.189374  2806 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:58.190368  2861 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:58.190479  2806 ts_tablet_manager.cc:1403] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.012s
I20260709 14:50:58.190501  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.191495  2805 raft_consensus.cc:359] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.191757  2760 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:58.192214  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:58.192838  2536 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.193667  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:58.183352  2978 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.195060  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:58.192231  2805 raft_consensus.cc:385] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.197295  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.196372  2861 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.197965  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:58.198279  2861 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.198675  2861 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:58.198918  2726 leader_election.cc:343] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.196362  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:58.195878  2805 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.204141  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:50:58.201277  2723 leader_election.cc:343] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:58.203579  2576 leader_election.cc:343] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.207772  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:58.208961  2972 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:58.200337  2805 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.213531  2805 ts_tablet_manager.cc:1434] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 14:50:58.214416  2805 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:58.208465  2807 raft_consensus.cc:359] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.215719  2807 raft_consensus.cc:385] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.216024  2807 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.201412  2978 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:58.215164  2882 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.224478  2806 raft_consensus.cc:359] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.229580  2811 tablet_bootstrap.cc:492] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:58.231478  2806 raft_consensus.cc:385] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.231676  2811 ts_tablet_manager.cc:1403] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.010s
I20260709 14:50:58.231873  2806 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.232666  2806 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.237869  2806 ts_tablet_manager.cc:1434] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 14:50:58.239049  2806 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:58.240116  2805 tablet_bootstrap.cc:654] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.241226  2811 raft_consensus.cc:359] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.242074  2811 raft_consensus.cc:385] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.242411  2811 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.245787  2806 tablet_bootstrap.cc:654] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.259482  2811 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.263234  2811 ts_tablet_manager.cc:1434] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 14:50:58.264608  2811 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:58.271008  2807 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.281092  2882 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:58.283772  2811 tablet_bootstrap.cc:654] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.270467  2536 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:58.292991  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:58.298192  2986 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:58.298806  2986 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:58.302515  2807 ts_tablet_manager.cc:1434] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.158s	user 0.003s	sys 0.004s
I20260709 14:50:58.299755  2986 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.305734  2807 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:58.313628  2805 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:58.314323  2805 ts_tablet_manager.cc:1403] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.100s	user 0.029s	sys 0.012s
I20260709 14:50:58.321269  2806 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:58.320668  2805 raft_consensus.cc:359] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.322465  2806 ts_tablet_manager.cc:1403] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.005s
I20260709 14:50:58.322474  2805 raft_consensus.cc:385] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.323117  2805 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.323990  2805 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.328620  2805 ts_tablet_manager.cc:1434] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 14:50:58.330215  2805 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:58.338434  2805 tablet_bootstrap.cc:654] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.338892  2978 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:58.332162  2861 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.350693  2878 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:50:58.368867  2811 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:58.369550  2811 ts_tablet_manager.cc:1403] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.105s	user 0.018s	sys 0.007s
I20260709 14:50:58.373204  2811 raft_consensus.cc:359] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.374464  2811 raft_consensus.cc:385] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.375144  2811 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.376483  2811 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.388538  2682 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.395321  2807 tablet_bootstrap.cc:654] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.396219  2868 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.396942  2806 raft_consensus.cc:359] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.394369  2811 ts_tablet_manager.cc:1434] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 14:50:58.400799  2972 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.401405  2972 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.403816  2972 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:58.404371  2972 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:58.408012  2811 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:58.408764  2806 raft_consensus.cc:385] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.409093  2806 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.409703  2806 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.414781  2806 ts_tablet_manager.cc:1434] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.001s
I20260709 14:50:58.422315  2805 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:58.423009  2805 ts_tablet_manager.cc:1403] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.093s	user 0.021s	sys 0.025s
I20260709 14:50:58.439641  2806 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:58.449745  2433 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:58.455152  2978 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.455405  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:58.463523  2868 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.465080  2868 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.465992  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:58.466676  2760 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.467332  2868 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.460681  2686 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:58.473724  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:58.474531  2681 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:58.477042  2500 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:58.477932  2878 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.478324  2868 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.478442  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.481506  2861 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:58.481739  2868 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.482213  2868 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.478459  2878 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.484488  2811 tablet_bootstrap.cc:654] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.490676  2878 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.492466  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:58.496577  2503 leader_election.cc:343] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.498783  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.499692  2612 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:50:58.500313  2872 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:58.478410  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:58.502539  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:58.499557  2972 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:50:58.503291  2502 leader_election.cc:343] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.504027  2978 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.508193  2978 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.510003  2978 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.510089  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.511039  2759 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:58.503456  2760 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:58.525381  2811 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:58.526333  2811 ts_tablet_manager.cc:1403] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.118s	user 0.011s	sys 0.004s
I20260709 14:50:58.529582  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:58.530731  2861 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:58.531297  2861 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:58.532219  2861 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.562907  2805 raft_consensus.cc:359] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.563728  2805 raft_consensus.cc:385] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.564097  2805 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
W20260709 14:50:58.561703  3023 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 14:50:58.576684  2805 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.589190  2805 ts_tablet_manager.cc:1434] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.166s	user 0.009s	sys 0.001s
I20260709 14:50:58.590854  2805 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:58.600610  2882 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.601365  2882 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.602027  2991 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:58.605194  2811 raft_consensus.cc:359] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.606050  2811 raft_consensus.cc:385] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.606330  2811 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.610407  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.610229  2811 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.608323  2882 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.612109  2807 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:58.612743  2807 ts_tablet_manager.cc:1403] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.309s	user 0.021s	sys 0.015s
W20260709 14:50:58.572206  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68936us
W20260709 14:50:58.612551  2576 leader_election.cc:343] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:58.576812  2503 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 72062.6us
I20260709 14:50:58.621929  2806 tablet_bootstrap.cc:654] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.624780  2811 ts_tablet_manager.cc:1434] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.098s	user 0.013s	sys 0.005s
I20260709 14:50:58.625219  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.629176  2811 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:58.622463  2882 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.637470  2502 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.639480  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.648386  2807 raft_consensus.cc:359] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.649288  2807 raft_consensus.cc:385] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.649598  2807 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.633845  2878 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.652169  2878 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.663021  2878 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.664803  2878 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.665285  2878 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.665648  2878 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.651542  2882 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.674249  2882 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.635159  2978 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.676226  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.677556  2978 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:58.678704  2811 tablet_bootstrap.cc:654] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.681465  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:58.682219  2685 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.689360  2807 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:50:58.692595  2652 leader_election.cc:343] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.687860  2878 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.712349  2433 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:58.715250  2685 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:58.721159  2807 ts_tablet_manager.cc:1434] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.004s
I20260709 14:50:58.727063  2807 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:58.739424  2868 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:58.744516  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:50:58.745021  2577 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:58.747666  2991 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:58.748255  2991 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:50:58.749214  2991 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.749742  2805 tablet_bootstrap.cc:654] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.749652  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:50:58.759442  2503 leader_election.cc:343] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.760983  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:50:58.735324  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98644.5us
I20260709 14:50:58.761786  2612 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.753500  2878 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.774322  3030 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:50:58.777571  2807 tablet_bootstrap.cc:654] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.799388  2806 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:58.800603  2806 ts_tablet_manager.cc:1403] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.361s	user 0.029s	sys 0.021s
I20260709 14:50:58.805807  2861 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.806730  2861 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.809959  2861 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:58.812351  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.813457  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:58.823378  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:58.790680  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:58.830849  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:58.831692  2537 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
W20260709 14:50:58.832124  2500 leader_election.cc:343] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.834028  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:58.834717  2537 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:58.835086  2923 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.835582  2923 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.836117  2923 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.839843  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.840696  2760 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:58.842752  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:58.843506  2759 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.849319  2882 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:58.850183  2882 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.853786  2759 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:58.856508  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:58.857336  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:58.857944  2759 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:58.859541  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:50:58.864109  3030 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.864530  3030 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:50:58.864367  2652 leader_election.cc:343] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.865065  3030 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.854809  2868 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.871111  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.872620  2976 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.873070  2976 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.873453  2976 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.874106  3030 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.858597  2612 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:50:58.881108  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:58.882501  2502 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:58.885387  2811 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:58.886098  2811 ts_tablet_manager.cc:1403] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.257s	user 0.032s	sys 0.058s
I20260709 14:50:58.878918  2882 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.891709  3032 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:58.892259  3032 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:58.892606  3032 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.900938  2807 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:58.901547  2807 ts_tablet_manager.cc:1403] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.179s	user 0.023s	sys 0.047s
I20260709 14:50:58.895798  2986 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 14:50:58.906014  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:58.907013  2684 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.915262  2502 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:58.916370  2878 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:58.917021  2878 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:58.918027  2878 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:50:58.918911  2502 leader_election.cc:343] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.921084  2811 raft_consensus.cc:359] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.923362  2811 raft_consensus.cc:385] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.923677  2811 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.924599  3030 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:58.924530  2811 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:58.926728  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:58.928620  2811 ts_tablet_manager.cc:1434] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.002s
I20260709 14:50:58.929077  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:58.929804  2757 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.929900  2811 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:58.931881  2806 raft_consensus.cc:359] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:50:58.932271  2652 leader_election.cc:343] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:58.932626  2806 raft_consensus.cc:385] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.932991  2806 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.933795  2806 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.937270  2806 ts_tablet_manager.cc:1434] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.136s	user 0.005s	sys 0.002s
I20260709 14:50:58.938644  2806 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:58.940925  2807 raft_consensus.cc:359] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.941989  2807 raft_consensus.cc:385] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:58.942288  2807 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:58.943349  2807 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.959412  2976 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:58.931010  2923 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:58.963210  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:58.964085  2537 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.964797  2923 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:58.966349  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:58.966675  2976 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:58.967880  2811 tablet_bootstrap.cc:654] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.970732  2806 tablet_bootstrap.cc:654] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:58.967473  2760 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:58.977018  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:50:58.979619  2805 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:58.980266  2805 ts_tablet_manager.cc:1403] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.390s	user 0.014s	sys 0.023s
I20260709 14:50:58.982472  2807 ts_tablet_manager.cc:1434] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 14:50:58.984745  2434 catalog_manager.cc:5697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:58.985788  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:58.991222  2533 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:58.987156  2652 leader_election.cc:343] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.004841  3032 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.007809  2807 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:58.987363  2684 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:59.018537  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:50:59.020679  2500 leader_election.cc:343] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:50:59.025738  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:50:59.028262  2537 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:59.031599  2805 raft_consensus.cc:359] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.009073  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:59.037153  2805 raft_consensus.cc:385] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.039083  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:59.040987  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:50:59.041599  2805 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.042477  2923 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.043745  2923 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:59.043746  2500 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:59.044754  2923 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.045172  2878 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.045466  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:50:59.046109  2878 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:59.042726  2861 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.048162  2878 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.042709  2976 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.050199  2533 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:59.049796  2807 tablet_bootstrap.cc:654] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.050228  2976 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:59.052552  2976 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.046144  2760 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.054397  2805 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.057010  2433 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:59.064384  2806 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:59.065287  2806 ts_tablet_manager.cc:1403] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.127s	user 0.030s	sys 0.017s
I20260709 14:50:59.058249  3032 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:59.068646  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:50:59.071966  2805 ts_tablet_manager.cc:1434] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.002s
I20260709 14:50:59.081476  2805 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:59.096051  2533 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.096093  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:59.100867  2882 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.101588  2882 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:50:59.102665  2882 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.102288  2434 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:59.116957  2806 raft_consensus.cc:359] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.119472  2806 raft_consensus.cc:385] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.120385  2806 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.121305  2760 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:59.126976  2811 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:59.127640  2811 ts_tablet_manager.cc:1403] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.198s	user 0.018s	sys 0.024s
I20260709 14:50:59.128333  2502 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:50:59.129670  2878 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.130229  2878 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:59.132098  2878 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.134724  2811 raft_consensus.cc:359] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.135650  2811 raft_consensus.cc:385] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.135663  2433 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:59.136727  2811 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.137744  2811 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.125334  2806 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.140215  2805 tablet_bootstrap.cc:654] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.156364  2435 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:59.159696  2986 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.165676  2806 ts_tablet_manager.cc:1434] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.002s
I20260709 14:50:59.167204  2806 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:59.167096  2811 ts_tablet_manager.cc:1434] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.001s
I20260709 14:50:59.168116  3032 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.168931  2811 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:59.168789  3032 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.172308  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.173534  3032 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:59.175418  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:59.188761  2807 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:59.190006  2807 ts_tablet_manager.cc:1403] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.182s	user 0.022s	sys 0.011s
I20260709 14:50:59.189328  2882 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.191690  2882 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.193791  2435 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:50:59.203763  2806 tablet_bootstrap.cc:654] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.205387  2991 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.214502  2807 raft_consensus.cc:359] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.233295  2807 raft_consensus.cc:385] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.234153  2807 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.247058  2503 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:59.253432  2811 tablet_bootstrap.cc:654] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.255272  2806 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:59.255897  2806 ts_tablet_manager.cc:1403] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.089s	user 0.024s	sys 0.024s
I20260709 14:50:59.259814  2882 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:59.260097  2807 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.262228  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:59.264029  2652 leader_election.cc:343] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.264812  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.267087  2807 ts_tablet_manager.cc:1434] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.000s
W20260709 14:50:59.268265  2651 leader_election.cc:343] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.268989  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:50:59.271829  2882 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:59.273592  2807 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:59.293749  2435 catalog_manager.cc:5697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:50:59.302765  3039 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.304386  3039 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.310312  2610 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.308883  3039 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:59.315750  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:59.316713  2878 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:50:59.318028  2723 leader_election.cc:343] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.318300  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:59.319136  2972 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 14:50:59.325660  2724 leader_election.cc:343] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.326253  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:59.327701  2872 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:59.340782  2805 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:59.341516  2805 ts_tablet_manager.cc:1403] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.260s	user 0.034s	sys 0.031s
I20260709 14:50:59.343019  2806 raft_consensus.cc:359] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.343880  2806 raft_consensus.cc:385] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.344187  2806 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.348001  2811 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:59.348663  2811 ts_tablet_manager.cc:1403] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.180s	user 0.018s	sys 0.007s
I20260709 14:50:59.349562  2805 raft_consensus.cc:359] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.350448  2805 raft_consensus.cc:385] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.350806  2805 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.351821  2805 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.351920  2811 raft_consensus.cc:359] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.352820  2811 raft_consensus.cc:385] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.353586  2811 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.354522  2805 ts_tablet_manager.cc:1434] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 14:50:59.354406  2811 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.356953  2811 ts_tablet_manager.cc:1434] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:50:59.358009  2811 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:59.359570  2805 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:59.352806  2806 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.364638  2806 ts_tablet_manager.cc:1434] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.108s	user 0.007s	sys 0.000s
I20260709 14:50:59.364756  2811 tablet_bootstrap.cc:654] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.365806  2806 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:59.367993  2805 tablet_bootstrap.cc:654] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.368445  2807 tablet_bootstrap.cc:654] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.378451  2811 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:59.379153  2811 ts_tablet_manager.cc:1403] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.001s
I20260709 14:50:59.383690  2811 raft_consensus.cc:359] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.384737  2811 raft_consensus.cc:385] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.385102  2811 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.388968  2811 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.400113  2811 ts_tablet_manager.cc:1434] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.002s
I20260709 14:50:59.401088  2878 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.403357  2878 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.404851  2806 tablet_bootstrap.cc:654] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.405650  2878 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:59.406941  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:59.411638  2684 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:59.412541  2805 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:59.413166  2805 ts_tablet_manager.cc:1403] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.006s
I20260709 14:50:59.408191  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:59.410058  2807 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:59.413044  2500 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:59.408005  2811 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
W20260709 14:50:59.414851  2503 leader_election.cc:343] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.414858  2878 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:59.416334  2878 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:59.416692  2878 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.421655  2807 ts_tablet_manager.cc:1403] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.148s	user 0.018s	sys 0.014s
I20260709 14:50:59.422058  2923 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.422663  2923 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.424985  2923 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:59.424750  2807 raft_consensus.cc:359] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.425549  2807 raft_consensus.cc:385] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.425849  2807 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.426566  2807 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.428519  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.428936  2807 ts_tablet_manager.cc:1434] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:50:59.429965  2807 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
W20260709 14:50:59.430410  2577 leader_election.cc:343] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.427094  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:59.433465  2536 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:50:59.436555  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:59.437122  2807 tablet_bootstrap.cc:654] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.437894  2986 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:59.438732  2986 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:59.439159  2986 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:50:59.450301  2502 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:59.455015  2805 raft_consensus.cc:359] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.455957  2805 raft_consensus.cc:385] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.456252  2805 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.457026  2805 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.459169  2878 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.463186  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:50:59.465328  2503 leader_election.cc:343] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.471729  2811 tablet_bootstrap.cc:654] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.481357  2805 ts_tablet_manager.cc:1434] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 14:50:59.481845  2878 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:59.483932  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:50:59.484854  2684 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.493578  2805 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:59.499195  3036 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.502137  2805 tablet_bootstrap.cc:654] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.515350  2816 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.516227  2816 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.520306  2816 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:59.520735  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:59.527251  2805 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:59.527262  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.528164  2805 ts_tablet_manager.cc:1403] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.035s	user 0.029s	sys 0.003s
I20260709 14:50:59.528425  2757 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:50:59.530746  2684 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:50:59.530531  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:50:59.532226  2500 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:50:59.532060  2805 raft_consensus.cc:359] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.532852  2805 raft_consensus.cc:385] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.533152  2805 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.533883  2805 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.536069  2685 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.536422  2986 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.541129  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:50:59.542912  2577 leader_election.cc:343] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.544322  2805 ts_tablet_manager.cc:1434] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
W20260709 14:50:59.545526  2503 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:59.547214  2976 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:50:59.548388  2923 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.554472  2806 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:50:59.555101  2806 ts_tablet_manager.cc:1403] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.190s	user 0.029s	sys 0.022s
I20260709 14:50:59.556217  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:50:59.556942  2533 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.558180  2806 raft_consensus.cc:359] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.533718  3036 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.565377  3036 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:50:59.565611  2807 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:59.566406  2807 ts_tablet_manager.cc:1403] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.137s	user 0.039s	sys 0.069s
I20260709 14:50:59.566450  3036 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.569190  2878 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:50:59.551822  2986 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:50:59.573304  2806 raft_consensus.cc:385] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.573763  2806 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.575335  2806 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.576429  2807 raft_consensus.cc:359] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.577775  2807 raft_consensus.cc:385] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.578282  2807 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.579152  2807 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:50:59.582072  2807 ts_tablet_manager.cc:1434] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 14:50:59.545522  2805 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:50:59.586740  2807 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:50:59.587316  2811 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:59.588155  2811 ts_tablet_manager.cc:1403] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.187s	user 0.012s	sys 0.007s
I20260709 14:50:59.588346  2976 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:50:59.590356  2878 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.591514  2811 raft_consensus.cc:359] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.591610  2878 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.593959  2878 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:59.595041  2811 raft_consensus.cc:385] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.595471  2811 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.596427  2811 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.600749  2806 ts_tablet_manager.cc:1434] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.003s
I20260709 14:50:59.602167  2806 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:50:59.602818  2807 tablet_bootstrap.cc:654] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.608788  2811 ts_tablet_manager.cc:1434] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 14:50:59.618486  2811 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:59.601822  2533 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:50:59.622633  2976 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.625034  2976 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.630594  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:50:59.632963  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.633749  2757 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:50:59.639603  2805 tablet_bootstrap.cc:654] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.675876  2811 tablet_bootstrap.cc:654] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.656234  3036 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 14:50:59.697647  2806 tablet_bootstrap.cc:654] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:50:59.694485  2976 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:50:59.752699  2611 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.756062  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:59.753808  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:50:59.804919  2872 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.806854  2811 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:50:59.806180  2872 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.812453  2868 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:50:59.813500  2868 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:50:59.815510  2872 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:59.808141  2811 ts_tablet_manager.cc:1403] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.191s	user 0.014s	sys 0.007s
I20260709 14:50:59.848508  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:50:59.866328  2816 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:59.867110  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 113933us
I20260709 14:50:59.867761  2816 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.869835  2986 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:50:59.870599  2986 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:50:59.885623  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:50:59.887292  2816 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:50:59.886996  2652 leader_election.cc:343] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.887877  3039 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:59.888276  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61608.1us
I20260709 14:50:59.888769  3039 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:50:59.891790  2805 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:50:59.892956  2805 ts_tablet_manager.cc:1403] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.348s	user 0.013s	sys 0.006s
I20260709 14:50:59.894119  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:50:59.898243  2652 leader_election.cc:343] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.905295  2811 raft_consensus.cc:359] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.906281  2811 raft_consensus.cc:385] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:50:59.906706  2811 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:50:59.881644  2882 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.116s
I20260709 14:50:59.907892  2811 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:50:59.912374  2811 ts_tablet_manager.cc:1434] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.093s	user 0.004s	sys 0.006s
I20260709 14:50:59.917502  2868 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:50:59.922719  2811 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:50:59.931766  2502 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:50:59.921015  2756 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:50:59.936983  2868 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:50:59.943785  2807 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:50:59.944162  2868 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:50:59.944739  2868 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.945241  2807 ts_tablet_manager.cc:1403] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.359s	user 0.020s	sys 0.010s
W20260709 14:50:59.948266  2652 consensus_peers.cc:597] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:50:59.948617  2986 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:50:59.952669  2500 leader_election.cc:343] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:50:59.955046  3039 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:50:59.966234  2757 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:50:59.968598  3075 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:50:59.937623  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 122314us
I20260709 14:51:00.003307  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:50:59.987146  2807 raft_consensus.cc:359] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.015491  2807 raft_consensus.cc:385] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.015978  2807 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.026578  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:00.031577  2755 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.038523  2986 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:00.045841  2806 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:00.047049  2805 raft_consensus.cc:359] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.048058  2805 raft_consensus.cc:385] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.048362  2805 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.049110  2805 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.033497  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:00.051499  2923 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.009297  3075 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.046657  2806 ts_tablet_manager.cc:1403] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.445s	user 0.015s	sys 0.027s
I20260709 14:51:00.036706  2807 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.057277  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:00.073824  2430 catalog_manager.cc:5697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:00.081027  2868 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.052284  2923 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.088505  2923 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:00.074678  2805 ts_tablet_manager.cc:1434] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.181s	user 0.009s	sys 0.002s
W20260709 14:51:00.073823  2726 leader_election.cc:343] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.117419  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:00.121037  2805 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:00.122222  2807 ts_tablet_manager.cc:1434] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.176s	user 0.012s	sys 0.002s
I20260709 14:51:00.125119  2806 raft_consensus.cc:359] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.126202  2806 raft_consensus.cc:385] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.126549  2806 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.127529  2806 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.131081  2807 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:00.125171  3075 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:00.137049  2502 leader_election.cc:343] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.150177  3081 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:51:00.151273  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:00.153793  2811 tablet_bootstrap.cc:654] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.160161  2807 tablet_bootstrap.cc:654] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
W20260709 14:51:00.167730  3093 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 14:51:00.180533  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:00.181514  2535 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:00.183459  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:00.189105  2503 leader_election.cc:343] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.189787  2503 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.193468  3035 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.195936  2805 tablet_bootstrap.cc:654] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.166740  2868 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:00.194988  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:00.215377  2575 leader_election.cc:343] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.224071  2684 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:00.227566  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:00.228742  2756 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:00.236212  2503 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:00.238193  2806 ts_tablet_manager.cc:1434] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.185s	user 0.009s	sys 0.002s
I20260709 14:51:00.208223  3035 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 14:51:00.251304  2945 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.253556  2945 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.255110  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:00.256649  2945 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:00.266142  2806 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:00.267457  2811 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:00.268103  2811 ts_tablet_manager.cc:1403] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.354s	user 0.039s	sys 0.038s
I20260709 14:51:00.273407  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:00.274212  2811 raft_consensus.cc:359] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.275024  2811 raft_consensus.cc:385] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.275352  2811 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.276160  2811 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.292227  3039 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:00.298074  3039 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.301142  3039 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:00.302894  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:00.315696  2811 ts_tablet_manager.cc:1434] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.003s
I20260709 14:51:00.317165  2811 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:00.292374  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.345568  2806 tablet_bootstrap.cc:654] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.335768  2868 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 14:51:00.363205  2945 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:00.363873  2945 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:00.363605  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 75311.9us
I20260709 14:51:00.364370  2945 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:00.368629  2651 leader_election.cc:343] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.369789  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.373481  2945 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.375895  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:00.376184  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:00.377012  2684 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.377220  3075 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:00.377408  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:00.377715  3075 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:00.377789  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:00.378221  3075 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.378710  2986 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:00.384375  2652 leader_election.cc:343] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.385432  2986 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.386047  2986 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.392529  2972 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.394659  2986 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:00.395871  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:00.397890  3081 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.406458  2811 tablet_bootstrap.cc:654] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.408602  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:00.409560  2686 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:00.413041  2805 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:00.413558  2805 ts_tablet_manager.cc:1403] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.293s	user 0.007s	sys 0.008s
I20260709 14:51:00.419230  2806 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:00.419847  2806 ts_tablet_manager.cc:1403] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.154s	user 0.010s	sys 0.005s
I20260709 14:51:00.420899  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:00.421909  2986 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:00.422302  2986 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:00.422652  2986 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.424350  2806 raft_consensus.cc:359] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.425101  2806 raft_consensus.cc:385] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.425464  2806 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.426267  2806 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.432098  2805 raft_consensus.cc:359] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.428361  2806 ts_tablet_manager.cc:1434] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:51:00.433027  2805 raft_consensus.cc:385] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.433359  2805 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.434021  2756 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:00.434298  2805 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.434340  2806 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:00.443668  2807 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:00.445494  2807 ts_tablet_manager.cc:1403] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.315s	user 0.030s	sys 0.007s
W20260709 14:51:00.459138  2577 leader_election.cc:343] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:51:00.470682  2577 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:00.476737  2502 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:00.446144  2945 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:00.461524  2807 raft_consensus.cc:359] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.479939  2807 raft_consensus.cc:385] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.480274  2807 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.480868  2807 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.482654  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.483736  2991 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 14:51:00.453738  3104 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 14:51:00.484109  2991 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:00.486500  2807 ts_tablet_manager.cc:1434] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.002s
I20260709 14:51:00.487963  2991 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:00.490589  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:00.494489  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:00.488488  2805 ts_tablet_manager.cc:1434] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.001s
I20260709 14:51:00.485493  3085 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:00.495711  3085 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:00.491818  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:00.496085  2805 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:00.497524  2758 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:00.496718  3085 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.505550  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:00.466820  2684 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:00.474450  3075 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.508287  3081 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:00.508741  3081 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:00.511597  3081 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.515518  2807 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:00.523291  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:00.517678  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:00.531230  2434 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:00.549474  2606 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.551131  3075 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:00.554173  2806 tablet_bootstrap.cc:654] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.563247  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:00.567849  2575 leader_election.cc:343] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.569010  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:00.570539  2972 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:00.573159  2805 tablet_bootstrap.cc:654] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.542686  3081 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.596504  2811 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:00.597232  2811 ts_tablet_manager.cc:1403] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.280s	user 0.032s	sys 0.014s
I20260709 14:51:00.576109  2986 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.603111  2811 raft_consensus.cc:359] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.605334  2811 raft_consensus.cc:385] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.605700  2811 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.605993  2807 tablet_bootstrap.cc:654] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.606505  2811 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.609299  3075 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:51:00.614766  2650 leader_election.cc:343] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.634680  2811 ts_tablet_manager.cc:1434] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 14:51:00.646311  3081 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:00.647890  2811 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:00.656184  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:00.665846  2756 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.675479  2811 tablet_bootstrap.cc:654] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.636346  2991 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.680948  2757 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:00.682346  2991 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:00.683877  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:00.684466  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:00.684926  2686 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:51:00.688198  2726 leader_election.cc:343] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.688776  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:00.690521  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:00.691390  2686 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.691704  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.693110  2945 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.693950  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:00.700969  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:00.701730  2536 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.704061  2435 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:00.716118  2536 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:00.716650  2986 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:00.717558  2806 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:00.718108  2806 ts_tablet_manager.cc:1403] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.284s	user 0.020s	sys 0.009s
W20260709 14:51:00.719496  2650 leader_election.cc:343] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.721360  2806 raft_consensus.cc:359] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.722172  2806 raft_consensus.cc:385] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.722460  2806 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.730607  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:00.733243  3102 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 14:51:00.733944  2577 leader_election.cc:343] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.736364  2805 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:00.736994  2805 ts_tablet_manager.cc:1403] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.241s	user 0.019s	sys 0.014s
I20260709 14:51:00.738354  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:00.742139  2805 raft_consensus.cc:359] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.742957  2805 raft_consensus.cc:385] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.743251  2805 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.740687  2806 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.744118  2805 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.745462  2756 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:00.747385  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:00.747961  2806 ts_tablet_manager.cc:1434] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 14:51:00.749172  2945 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:00.749747  2945 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:00.750805  2945 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.749836  2806 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:00.755261  2986 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:00.755862  2986 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:00.756778  2986 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.754638  2805 ts_tablet_manager.cc:1434] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 14:51:00.760408  2805 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:00.761436  2806 tablet_bootstrap.cc:654] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.783026  2807 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:00.783668  2807 ts_tablet_manager.cc:1403] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.268s	user 0.021s	sys 0.015s
I20260709 14:51:00.787513  2807 raft_consensus.cc:359] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.790614  2807 raft_consensus.cc:385] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.791136  2807 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.792294  2807 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.813702  2686 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:00.804314  3091 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.819820  3091 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.820462  2807 ts_tablet_manager.cc:1434] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 14:51:00.821851  2807 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:00.823921  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:00.825168  2861 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:00.825721  2861 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:00.826670  2861 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:00.829061  2807 tablet_bootstrap.cc:654] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.822508  3091 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:00.815414  3075 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.836439  2805 tablet_bootstrap.cc:654] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.837328  3075 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.841379  3075 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:00.849172  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:00.855294  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:00.860086  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:00.861160  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:00.865039  2503 leader_election.cc:343] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.867447  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:00.867118  2435 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:00.868459  2500 leader_election.cc:343] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.869000  2500 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:00.869902  3077 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.873106  2756 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.875489  2811 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:00.876067  2811 ts_tablet_manager.cc:1403] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.229s	user 0.038s	sys 0.027s
I20260709 14:51:00.892990  2811 raft_consensus.cc:359] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.894141  2811 raft_consensus.cc:385] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.894526  2811 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.895677  2811 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.897693  2878 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:00.901702  3102 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.902467  3102 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:00.926183  2811 ts_tablet_manager.cc:1434] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 14:51:00.927634  2811 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:00.935693  2806 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:00.936286  2806 ts_tablet_manager.cc:1403] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.187s	user 0.010s	sys 0.021s
I20260709 14:51:00.941403  2811 tablet_bootstrap.cc:654] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:00.944455  2806 raft_consensus.cc:359] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.945250  2806 raft_consensus.cc:385] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:00.945521  2806 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:00.947530  3030 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.948101  3030 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:00.948546  3102 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:00.951411  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:00.953477  2726 leader_election.cc:343] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:00.955365  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:00.956523  2535 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:00.946439  2806 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:00.965319  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:00.966959  3039 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:00.967229  2807 tablet_bootstrap.cc:492] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:00.967386  3039 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:00.967826  3039 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:00.967864  2807 ts_tablet_manager.cc:1403] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.004s
I20260709 14:51:00.981181  2430 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:00.984421  2430 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:00.985826  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:00.963531  3030 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:00.999712  2878 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:00.999918  3035 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.000306  2878 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.002882  2878 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.017911  2806 ts_tablet_manager.cc:1434] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 14:51:01.007085  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.019853  2805 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:01.020484  2805 ts_tablet_manager.cc:1403] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.260s	user 0.007s	sys 0.022s
I20260709 14:51:01.022017  2807 raft_consensus.cc:359] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.023654  2805 raft_consensus.cc:359] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.024467  2805 raft_consensus.cc:385] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.024745  2805 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.026855  2805 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.026611  2807 raft_consensus.cc:385] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.028491  2807 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.030118  2807 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:01.029755  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51569.6us
I20260709 14:51:01.031096  2805 ts_tablet_manager.cc:1434] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 14:51:01.036782  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:01.052649  2502 leader_election.cc:343] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.045867  2683 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:51:01.059343  2807 ts_tablet_manager.cc:1434] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 14:51:01.060691  2807 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:01.068169  2991 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:01.068991  2991 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.067095  2500 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:01.041854  3030 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.072199  3039 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.076601  3035 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:01.077188  3035 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:01.077436  2807 tablet_bootstrap.cc:654] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.077605  3035 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.080644  2991 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:01.082592  2806 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:01.081172  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:01.072057  3030 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.087798  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:01.088058  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:01.089381  2535 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.089850  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:01.091182  2991 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:01.091538  3121 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 14:51:01.093452  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:01.097918  2726 leader_election.cc:343] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.092167  3039 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:01.102799  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:01.091218  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:01.106645  2536 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 1: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:01.106444  3030 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.108004  2805 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:01.109958  2878 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.114199  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.115044  2757 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
W20260709 14:51:01.109371  2652 leader_election.cc:343] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.125138  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:01.126406  2811 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:01.126266  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:01.127089  2811 ts_tablet_manager.cc:1403] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.200s	user 0.034s	sys 0.022s
I20260709 14:51:01.128265  2805 tablet_bootstrap.cc:654] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.128695  2816 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:01.129143  2816 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:01.129500  2816 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.130944  2945 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.133297  2878 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.133926  2878 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.137373  2806 tablet_bootstrap.cc:654] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.138280  2878 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.141347  2535 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:01.147716  2945 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:01.151813  2811 raft_consensus.cc:359] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.140493  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:01.152988  2811 raft_consensus.cc:385] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.157069  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:01.167702  3039 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:01.168308  3039 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:01.168443  2811 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.169337  3039 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.175238  2811 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.177726  3081 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.224440  2608 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.225471  2811 ts_tablet_manager.cc:1434] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.000s
I20260709 14:51:01.238062  2923 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.240288  2923 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.251973  2923 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.254077  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:01.255298  2535 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.255555  2684 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:01.256278  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.257246  2757 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 1: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:01.259275  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.264889  2923 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.267513  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:01.269572  2861 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.298013  2811 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:01.221827  3035 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.304968  2806 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:01.305006  2807 tablet_bootstrap.cc:492] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:01.324018  2806 ts_tablet_manager.cc:1403] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.279s	user 0.047s	sys 0.039s
I20260709 14:51:01.324404  2807 ts_tablet_manager.cc:1403] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.264s	user 0.015s	sys 0.020s
W20260709 14:51:01.234347  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51725.6us
I20260709 14:51:01.353484  2806 raft_consensus.cc:359] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.355438  2806 raft_consensus.cc:385] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.355733  2806 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.357398  2806 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.362005  2811 tablet_bootstrap.cc:654] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.365810  2805 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:01.366492  2805 ts_tablet_manager.cc:1403] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.320s	user 0.033s	sys 0.018s
I20260709 14:51:01.385424  2806 ts_tablet_manager.cc:1434] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.001s
I20260709 14:51:01.384155  2807 raft_consensus.cc:359] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.387542  2807 raft_consensus.cc:385] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.387867  2807 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.392536  2861 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:01.402132  2807 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.412246  2805 raft_consensus.cc:359] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.413254  2805 raft_consensus.cc:385] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.413556  2805 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.416455  2805 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.423164  2806 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:01.391001  2816 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.426286  2807 ts_tablet_manager.cc:1434] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.004s
I20260709 14:51:01.430202  2805 ts_tablet_manager.cc:1434] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.003s
I20260709 14:51:01.413618  3140 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.434034  2435 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:01.456362  2816 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.415453  3081 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 14:51:01.457858  2502 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:01.459519  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:01.460254  2685 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.462003  2868 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:01.458242  2805 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:01.482877  3035 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:01.484853  3135 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 14:51:01.485615  3075 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.493180  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:01.494028  2611 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.497578  2991 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.498241  2991 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.501106  2991 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:01.503100  2577 consensus_peers.cc:597] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:01.505941  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:01.507270  2807 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:01.515573  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:01.519364  2724 leader_election.cc:343] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.451241  3140 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.530110  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:01.526377  2652 leader_election.cc:343] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.530884  3140 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:01.534353  3137 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 14:51:01.557847  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:01.559051  2758 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:01.534368  2726 leader_election.cc:343] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.567389  2806 tablet_bootstrap.cc:654] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
W20260709 14:51:01.587883  3142 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
I20260709 14:51:01.594381  2811 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:01.598089  2811 ts_tablet_manager.cc:1403] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.337s	user 0.018s	sys 0.008s
I20260709 14:51:01.569679  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:01.581851  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 94004.4us
I20260709 14:51:01.605440  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:01.619648  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.607529  2972 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:51:01.621557  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 89955.8us
I20260709 14:51:01.630895  2685 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
W20260709 14:51:01.615231  2652 leader_election.cc:343] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.661831  3039 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 14:51:01.673707  2651 leader_election.cc:343] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.674381  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:01.638294  2502 leader_election.cc:343] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.669890  2805 tablet_bootstrap.cc:654] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.696627  3140 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:01.690781  2500 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:01.703907  2868 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:01.704424  2868 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:01.705065  2807 tablet_bootstrap.cc:654] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.705221  2868 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:01.684823  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45440.6us
W20260709 14:51:01.689685  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52373.4us
W20260709 14:51:01.734330  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:01.744323  2758 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
W20260709 14:51:01.720108  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 98423.4us
I20260709 14:51:01.768143  2806 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:01.768748  2806 ts_tablet_manager.cc:1403] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.356s	user 0.018s	sys 0.013s
I20260709 14:51:01.770139  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:01.773240  2861 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:01.772779  2816 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 14:51:01.773131  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45077.5us
I20260709 14:51:01.774793  2811 raft_consensus.cc:359] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.783008  2811 raft_consensus.cc:385] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.782498  2816 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:01.784634  2806 raft_consensus.cc:359] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.788133  2806 raft_consensus.cc:385] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.785112  2811 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.789863  3035 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:01.790192  2726 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:01.790446  3035 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.792810  3035 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:01.789317  2806 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.797048  2806 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.799154  2806 ts_tablet_manager.cc:1434] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 14:51:01.799757  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:01.800263  2806 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:01.811316  2811 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.820317  2807 tablet_bootstrap.cc:492] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:01.820956  2807 ts_tablet_manager.cc:1403] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.343s	user 0.008s	sys 0.014s
I20260709 14:51:01.817883  2683 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:51:01.786917  2816 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.831148  2500 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:01.831755  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:01.832115  2868 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:01.832504  2868 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:01.832835  2868 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.833532  2807 raft_consensus.cc:359] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.834338  2807 raft_consensus.cc:385] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 14:51:01.835652  2503 leader_election.cc:343] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.834616  2807 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.841591  2805 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:01.842428  2805 ts_tablet_manager.cc:1403] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.391s	user 0.014s	sys 0.014s
I20260709 14:51:01.842165  2807 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:01.845665  2807 ts_tablet_manager.cc:1434] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 14:51:01.846912  2807 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:01.847736  2868 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.854491  2807 tablet_bootstrap.cc:654] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.856329  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:01.845297  2811 ts_tablet_manager.cc:1434] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.247s	user 0.010s	sys 0.000s
I20260709 14:51:01.857410  2684 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:01.859210  2811 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:01.864032  2868 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:01.864207  2805 raft_consensus.cc:359] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.867123  2805 raft_consensus.cc:385] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.867646  2805 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.868229  2806 tablet_bootstrap.cc:654] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.870806  2805 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.892298  2805 ts_tablet_manager.cc:1434] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.003s
I20260709 14:51:01.893594  2805 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:01.903290  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:01.886765  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
W20260709 14:51:01.905099  2503 leader_election.cc:343] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.905922  2500 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:01.906993  3035 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:01.907531  3035 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:01.908437  3035 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:01.910615  2811 tablet_bootstrap.cc:654] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.915755  3146 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:01.919322  3146 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:01.919188  2431 catalog_manager.cc:5697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:01.921903  3146 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:01.925118  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:01.927557  2500 leader_election.cc:343] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.937006  2807 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:01.940671  2807 ts_tablet_manager.cc:1403] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.043s
I20260709 14:51:01.943938  2807 raft_consensus.cc:359] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.948201  2807 raft_consensus.cc:385] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:01.948559  2807 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:01.949215  2807 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:01.951501  2807 ts_tablet_manager.cc:1434] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 14:51:01.929605  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:01.940033  2805 tablet_bootstrap.cc:654] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.959129  2435 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:51:01.964403  2502 leader_election.cc:343] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:01.965055  2502 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:01.966318  3035 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:01.971836  2811 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:01.972548  2811 ts_tablet_manager.cc:1403] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.114s	user 0.019s	sys 0.003s
I20260709 14:51:01.986797  2806 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:01.988771  2806 ts_tablet_manager.cc:1403] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.189s	user 0.014s	sys 0.001s
I20260709 14:51:01.987679  2807 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:02.000545  2807 tablet_bootstrap.cc:654] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:01.989326  2811 raft_consensus.cc:359] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.001211  2811 raft_consensus.cc:385] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.001569  2811 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.002379  2811 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.004024  2431 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:02.005191  2811 ts_tablet_manager.cc:1434] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 14:51:02.008831  2806 raft_consensus.cc:359] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.009748  2806 raft_consensus.cc:385] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.010142  2806 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.011056  2806 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.006793  2811 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:02.014580  2805 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:02.015465  2805 ts_tablet_manager.cc:1403] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.122s	user 0.010s	sys 0.004s
I20260709 14:51:02.021801  2805 raft_consensus.cc:359] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.022630  2805 raft_consensus.cc:385] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.023131  2811 tablet_bootstrap.cc:654] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.014582  2806 ts_tablet_manager.cc:1434] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 14:51:02.028538  2806 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:02.053526  2806 tablet_bootstrap.cc:654] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.022951  2805 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.060633  2807 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:02.061218  2807 ts_tablet_manager.cc:1403] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.074s	user 0.026s	sys 0.020s
I20260709 14:51:02.064606  2807 raft_consensus.cc:359] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.065446  2807 raft_consensus.cc:385] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.065843  2807 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.069298  2806 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:02.066783  2807 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.069860  2806 ts_tablet_manager.cc:1403] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.002s
I20260709 14:51:02.072927  2807 ts_tablet_manager.cc:1434] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 14:51:02.074229  2807 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:02.081478  2807 tablet_bootstrap.cc:654] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.089635  2811 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:02.090440  2811 ts_tablet_manager.cc:1403] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.005s
I20260709 14:51:02.092191  3077 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.072883  2806 raft_consensus.cc:359] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.092825  3077 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.093703  2806 raft_consensus.cc:385] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.067755  2805 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.097219  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:02.099102  3077 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:02.100327  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:02.102613  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:02.103657  2807 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:02.099792  2805 ts_tablet_manager.cc:1434] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 14:51:02.103883  3077 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:02.104254  2807 ts_tablet_manager.cc:1403] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.001s
I20260709 14:51:02.107803  2805 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:02.111747  2807 raft_consensus.cc:359] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.112519  2807 raft_consensus.cc:385] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.112807  2807 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.113498  2807 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.115659  2807 ts_tablet_manager.cc:1434] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 14:51:02.111107  2811 raft_consensus.cc:359] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.116529  2811 raft_consensus.cc:385] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.116792  2807 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:02.116839  2811 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.119431  2811 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.094004  2806 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.121181  2806 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.124022  2806 ts_tablet_manager.cc:1434] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 14:51:02.125504  2807 tablet_bootstrap.cc:654] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.129375  2811 ts_tablet_manager.cc:1434] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 14:51:02.129712  2806 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:02.130160  3077 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.130743  3077 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.131325  2811 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:02.135650  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:02.136430  2537 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:02.138381  2811 tablet_bootstrap.cc:654] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.138928  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:02.139899  2923 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:02.140290  2923 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:02.140651  2923 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.137660  3077 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:02.137609  2806 tablet_bootstrap.cc:654] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.161147  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:02.162252  2683 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:02.166721  2806 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:02.167486  2806 ts_tablet_manager.cc:1403] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.004s
I20260709 14:51:02.180668  2806 raft_consensus.cc:359] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.181703  2806 raft_consensus.cc:385] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.182034  2806 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.182902  2806 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.192442  2806 ts_tablet_manager.cc:1434] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 14:51:02.193619  2806 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:02.201869  2806 tablet_bootstrap.cc:654] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.205219  3077 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.205894  3077 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.209637  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:02.210457  2756 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:02.210531  3077 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:02.212110  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:02.225705  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:02.226734  2537 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:02.233068  2608 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.241546  2861 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:02.242107  2861 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:02.242445  2861 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.259382  3091 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:02.265463  2861 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.292819  2861 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:02.250500  2806 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:02.296203  2806 ts_tablet_manager.cc:1403] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.006s
I20260709 14:51:02.298069  2805 tablet_bootstrap.cc:654] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.278313  3077 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.286603  2816 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:02.304838  2923 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.307420  2806 raft_consensus.cc:359] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.306437  3077 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.310094  3081 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.310653  3081 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.311125  3077 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:02.312871  3081 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:02.313553  2923 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:02.319588  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:02.321278  2651 leader_election.cc:343] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.308068  2806 raft_consensus.cc:385] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.322762  2806 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.317255  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.323566  3039 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.323585  2806 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.331764  2806 ts_tablet_manager.cc:1434] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 14:51:02.335640  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:02.328270  3039 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.336400  2535 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.353304  2807 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:02.354374  2807 ts_tablet_manager.cc:1403] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.238s	user 0.031s	sys 0.048s
W20260709 14:51:02.356029  2652 leader_election.cc:343] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.356792  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:02.327432  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56626.9us
I20260709 14:51:02.383035  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:02.383162  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:02.383739  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:02.384107  2534 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.384259  2756 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.371773  3140 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:02.371341  3039 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:02.406397  2503 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:02.409863  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:02.411490  2611 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.418350  2806 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:02.410688  2685 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:02.427023  2724 leader_election.cc:343] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.437529  2811 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:02.438685  2811 ts_tablet_manager.cc:1403] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.308s	user 0.037s	sys 0.034s
I20260709 14:51:02.445390  2534 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:02.466974  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:02.470758  2805 tablet_bootstrap.cc:492] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:02.471453  2805 ts_tablet_manager.cc:1403] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.364s	user 0.013s	sys 0.014s
I20260709 14:51:02.449137  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:02.477725  2755 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.462709  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:02.491029  3178 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:02.491573  3178 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:02.489676  3146 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.492430  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.493099  3146 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.487375  3035 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.492483  3178 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:02.495276  3035 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.512045  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:02.519438  2726 leader_election.cc:343] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.480504  2535 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:02.525641  2807 raft_consensus.cc:359] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.526816  2807 raft_consensus.cc:385] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.527246  2807 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.532707  2807 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.536782  2807 ts_tablet_manager.cc:1434] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.182s	user 0.008s	sys 0.000s
I20260709 14:51:02.544586  2807 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:02.546145  3039 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.546882  3039 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:02.555002  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45115.5us
I20260709 14:51:02.524160  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:02.574119  3187 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:02.576079  2685 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:02.569449  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:02.553988  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84692.3us
W20260709 14:51:02.612082  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48637.9us
I20260709 14:51:02.612306  3179 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:02.612888  3179 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:02.613862  3179 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.616449  3183 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.616796  2807 tablet_bootstrap.cc:654] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.617007  3183 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.619402  3183 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:02.620357  2805 raft_consensus.cc:359] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.621184  2805 raft_consensus.cc:385] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.621492  2805 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.625638  2811 raft_consensus.cc:359] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.626478  2811 raft_consensus.cc:385] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.627573  3146 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:02.626770  2811 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
W20260709 14:51:02.608337  2503 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46524.2us
I20260709 14:51:02.622210  2805 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.630689  2811 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:02.651083  2805 ts_tablet_manager.cc:1434] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.179s	user 0.009s	sys 0.000s
I20260709 14:51:02.611048  3035 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:02.654165  2811 ts_tablet_manager.cc:1434] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.215s	user 0.008s	sys 0.000s
I20260709 14:51:02.637658  3183 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:02.609886  3140 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260709 14:51:02.677515  2806 tablet_bootstrap.cc:654] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.678198  2816 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.679023  2816 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.669752  3189 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.689073  2816 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:02.695981  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.697206  3039 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:02.698915  2503 leader_election.cc:343] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.713622  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:02.714779  2686 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:02.698704  3189 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.725932  3189 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:02.729146  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:02.730202  2758 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:02.731146  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:02.729275  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:02.737689  2811 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:02.738049  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:02.744048  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 230189us
I20260709 14:51:02.746789  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.749383  3197 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:02.750288  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:02.750842  2611 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:02.750957  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:02.761881  2503 leader_election.cc:343] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:51:02.759922  2500 leader_election.cc:343] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.784884  2500 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:02.790218  2712 tablet.cc:2406] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 14:51:02.759291  3198 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
W20260709 14:51:02.809007  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 14:51:02.759428  2502 leader_election.cc:343] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.798316  3194 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:51:02.819191  2500 leader_election.cc:343] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:02.834421  2807 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:02.835129  2807 ts_tablet_manager.cc:1403] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.291s	user 0.017s	sys 0.007s
I20260709 14:51:02.838056  2805 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
W20260709 14:51:02.822553  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58374.2us
I20260709 14:51:02.806329  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:02.845739  2807 raft_consensus.cc:359] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.849854  2807 raft_consensus.cc:385] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.850857  2807 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.851773  3039 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:02.849067  2806 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:02.852281  3039 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:02.852727  2806 ts_tablet_manager.cc:1403] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.460s	user 0.019s	sys 0.019s
I20260709 14:51:02.856704  2806 raft_consensus.cc:359] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.852694  3039 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:02.858379  2807 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.863998  2806 raft_consensus.cc:385] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:02.865602  2806 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:02.868126  2806 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.868850  2811 tablet_bootstrap.cc:654] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:02.873549  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.874488  2606 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:02.877264  2500 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:02.878952  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 85726.6us
I20260709 14:51:02.882169  3200 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.882808  3200 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:02.850750  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49027.4us
I20260709 14:51:02.896320  2806 ts_tablet_manager.cc:1434] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 14:51:02.842772  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:02.879884  3146 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:02.923218  2816 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:02.923765  2816 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:02.923787  2807 ts_tablet_manager.cc:1434] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.088s	user 0.010s	sys 0.000s
I20260709 14:51:02.924208  2816 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:02.923221  3194 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.927855  3194 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:02.929737  3200 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:02.930179  3194 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:02.933697  2806 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:02.949216  3187 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.950237  3187 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.953019  2755 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:02.954953  3153 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:02.955523  3153 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.957795  3153 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:02.960554  2755 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:02.962347  2807 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:02.969635  2816 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:02.978945  2816 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:02.976375  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.981335  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:02.926195  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 95415.2us
I20260709 14:51:02.994720  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:02.995842  2609 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:51:02.997961  2503 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:02.999380  3189 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:02.999913  3189 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:03.000345  3189 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.001926  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:03.011951  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:03.013062  2682 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:03.021709  3187 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:03.023669  2806 tablet_bootstrap.cc:654] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.029174  2805 tablet_bootstrap.cc:654] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.002756  2534 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:03.024165  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:03.047369  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:03.048228  2609 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:03.052821  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:03.054793  3200 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:03.055474  3200 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:03.055855  3200 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.056715  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:03.057504  2760 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
W20260709 14:51:03.058107  2502 leader_election.cc:343] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.063192  2502 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:03.068171  3035 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:03.070447  3200 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:03.072576  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:03.072913  3200 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:03.073530  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:03.074745  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:03.075101  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:03.075519  2537 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.084033  3189 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.094790  3177 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:03.096899  3177 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:03.060060  3039 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:03.060559  2577 leader_election.cc:343] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.107720  2807 tablet_bootstrap.cc:654] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.107394  3189 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:03.123246  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62731.1us
I20260709 14:51:03.138782  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:03.143601  3205 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:03.144212  3205 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:03.144335  2757 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:03.144676  3205 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.146380  3039 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:03.148716  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:03.161808  2806 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:03.163951  2609 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:03.167747  2806 ts_tablet_manager.cc:1403] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.252s	user 0.022s	sys 0.011s
I20260709 14:51:03.167627  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:03.172308  2431 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:03.152642  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91054.7us
I20260709 14:51:03.179476  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:03.180220  2606 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.174803  3200 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:03.183473  3200 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:03.184490  3200 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.174569  2431 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:03.198491  2811 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:03.199239  2811 ts_tablet_manager.cc:1403] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.462s	user 0.034s	sys 0.017s
I20260709 14:51:03.199831  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:03.222848  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:03.233335  2606 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:03.236847  2537 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
W20260709 14:51:03.244485  2651 leader_election.cc:343] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.243273  2811 raft_consensus.cc:359] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.246316  2811 raft_consensus.cc:385] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.246954  2811 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.271158  2805 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:03.271848  2805 ts_tablet_manager.cc:1403] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.445s	user 0.020s	sys 0.031s
I20260709 14:51:03.260658  3205 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.290313  2805 raft_consensus.cc:359] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:03.291180  2805 raft_consensus.cc:385] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.291481  2805 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.292186  2805 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:03.297991  2811 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.300876  2811 ts_tablet_manager.cc:1434] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.003s
I20260709 14:51:03.297996  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:03.301772  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:03.288128  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:03.308640  2757 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:03.302520  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56643.4us
I20260709 14:51:03.286556  3205 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:03.311650  2503 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:03.314361  2805 ts_tablet_manager.cc:1434] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 14:51:03.313210  2806 raft_consensus.cc:359] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.315980  2806 raft_consensus.cc:385] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.316486  2806 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.317843  2806 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.320048  2811 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:03.330395  2805 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:03.331233  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:03.325515  2577 leader_election.cc:343] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.328001  3091 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 14:51:03.371081  3213 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 14:51:03.335816  3153 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:03.373142  3199 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:03.373808  3199 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:03.385736  2807 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:03.386334  2807 ts_tablet_manager.cc:1403] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.424s	user 0.015s	sys 0.009s
W20260709 14:51:03.368151  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46301.8us
I20260709 14:51:03.385123  3091 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:03.391724  3091 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.383932  3153 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:03.398047  2811 tablet_bootstrap.cc:654] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.397819  3153 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:03.401506  3153 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:03.401727  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:03.406148  2536 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:03.406148  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:03.407078  2681 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.382649  3200 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:03.410254  3200 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:03.411629  3218 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 14:51:03.413054  3200 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:03.416620  3077 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:03.417335  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:03.418013  2760 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.423410  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:03.433535  2807 raft_consensus.cc:359] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.374752  3199 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:03.434811  2807 raft_consensus.cc:385] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.445566  2807 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.447198  3193 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:03.448026  3193 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.461308  3193 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:03.462914  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:03.463836  2806 ts_tablet_manager.cc:1434] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.295s	user 0.014s	sys 0.005s
W20260709 14:51:03.464843  2503 leader_election.cc:343] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.465137  2806 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:03.468465  2807 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:03.471240  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83867.7us
I20260709 14:51:03.481312  2807 ts_tablet_manager.cc:1434] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.095s	user 0.009s	sys 0.000s
I20260709 14:51:03.488394  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:03.494599  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:03.496266  2805 tablet_bootstrap.cc:654] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.497661  3222 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:03.499862  2861 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:03.494618  3153 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:03.474992  2681 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:03.500191  2807 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:03.504355  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:03.506592  3077 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:03.508806  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:03.509135  3077 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:03.507810  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:03.511830  2807 tablet_bootstrap.cc:654] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.512281  2757 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
W20260709 14:51:03.517334  2652 leader_election.cc:343] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.517246  2760 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:03.521456  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:03.540792  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 211227us
I20260709 14:51:03.510228  3077 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:03.540904  2502 leader_election.cc:343] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.543105  3153 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:03.554389  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:03.561511  2537 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:03.553273  2502 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:03.622583  2811 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:03.623231  2811 ts_tablet_manager.cc:1403] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.310s	user 0.015s	sys 0.016s
W20260709 14:51:03.625931  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85955.1us
I20260709 14:51:03.626488  2811 raft_consensus.cc:359] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.627311  2811 raft_consensus.cc:385] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.627631  2811 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.627712  3146 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:03.628400  2811 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.633388  2811 ts_tablet_manager.cc:1434] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 14:51:03.634639  2811 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:03.648967  2811 tablet_bootstrap.cc:654] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.654565  2806 tablet_bootstrap.cc:654] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.663085  2807 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:03.663767  2807 ts_tablet_manager.cc:1403] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.164s	user 0.013s	sys 0.040s
I20260709 14:51:03.690876  2609 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:51:03.700376  3199 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 14:51:03.689172  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 119023us
I20260709 14:51:03.720729  2807 raft_consensus.cc:359] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.721694  2807 raft_consensus.cc:385] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.722005  2807 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.723323  2807 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:03.700850  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116282us
I20260709 14:51:03.746179  2807 ts_tablet_manager.cc:1434] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.003s
I20260709 14:51:03.748684  2805 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:03.756042  2805 ts_tablet_manager.cc:1403] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.426s	user 0.028s	sys 0.069s
I20260709 14:51:03.747839  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:03.779067  3199 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:03.779589  3199 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:03.780012  3199 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.802281  2811 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:03.802953  2811 ts_tablet_manager.cc:1403] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.169s	user 0.026s	sys 0.016s
I20260709 14:51:03.805311  2811 raft_consensus.cc:359] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.805967  2811 raft_consensus.cc:385] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.806257  2811 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.806979  2811 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.810269  2811 ts_tablet_manager.cc:1434] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:51:03.809172  3199 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.812649  2811 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:03.816140  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:03.817006  2607 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.819233  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:03.820055  2533 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:03.821728  2805 raft_consensus.cc:359] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.826056  2805 raft_consensus.cc:385] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.826659  2805 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.827647  2805 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:03.830050  2805 ts_tablet_manager.cc:1434] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.001s
I20260709 14:51:03.831027  2807 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:03.831306  2805 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:03.835521  2533 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:03.853735  2607 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:03.853902  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:03.826360  3199 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:03.870949  3200 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:03.871538  3200 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:03.872479  3200 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.861320  2806 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:03.874643  2806 ts_tablet_manager.cc:1403] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.410s	user 0.029s	sys 0.029s
I20260709 14:51:03.859251  2807 tablet_bootstrap.cc:654] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.866636  2805 tablet_bootstrap.cc:654] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.879405  2760 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 14:51:03.883332  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:03.889765  3236 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:03.900871  3198 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:03.902591  2811 tablet_bootstrap.cc:654] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:03.927911  2806 raft_consensus.cc:359] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.922591  2816 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:03.929430  2816 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.930282  2806 raft_consensus.cc:385] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:03.930759  2806 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:03.931396  3199 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:03.931630  2806 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.932969  2816 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:03.934248  2806 ts_tablet_manager.cc:1434] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 14:51:03.950778  2806 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:03.949529  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:03.955948  2652 leader_election.cc:343] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.956343  3077 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:03.957010  3077 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:03.958966  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:03.968482  2651 leader_election.cc:343] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.969177  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:03.970896  3199 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:03.985608  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:03.986400  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:03.986485  3077 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:03.988615  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:03.991888  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:03.993994  2577 leader_election.cc:343] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:03.998217  2816 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:03.997831  3199 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.000211  3199 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:03.998051  3077 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.006307  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.006974  3077 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.007408  2533 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:04.007665  3077 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.008924  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:04.010231  3235 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.010604  3235 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.010968  3235 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.025102  3198 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.037489  3199 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:04.036749  2433 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.038726  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.044229  2433 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 2, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:04.051277  3235 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.056479  2811 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:04.056888  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:04.057577  2533 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:04.059775  2652 leader_election.cc:343] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.057302  2811 ts_tablet_manager.cc:1403] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.245s	user 0.022s	sys 0.017s
I20260709 14:51:04.068815  2811 raft_consensus.cc:359] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.071344  2811 raft_consensus.cc:385] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.076754  2811 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.077801  2811 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.078414  2807 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:04.078204  3077 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.079203  2807 ts_tablet_manager.cc:1403] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.248s	user 0.014s	sys 0.008s
I20260709 14:51:04.080565  2811 ts_tablet_manager.cc:1434] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 14:51:04.081800  2811 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:04.083196  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:04.083657  2806 tablet_bootstrap.cc:654] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.083580  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:04.083896  2684 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.085126  2533 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
W20260709 14:51:04.085280  2577 leader_election.cc:343] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.086498  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:04.087611  3199 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:04.088070  3077 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:04.088160  3199 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:04.089265  3199 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.094997  2805 tablet_bootstrap.cc:492] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:04.095652  2805 ts_tablet_manager.cc:1403] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.265s	user 0.026s	sys 0.017s
I20260709 14:51:04.096097  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:04.097816  2652 leader_election.cc:343] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.101274  2805 raft_consensus.cc:359] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.101298  3235 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:04.102787  2805 raft_consensus.cc:385] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.103941  2805 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.104655  2805 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.108306  2684 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:04.111685  2805 ts_tablet_manager.cc:1434] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 14:51:04.112977  2805 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:04.114254  3039 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.114781  3039 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.118439  3039 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.126053  2807 raft_consensus.cc:359] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.127449  2807 raft_consensus.cc:385] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.127882  2807 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.133541  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:04.148674  2807 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.147739  3077 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:04.153200  3077 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:04.154222  3077 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.159758  2807 ts_tablet_manager.cc:1434] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 14:51:04.155642  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:04.176595  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.179160  2759 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.183357  2861 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.194336  3039 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.195441  3039 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.199442  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:04.201323  2726 leader_election.cc:343] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.203419  2811 tablet_bootstrap.cc:654] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.207603  2684 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:04.207984  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.208695  2533 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:04.214300  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:04.231662  2807 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:04.235077  2806 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:04.235656  2806 ts_tablet_manager.cc:1403] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.285s	user 0.023s	sys 0.020s
I20260709 14:51:04.236460  2805 tablet_bootstrap.cc:654] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.236155  2430 catalog_manager.cc:5697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.234673  2861 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.240193  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:04.241178  3184 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.241561  3184 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.241904  3184 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.241987  2861 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.242623  2861 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.245448  2861 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.257690  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.262344  2806 raft_consensus.cc:359] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.263286  2806 raft_consensus.cc:385] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.263592  2806 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.263731  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:04.264333  2806 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.247167  3039 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:04.279793  2811 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:04.280515  2811 ts_tablet_manager.cc:1403] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.199s	user 0.008s	sys 0.005s
I20260709 14:51:04.283008  2806 ts_tablet_manager.cc:1434] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 14:51:04.284466  2806 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:04.304139  2431 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.310619  2805 tablet_bootstrap.cc:492] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:04.311398  2805 ts_tablet_manager.cc:1403] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.199s	user 0.022s	sys 0.018s
I20260709 14:51:04.324319  3153 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.348030  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:04.350261  2861 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.346271  3184 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.367363  2536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:04.371090  2536 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.373627  3200 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.373818  2811 raft_consensus.cc:359] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.376595  2811 raft_consensus.cc:385] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.377087  2811 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.369858  2807 tablet_bootstrap.cc:654] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.379220  2811 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.379873  2805 raft_consensus.cc:359] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.380609  2805 raft_consensus.cc:385] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.380901  2805 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.381600  2805 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.391072  2805 ts_tablet_manager.cc:1434] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 14:51:04.392411  2805 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:04.397255  2433 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.401422  2811 ts_tablet_manager.cc:1434] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.002s
I20260709 14:51:04.402699  2811 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:04.418494  2806 tablet_bootstrap.cc:654] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
W20260709 14:51:04.415277  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 201065us
I20260709 14:51:04.426729  3192 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.427561  2811 tablet_bootstrap.cc:654] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.427515  3192 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.432293  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:04.433825  3192 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:04.435336  2502 leader_election.cc:343] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.440479  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:04.443867  2500 leader_election.cc:343] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.444057  2805 tablet_bootstrap.cc:654] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.437788  2430 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:04.444409  2500 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:04.448493  3198 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.450729  2430 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:04.451718  3199 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:04.456250  3184 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:04.474803  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:04.479650  2726 leader_election.cc:343] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.493554  2811 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:04.494299  2811 ts_tablet_manager.cc:1403] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.011s
I20260709 14:51:04.498925  2811 raft_consensus.cc:359] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.499805  2811 raft_consensus.cc:385] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.500123  2811 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.500950  2811 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.501284  3200 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.503619  2536 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:04.505170  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:04.506763  3153 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:04.507634  3153 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:04.516654  2811 ts_tablet_manager.cc:1434] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.004s
I20260709 14:51:04.523382  2811 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:04.523538  3235 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.532594  3235 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.533123  3153 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.538321  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.539942  2807 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:04.539676  2537 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:04.540766  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:04.541211  3235 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:04.544823  2651 leader_election.cc:343] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.545909  2807 ts_tablet_manager.cc:1403] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.314s	user 0.030s	sys 0.009s
W20260709 14:51:04.549831  2652 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:04.552115  2811 tablet_bootstrap.cc:654] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.555449  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:04.559108  3236 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.559549  3236 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.559907  3236 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.562014  2807 raft_consensus.cc:359] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.563206  2807 raft_consensus.cc:385] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.563512  2807 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.569145  2807 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.579021  2807 ts_tablet_manager.cc:1434] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 14:51:04.581702  2806 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:04.595160  2806 ts_tablet_manager.cc:1403] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.311s	user 0.025s	sys 0.004s
I20260709 14:51:04.604346  2807 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:04.586300  3236 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.614074  2806 raft_consensus.cc:359] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.617303  2806 raft_consensus.cc:385] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.616283  3236 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:04.617730  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:04.615433  2805 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:04.619278  2806 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
W20260709 14:51:04.622722  2651 leader_election.cc:343] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.623061  2805 ts_tablet_manager.cc:1403] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.231s	user 0.026s	sys 0.012s
I20260709 14:51:04.624033  2806 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.625156  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:04.625901  2533 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.637583  2811 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:04.638271  2811 ts_tablet_manager.cc:1403] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.012s
I20260709 14:51:04.638347  2806 ts_tablet_manager.cc:1434] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 14:51:04.639761  2806 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:04.641487  2811 raft_consensus.cc:359] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.642279  2811 raft_consensus.cc:385] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.642560  2811 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.644599  2805 raft_consensus.cc:359] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.645395  2805 raft_consensus.cc:385] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.645689  2805 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.646505  2805 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.646569  2433 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.648767  2805 ts_tablet_manager.cc:1434] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 14:51:04.657935  3140 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.667567  3140 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.670413  3140 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:04.671941  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:04.675946  2651 leader_election.cc:343] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.649497  2811 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.678378  2811 ts_tablet_manager.cc:1434] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 14:51:04.681636  2533 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:04.688293  2606 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.656885  2807 tablet_bootstrap.cc:654] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.692147  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:04.692668  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.693413  2537 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:04.694710  3235 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:04.695298  3235 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:04.696975  3235 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.699280  2759 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.683202  2811 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Bootstrap starting.
I20260709 14:51:04.712024  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:04.713207  3199 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.715502  3199 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.723268  3199 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.649801  2805 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:04.740923  3140 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.744268  3184 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.744510  3140 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.744824  3184 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:04.747969  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.748410  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:04.749635  2652 leader_election.cc:343] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:51:04.750197  2651 leader_election.cc:343] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.750654  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:04.750723  3184 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.751469  2816 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.754105  2806 tablet_bootstrap.cc:654] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.756916  3229 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.757771  3077 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.757803  3140 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:04.757767  3229 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.763566  2861 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.764385  2861 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.774134  2861 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.780730  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:04.781581  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:04.784704  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:04.785123  3229 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.783905  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:04.780778  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.791332  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:04.789180  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:04.797415  2805 tablet_bootstrap.cc:654] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:04.804816  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:04.794359  3192 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:04.805657  3199 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.805922  3192 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.799033  3187 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.805688  2684 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:51:04.817512  3187 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:04.817852  2500 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:04.820003  3229 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:04.820101  3199 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:04.820436  3192 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:04.820533  3229 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:04.821049  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.821287  3229 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.826412  2811 tablet_bootstrap.cc:654] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Neither blocks nor log segments found. Creating new log.
W20260709 14:51:04.829053  2503 leader_election.cc:343] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.833003  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:04.834050  2807 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:04.834656  2807 ts_tablet_manager.cc:1403] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.232s	user 0.018s	sys 0.002s
W20260709 14:51:04.841591  2651 leader_election.cc:343] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.846654  2807 raft_consensus.cc:359] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.847671  2807 raft_consensus.cc:385] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.848027  2807 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.848235  2433 catalog_manager.cc:5697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:04.856516  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:04.857515  2533 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.864113  2807 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:04.851322  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:04.852277  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:04.877067  2807 ts_tablet_manager.cc:1434] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
W20260709 14:51:04.882133  2503 leader_election.cc:343] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.882910  3153 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:04.885752  3153 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:04.886240  3153 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:04.896492  2502 leader_election.cc:343] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.897104  2502 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:04.899380  3192 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:04.951392  2807 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:04.965835  3229 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:04.971598  2608 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.973502  2533 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:04.974442  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:04.976326  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:04.974668  3229 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:04.978478  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:04.984261  2503 leader_election.cc:343] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:04.984666  2806 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:04.985029  2805 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:04.985213  2806 ts_tablet_manager.cc:1403] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.346s	user 0.014s	sys 0.005s
I20260709 14:51:04.985630  2805 ts_tablet_manager.cc:1403] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.336s	user 0.015s	sys 0.006s
I20260709 14:51:04.975195  2684 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:04.994582  2811 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: No bootstrap required, opened a new log
I20260709 14:51:04.995253  2811 ts_tablet_manager.cc:1403] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent bootstrapping tablet: real 0.312s	user 0.016s	sys 0.013s
I20260709 14:51:04.996261  2806 raft_consensus.cc:359] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:04.997056  2806 raft_consensus.cc:385] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:04.997328  2806 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:04.999409  2806 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.004084  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:05.005390  3236 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:05.013846  3236 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:05.017586  2811 raft_consensus.cc:359] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.018596  2811 raft_consensus.cc:385] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.018975  2811 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.019354  2684 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:05.020521  2500 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:05.019905  2811 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.030141  2811 ts_tablet_manager.cc:1434] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 14:51:05.013733  3153 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.032611  2759 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:05.032693  2610 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.033514  3153 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:05.036521  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:05.036800  3187 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:05.037976  3187 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:05.038345  3187 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:05.039198  3187 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.040674  3187 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:05.046887  2806 ts_tablet_manager.cc:1434] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.002s
I20260709 14:51:05.049039  2806 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:05.055187  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:05.057580  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:05.058794  3184 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:05.015790  3236 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.055770  2806 tablet_bootstrap.cc:654] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:05.041472  3198 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:05.060290  3198 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:05.061295  3198 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:05.066759  2807 tablet_bootstrap.cc:654] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:05.069216  3153 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 14:51:05.064831  2805 raft_consensus.cc:359] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.073613  2805 raft_consensus.cc:385] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.074050  2805 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.078259  2805 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.083303  2806 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: No bootstrap required, opened a new log
I20260709 14:51:05.085268  2806 ts_tablet_manager.cc:1403] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.002s
I20260709 14:51:05.087679  2861 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.088277  2861 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.088867  2805 ts_tablet_manager.cc:1434] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.103s	user 0.010s	sys 0.000s
I20260709 14:51:05.092736  2806 raft_consensus.cc:359] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.093897  2806 raft_consensus.cc:385] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.094277  2806 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.099594  3153 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.100317  3153 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.100600  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:05.103426  3153 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:05.104574  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:05.108738  2806 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.111187  2861 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:05.100883  2805 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:05.114703  2806 ts_tablet_manager.cc:1434] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 14:51:05.115324  3153 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:05.115831  3198 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.116369  3198 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.119221  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:05.119938  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:05.119441  3198 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:05.121568  2502 leader_election.cc:343] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:05.126004  2807 tablet_bootstrap.cc:492] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:05.125720  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:05.126744  2807 ts_tablet_manager.cc:1403] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.180s	user 0.012s	sys 0.006s
I20260709 14:51:05.129956  2807 raft_consensus.cc:359] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.137065  2534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:05.138528  2807 raft_consensus.cc:385] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.138953  2807 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.139724  2534 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:05.141181  2805 tablet_bootstrap.cc:654] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:05.139734  2807 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.159752  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:05.160557  2759 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.170243  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:05.200145  2503 leader_election.cc:343] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:05.219445  2503 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:05.229763  3198 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:05.241096  2805 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:05.241750  2805 ts_tablet_manager.cc:1403] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.141s	user 0.023s	sys 0.000s
I20260709 14:51:05.243140  2807 ts_tablet_manager.cc:1434] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.116s	user 0.015s	sys 0.011s
I20260709 14:51:05.246718  2807 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:05.245837  2435 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:05.274012  3184 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.285336  3184 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.290977  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:05.291735  2684 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:05.250885  2861 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:05.294327  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:05.297101  3258 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:05.297504  3258 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:05.297816  3258 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.298965  3184 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:05.302881  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:05.303773  2609 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:05.307542  3190 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.308172  3190 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.313042  3190 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:05.313601  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:05.315148  2502 leader_election.cc:343] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:05.325273  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:05.328364  2805 raft_consensus.cc:359] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.328994  2684 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 0.
I20260709 14:51:05.341897  2500 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:05.342662  3153 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:05.342995  3229 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:05.325727  2430 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:05.343506  3229 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:05.343921  3229 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.345187  3236 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:05.352437  2807 tablet_bootstrap.cc:654] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:05.355453  3229 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.358475  3235 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:05.360188  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:05.361562  2684 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.329245  2805 raft_consensus.cc:385] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.336481  3258 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.369042  3258 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:05.369756  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:05.373504  2608 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.371677  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:05.379383  2682 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:05.384245  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:05.386159  2502 leader_election.cc:343] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:05.371095  2805 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.410667  2805 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.423808  2684 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:05.438095  2682 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:05.457124  2500 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:05.459686  2805 ts_tablet_manager.cc:1434] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.218s	user 0.013s	sys 0.006s
I20260709 14:51:05.392675  3229 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:05.466444  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:05.465339  3091 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:05.494771  2807 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:05.496821  2807 ts_tablet_manager.cc:1403] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.250s	user 0.036s	sys 0.025s
W20260709 14:51:05.513000  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 49700.9us
I20260709 14:51:05.495911  3290 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:05.517133  3290 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:05.520239  2608 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:05.522626  2807 raft_consensus.cc:359] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.523561  2807 raft_consensus.cc:385] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:05.523859  2807 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:05.524586  2807 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.548615  2805 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:05.550710  2807 ts_tablet_manager.cc:1434] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.002s
I20260709 14:51:05.512992  3091 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:05.561764  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 94510.3us
I20260709 14:51:05.555073  3184 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:05.572659  3184 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:05.617317  3140 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:05.518471  3290 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:05.662344  2537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:05.663475  2537 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:05.669896  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:05.672513  2805 tablet_bootstrap.cc:654] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:05.673081  2611 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:05.675377  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:51:05.685737  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 66178us
I20260709 14:51:05.639657  3184 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:05.698724  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 118277us
I20260709 14:51:05.692219  3258 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 14:51:05.740352  2651 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:05.725240  3258 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:05.663641  3091 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:05.769309  2807 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:05.764569  3258 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:51:05.840411  2503 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:05.909614  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 152690us
I20260709 14:51:05.972317  2805 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:05.973313  2805 ts_tablet_manager.cc:1403] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.425s	user 0.018s	sys 0.052s
W20260709 14:51:06.004067  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 236299us
W20260709 14:51:06.008994  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 246052us
I20260709 14:51:06.114040  2807 tablet_bootstrap.cc:654] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:06.115243  3258 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:06.118316  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 354952us
W20260709 14:51:06.175568  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86599.4us
W20260709 14:51:06.200809  2503 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107566us
W20260709 14:51:06.215761  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111188us
W20260709 14:51:06.219226  2502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 130308us
W20260709 14:51:06.226449  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 126968us
I20260709 14:51:06.232080  2805 raft_consensus.cc:359] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:06.233309  2805 raft_consensus.cc:385] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:06.233682  2805 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
W20260709 14:51:06.259130  2500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69953.7us
I20260709 14:51:06.234579  2805 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:06.302126  2805 ts_tablet_manager.cc:1434] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.328s	user 0.010s	sys 0.001s
I20260709 14:51:06.338014  3258 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:06.300889  3300 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 14:51:06.346235  2807 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:06.350600  2807 ts_tablet_manager.cc:1403] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.596s	user 0.019s	sys 0.008s
I20260709 14:51:06.362673  2431 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:51:06.365939  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:06.398231  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:06.399132  2535 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.406335  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:06.407552  2608 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.407660  3184 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:06.408330  3184 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:06.411592  2502 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 -> Peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Couldn't send request to peer 4f1347e243fd46a5bb7cee5b2e23b665. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:06.417101  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:06.439975  2807 raft_consensus.cc:359] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.440760  2807 raft_consensus.cc:385] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:06.441042  2807 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:06.445463  2807 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.462630  2532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:06.448212  3153 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 14:51:06.448482  3184 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:06.471382  3315 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:06.472164  3315 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.472574  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:06.479099  2608 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:06.481256  2807 ts_tablet_manager.cc:1434] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.130s	user 0.007s	sys 0.000s
I20260709 14:51:06.483132  3319 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:06.488214  2535 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:06.488771  2805 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:06.476500  3184 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:06.504570  2807 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:06.500672  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:06.496366  3315 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:06.521695  3304 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:06.522696  3304 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.514366  3184 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:06.530974  3184 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:06.532321  3184 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:06.549412  2755 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.550611  2430 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:06.559429  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:06.571482  3258 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:06.577450  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:06.578250  2610 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:06.562856  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:06.579286  2612 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 0.
I20260709 14:51:06.580034  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:06.580294  3304 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:06.564947  2651 leader_election.cc:343] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:06.585136  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:06.603897  3305 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:06.604417  3305 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:06.604799  3305 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.605962  2805 tablet_bootstrap.cc:654] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:06.608769  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:06.581907  3140 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:06.620776  3140 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:06.638612  2758 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:06.653381  3140 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:06.706601  2805 tablet_bootstrap.cc:492] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:06.707317  2805 ts_tablet_manager.cc:1403] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.219s	user 0.014s	sys 0.014s
I20260709 14:51:06.776022  2807 tablet_bootstrap.cc:654] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:06.812165  2805 raft_consensus.cc:359] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.812994  2805 raft_consensus.cc:385] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:06.813274  2805 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:06.814910  2805 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.823484  2805 ts_tablet_manager.cc:1434] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.000s
I20260709 14:51:06.811430  3305 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.847769  2805 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:06.860985  3140 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.867820  2805 tablet_bootstrap.cc:654] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:06.873890  3305 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:06.880389  3184 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:06.882122  3184 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.883589  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:06.885030  3184 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:06.890111  2807 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:06.896510  2807 ts_tablet_manager.cc:1403] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.411s	user 0.019s	sys 0.030s
I20260709 14:51:06.885727  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:06.885665  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:06.905949  2680 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:06.886126  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:06.902575  2604 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.908071  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:06.888263  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:06.911181  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:06.885293  2757 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:06.888316  3153 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:06.918951  3319 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:06.919564  3319 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:06.919258  3153 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.921800  3319 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:06.922093  3153 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:06.923332  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:06.924069  2754 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:06.923985  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:06.923372  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:06.924201  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:06.926115  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:06.927696  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:06.928175  2503 leader_election.cc:343] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:51:06.929564  2502 leader_election.cc:343] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:51:06.930001  2502 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:06.931154  3328 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:06.933748  3184 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:06.936177  3184 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:06.941577  3184 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:06.945155  2757 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:06.927695  2435 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:06.949599  2807 raft_consensus.cc:359] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:06.886658  3140 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:06.975595  2807 raft_consensus.cc:385] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:06.976083  2807 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:06.976974  2807 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:06.978794  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:06.979796  3140 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:06.980340  3140 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:06.981693  3303 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:51:06.981526  3140 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.984838  2807 ts_tablet_manager.cc:1434] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.004s
I20260709 14:51:06.980295  3184 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:06.986882  2807 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Bootstrap starting.
I20260709 14:51:06.989190  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:06.989920  2684 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:06.990322  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:06.996500  2807 tablet_bootstrap.cc:654] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:06.998821  2805 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:07.000936  2805 ts_tablet_manager.cc:1403] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.154s	user 0.036s	sys 0.038s
I20260709 14:51:07.004415  2805 raft_consensus.cc:359] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:07.006609  2805 raft_consensus.cc:385] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:07.007014  2805 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:07.007781  2805 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:07.008673  2684 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:07.010208  2805 ts_tablet_manager.cc:1434] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:51:07.011468  2805 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Bootstrap starting.
I20260709 14:51:06.954563  2604 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:06.963927  2608 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:07.015745  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:07.020551  2805 tablet_bootstrap.cc:654] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Neither blocks nor log segments found. Creating new log.
I20260709 14:51:07.021238  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:07.025444  3153 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:07.030339  3153 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:07.031689  3153 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:07.015777  3184 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:07.022720  3315 raft_consensus.cc:2764] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 14:51:07.058189  3184 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:07.069314  3303 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:07.143977  2807 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: No bootstrap required, opened a new log
I20260709 14:51:07.144857  2807 ts_tablet_manager.cc:1403] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent bootstrapping tablet: real 0.158s	user 0.038s	sys 0.015s
I20260709 14:51:07.152384  2612 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:07.158245  2757 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:07.163071  2805 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: No bootstrap required, opened a new log
I20260709 14:51:07.163712  2805 ts_tablet_manager.cc:1403] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent bootstrapping tablet: real 0.152s	user 0.007s	sys 0.006s
I20260709 14:51:07.166747  2805 raft_consensus.cc:359] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.167695  2805 raft_consensus.cc:385] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:07.167994  2805 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Initialized, Role: FOLLOWER
I20260709 14:51:07.168833  2805 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.171342  2805 ts_tablet_manager.cc:1434] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 14:51:07.180286  2435 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:07.183153  2807 raft_consensus.cc:359] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.184065  2807 raft_consensus.cc:385] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:51:07.190477  2807 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Initialized, Role: FOLLOWER
I20260709 14:51:07.191385  2807 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.195425  2430 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:07.200140  2807 ts_tablet_manager.cc:1434] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 14:51:07.230942  3314 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:07.231573  3314 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:07.236464  2533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:07.239892  3303 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:07.240576  3303 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:07.258560  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:07.364630  3303 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:07.373553  3314 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:07.385447  3290 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:07.437983  3362 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:07.438884  3362 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.442188  3362 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:07.388130  3290 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.449146  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:07.455546  2535 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:07.463486  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:07.468181  3296 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:51:07.468999  3296 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:07.469471  3296 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:07.465483  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:07.473932  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 214978us
I20260709 14:51:07.477893  3290 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:07.479856  2754 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 0.
I20260709 14:51:07.481878  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:07.496024  2483 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:51:07.507521  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:07.533063  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:07.537616  3296 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.539640  2604 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 1: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:07.540750  3296 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:07.546532  3362 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:07.547784  2535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:07.550081  2531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:07.550779  2531 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 0.
I20260709 14:51:07.553131  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:07.553580  3326 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:07.553999  2754 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:07.559083  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:07.560278  3153 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:07.561600  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:07.571404  2531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:07.572268  2531 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:51:07.573028  2680 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:07.574682  2754 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:07.575328  3184 raft_consensus.cc:2764] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 14:51:07.575567  3290 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:07.575994  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:07.577550  3362 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:51:07.578110  3362 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:07.579121  3362 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:07.602007  3365 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:07.603811  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:07.612728  2531 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:07.644125  2612 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:07.684336  2612 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:07.697065  3184 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:08.167055  3362 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:51:08.192265  3314 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:51:09.015897   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 14:51:09.044649  2724 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 183 similar messages]
W20260709 14:51:09.067701  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.145637  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.154062  2576 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.159257  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71332.2us
W20260709 14:51:09.162575  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84387.3us
W20260709 14:51:09.169533  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 82446.5us
W20260709 14:51:09.197074  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.209535  2724 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.212652  2576 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.228132  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.229046  2724 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.233883   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 14:51:09.236289  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54887.5us
W20260709 14:51:09.238185  2650 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.245620  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.246377  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.244222   431 tablet_replica.cc:333] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.247851   431 raft_consensus.cc:2243] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.248591   431 raft_consensus.cc:2272] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.251629   431 tablet_replica.cc:333] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
W20260709 14:51:09.252184  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.253521   431 raft_consensus.cc:2243] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.254246   431 raft_consensus.cc:2272] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.257232   431 tablet_replica.cc:333] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.258296   431 raft_consensus.cc:2243] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.258972   431 raft_consensus.cc:2272] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.260012  2650 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.260732  2650 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.261325  2650 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.267403   431 tablet_replica.cc:333] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.268654   431 raft_consensus.cc:2243] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 14:51:09.269194  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.269322   431 raft_consensus.cc:2272] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.274946   431 tablet_replica.cc:333] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.276044   431 raft_consensus.cc:2243] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.276875   431 raft_consensus.cc:2272] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.282383   431 tablet_replica.cc:333] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.283527   431 raft_consensus.cc:2243] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.284196   431 raft_consensus.cc:2272] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.287135   431 tablet_replica.cc:333] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.288261   431 raft_consensus.cc:2243] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.289479   431 raft_consensus.cc:2272] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.292282  2724 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.292483   431 tablet_replica.cc:333] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.293761   431 raft_consensus.cc:2243] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.295099   431 raft_consensus.cc:2272] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.296324  2650 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.298053   431 tablet_replica.cc:333] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.299252   431 raft_consensus.cc:2243] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.300274   431 raft_consensus.cc:2272] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.303970   431 tablet_replica.cc:333] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.305065   431 raft_consensus.cc:2243] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:51:09.306314  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.306453   431 raft_consensus.cc:2272] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.310024   431 tablet_replica.cc:333] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.311419   431 raft_consensus.cc:2243] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.312635   431 raft_consensus.cc:2272] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.315884   431 tablet_replica.cc:333] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.316943   431 raft_consensus.cc:2243] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.317678   431 raft_consensus.cc:2272] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.321918  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.321981   431 tablet_replica.cc:333] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.323428   431 raft_consensus.cc:2243] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.324118   431 raft_consensus.cc:2272] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.327167   431 tablet_replica.cc:333] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.328181   431 raft_consensus.cc:2243] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.329488   431 raft_consensus.cc:2272] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.332662   431 tablet_replica.cc:333] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.333688   431 raft_consensus.cc:2243] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.334956   431 raft_consensus.cc:2272] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.338158   431 tablet_replica.cc:333] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.340627   431 raft_consensus.cc:2243] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:51:09.341292  2576 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.341976   431 raft_consensus.cc:2272] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.345016   431 tablet_replica.cc:333] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.346127   431 raft_consensus.cc:2243] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.346805   431 raft_consensus.cc:2272] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.349889   431 tablet_replica.cc:333] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.350977   431 raft_consensus.cc:2243] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.351624   431 raft_consensus.cc:2272] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.354743   431 tablet_replica.cc:333] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.355764   431 raft_consensus.cc:2243] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.356346   431 raft_consensus.cc:2272] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.359288   431 tablet_replica.cc:333] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
W20260709 14:51:09.359661  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.360311   431 raft_consensus.cc:2243] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.361614   431 raft_consensus.cc:2272] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.363873  2724 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.364882   431 tablet_replica.cc:333] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.365995   431 raft_consensus.cc:2243] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.366658   431 raft_consensus.cc:2272] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.369884   431 tablet_replica.cc:333] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.370864   431 raft_consensus.cc:2243] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.371487   431 raft_consensus.cc:2272] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.374446   431 tablet_replica.cc:333] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.375443   431 raft_consensus.cc:2243] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.376317   431 raft_consensus.cc:2272] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.379684   431 tablet_replica.cc:333] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.380666   431 raft_consensus.cc:2243] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.381349   431 raft_consensus.cc:2272] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.384476   431 tablet_replica.cc:333] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.385494   431 raft_consensus.cc:2243] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.386706   431 raft_consensus.cc:2272] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.389964   431 tablet_replica.cc:333] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.390902   431 raft_consensus.cc:2243] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.392050   431 raft_consensus.cc:2272] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.395125   431 tablet_replica.cc:333] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.396049   431 raft_consensus.cc:2243] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.396663   431 raft_consensus.cc:2272] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.399673   431 tablet_replica.cc:333] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.400580   431 raft_consensus.cc:2243] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.401193   431 raft_consensus.cc:2272] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.404429   431 tablet_replica.cc:333] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.405321   431 raft_consensus.cc:2243] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.405920   431 raft_consensus.cc:2272] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.408990   431 tablet_replica.cc:333] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.410032   431 raft_consensus.cc:2243] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.410682   431 raft_consensus.cc:2272] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.413748   431 tablet_replica.cc:333] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.414738   431 raft_consensus.cc:2243] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.415385   431 raft_consensus.cc:2272] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.418484   431 tablet_replica.cc:333] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.419435   431 raft_consensus.cc:2243] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.420580   431 raft_consensus.cc:2272] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.423668   431 tablet_replica.cc:333] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.424654   431 raft_consensus.cc:2243] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:51:09.428045  2576 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.428356   431 raft_consensus.cc:2272] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.431103   431 tablet_replica.cc:333] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.431771   431 raft_consensus.cc:2243] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.432621   431 raft_consensus.cc:2272] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.435182   431 tablet_replica.cc:333] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.446204   431 raft_consensus.cc:2243] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.446947   431 raft_consensus.cc:2272] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.449960   431 tablet_replica.cc:333] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.450922   431 raft_consensus.cc:2243] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.451560   431 raft_consensus.cc:2272] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.454582   431 tablet_replica.cc:333] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.455449   431 raft_consensus.cc:2243] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.456080   431 raft_consensus.cc:2272] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.459290   431 tablet_replica.cc:333] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.460167   431 raft_consensus.cc:2243] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.461305   431 raft_consensus.cc:2272] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.462818  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.464422   431 tablet_replica.cc:333] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.465492   431 raft_consensus.cc:2243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.466149   431 raft_consensus.cc:2272] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.469406   431 tablet_replica.cc:333] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.470301   431 raft_consensus.cc:2243] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.471487   431 raft_consensus.cc:2272] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.474738   431 tablet_replica.cc:333] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.475653   431 raft_consensus.cc:2243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.476876   431 raft_consensus.cc:2272] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.480065   431 tablet_replica.cc:333] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.480870   431 raft_consensus.cc:2243] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:51:09.481986   431 raft_consensus.cc:2272] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.484910   431 tablet_replica.cc:333] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.485682   431 raft_consensus.cc:2243] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.486238   431 raft_consensus.cc:2272] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:51:09.488865   431 tablet_replica.cc:333] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.489576   431 raft_consensus.cc:2243] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.490113   431 raft_consensus.cc:2272] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.490181  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.499614   431 tablet_replica.cc:333] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: stopping tablet replica
I20260709 14:51:09.500491   431 raft_consensus.cc:2243] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:51:09.501107   431 raft_consensus.cc:2272] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:51:09.590930  2650 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:09.593303  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:09.686527   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 14:51:09.816370   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:51:09.837164  3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:51:09.840222  3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:51:09.843271  3400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:51:09.896970   431 server_base.cc:1061] running on GCE node
I20260709 14:51:09.941079   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:51:09.941618   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:51:09.941984   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608669941958 us; error 0 us; skew 500 ppm
I20260709 14:51:09.949426   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:51:10.089706   431 webserver.cc:533] Webserver started at http://127.0.107.193:33325/ using document root <none> and password file <none>
I20260709 14:51:10.091339   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:51:10.091840   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 14:51:10.120007   431 fs_manager.cc:714] Time spent opening directory manager: real 0.013s	user 0.009s	sys 0.001s
I20260709 14:51:10.136942  3408 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:51:10.138125   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.000s	sys 0.003s
I20260709 14:51:10.138556   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "c1b0e692e4294ee3b8694bbada174ba1"
format_stamp: "Formatted at 2026-07-09 14:50:51 on dist-test-slave-wg8n"
I20260709 14:51:10.139499   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:51:10.200765   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:51:10.203339   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:51:10.216732   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:51:10.232050  3415 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 14:51:10.346911  3383 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.348162  3383 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.353683  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.357215  3383 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:10.362797  2650 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.363539  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.364815  3383 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.383155  3296 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.384053  3296 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.386504  3296 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:10.392290  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.394441  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.396270  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.399185  3389 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.402788  3296 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.403510  3296 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.409806  3385 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.410514  3385 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.414937  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.418689  3385 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:10.425277  2724 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.426007  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.429067  3385 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.430157  3389 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.430418  3404 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.431017  3389 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.433499  3389 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.444439  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.431022  3404 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.448889  3404 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.450884  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.454134  3378 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.454797  3378 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.456383  3392 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.457298  3378 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.457134  3392 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.459954  3392 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.460948  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.462760  3389 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.463521  3389 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.466229  3389 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:10.468035  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.469485  3296 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:10.471189  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.479316  3387 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:10.480080  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.480098  3387 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.480684  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.483629  3387 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.485318  3389 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.485323  3387 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.486264  3387 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.488788  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.494134  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.498502  3387 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:10.499388  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.511561  3389 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.515125  3389 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.518085  3389 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.519364  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.520532  3404 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:10.520789  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.534219  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.531212  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.532204  3404 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.543823  3404 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.546381  3404 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.546648  3385 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.547528  3385 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.548393  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.550254  3385 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.551362  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.552105  2680 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:10.552628  3383 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.554067  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.535600  3387 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.555123  3387 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.556653  3290 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.557072  3290 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.557410  3387 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.557453  3290 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:10.559096  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.559623  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.560499  3389 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:10.561581  2576 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.562064  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.562809  3389 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.563098  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:10.530072  2724 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.565239  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.567090  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.567343  3387 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.567875  2612 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:51:10.573077  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52333.1us
I20260709 14:51:10.578555  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:10.580049  3385 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.580497  3385 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.580914  3385 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.589812  3378 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.590624  3378 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.593147  3378 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:10.601770  3389 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.602784  3389 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.605840  3389 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:10.609015  2576 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.608873  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.609560  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.609860  2680 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:10.610507  3389 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.617288  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:10.618685  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.623405  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.624396  2679 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:10.633426  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.634756  3389 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.635281  3389 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.635846  3389 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.649721  3389 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.626987  3385 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.654098  3314 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.654762  3314 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.656314  3385 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.657121  3314 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:10.659096  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:10.660089  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.660516  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.629978  3418 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.663033  3418 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.663494  3418 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.664752  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:10.667024  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.670071  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.671012  3314 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.674132  3290 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:10.677098  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87775.6us
I20260709 14:51:10.677412  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:10.678251  2679 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.695057  3387 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.709071   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 14:51:10.709614   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.484s	user 0.006s	sys 0.011s
I20260709 14:51:10.710062   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
W20260709 14:51:10.711671  2650 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.712307  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.712809  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:10.713537  2604 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.714059  3314 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.714495  3290 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.714727  3314 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.717142  3314 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:10.717271  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.718437  3404 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.719574  3404 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.719712  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.720151  3404 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.722604  3404 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.724267  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.727500  3415 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:10.736052  3404 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.736816  3404 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:10.738462  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.749184  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:10.758945  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.742758  2604 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:10.743005  3378 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.761137  3389 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:10.760453  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:10.761852  2680 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.761325  3378 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.763969  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.767267  2612 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:10.768657  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.769681  3392 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.770118  3392 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.770542  3392 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.770282  3378 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.774443  3383 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.775142  3383 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.778131  3383 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.780041  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.780818  2754 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:10.783198  3378 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.784224  3403 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.783998  3378 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.784924  3403 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.782513  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.789363  3378 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.761925  3385 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:10.792057  3385 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:10.793268  3383 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.793812  3383 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.794206  3383 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:10.765391  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80768us
I20260709 14:51:10.766537  3415 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:10.789763  3403 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:10.795967  3415 tablet_bootstrap.cc:492] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:10.793228  3385 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.796047  2680 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:10.804097  3415 ts_tablet_manager.cc:1403] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.012s
I20260709 14:51:10.808933  3415 raft_consensus.cc:359] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.809778  3415 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:10.813702  3415 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.764734  3404 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.820762  3403 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.821874  3403 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.832438  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.833333  2604 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:10.834630  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:10.835309  2724 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.835867  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.835959  3403 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.837198  3415 ts_tablet_manager.cc:1434] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.007s
I20260709 14:51:10.837215  3290 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.838147  3403 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.838483  3415 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:10.838785  3403 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.839347  3403 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.842231  3389 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.842970  3389 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.850875  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:10.851976  3290 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:10.853214  3290 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:10.854348  3290 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.861304  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:10.862185  2604 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:10.865599  3423 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:10.869184  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86892us
I20260709 14:51:10.871291  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:10.872189  2754 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:10.874506  3389 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:10.876109  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.880613  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.881494  2679 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:10.884413  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.885464  3403 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.886859  3314 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:10.895071  3415 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:10.897835  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:10.898859  2612 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.899706  3415 tablet_bootstrap.cc:492] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:10.889801  3296 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.900766  3415 ts_tablet_manager.cc:1403] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.063s	user 0.021s	sys 0.000s
I20260709 14:51:10.900993  3296 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.901585  3296 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:10.904088  3415 raft_consensus.cc:359] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.914592  3415 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:10.915767  3415 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.918325  3415 ts_tablet_manager.cc:1434] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 14:51:10.919375  3415 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
W20260709 14:51:10.920847  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83971.4us
I20260709 14:51:10.922323  2612 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:10.821556  3392 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.929476  3403 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.931936  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:10.932965  3404 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:10.933542  3404 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:10.936930  3404 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:10.938031  3418 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:10.945672  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63299.8us
W20260709 14:51:10.949791  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.961750  3420 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:10.962503  3420 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:10.963330  3392 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.964798  3420 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:10.966995  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:10.967674  2612 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:10.956321  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:10.969046  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 84760.3us
W20260709 14:51:10.969987  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:10.978091  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.981029  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:10.981925  2679 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:10.988508  2435 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:10.990406  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:10.991693  3385 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:10.992204  3385 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:10.999521  2650 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.001856  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:10.866572  3423 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:10.982944  3418 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:11.005584  3426 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:11.005874  3429 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:10.992624  3385 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.011276  2612 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
W20260709 14:51:11.014685  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.001402  3403 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:11.016914  2650 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.001070  3314 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:10.995076  3383 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.008960  3378 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.023228  3378 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.022507  3314 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.025763  3378 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:11.022459  3430 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.026945  3422 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.027192  3430 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.027601  3422 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:11.020504  2650 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.029172  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.030294  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.031412  2724 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.038201  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:11.039131  2683 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.039184  3422 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:11.046324  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.047816  3422 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.048404  3422 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.049369  3422 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.050874  3383 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.052752  3423 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.020697  3403 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.044533  3296 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.055922  3430 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:11.059397  3415 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:11.059208  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:11.060024  2681 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:11.060571  3415 tablet_bootstrap.cc:492] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:11.061436  3415 ts_tablet_manager.cc:1403] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.142s	user 0.017s	sys 0.019s
I20260709 14:51:11.061475  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.061930  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.062510  3430 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.062939  3430 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.063313  3430 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.063457  3296 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:11.068372  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51372.9us
W20260709 14:51:11.068908  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.073957  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:11.075145  2604 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:11.079223  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:11.079473  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.080118  2612 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
W20260709 14:51:11.078408  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.083220  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.071614  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.091712  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.082022  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.071424  3423 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.112262  3415 raft_consensus.cc:359] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.113420  3415 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:11.115352  3415 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.101543  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:11.125484  2681 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:11.134447  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.135340  2650 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.135854  2650 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.129192  2683 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:11.143843  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:11.111040  3418 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.111251  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:11.111312  3423 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:11.156997  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 161380us
I20260709 14:51:11.154436  3415 ts_tablet_manager.cc:1434] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.093s	user 0.013s	sys 0.000s
I20260709 14:51:11.158314  3415 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:11.161926  3403 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.162189  3378 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.162714  3378 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.163039  2608 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:11.164799  3423 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.154398  3418 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.165933  3418 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:11.167533  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.157894  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.163154  3378 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.174175  3420 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.174633  3420 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.175048  3420 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.186594  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.195369  3403 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.195994  3403 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.196429  3403 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.182314  3430 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.161671  2430 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:11.222054  2650 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.235081  2681 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:11.235668  3418 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.236799  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.238701  3418 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:11.246013  2724 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.257562  3420 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.258968  3385 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.261714  3437 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.262507  3437 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:11.263639  3437 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.259014  3423 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.261881  3385 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:11.273833  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.274972  3423 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.279325  3430 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:11.280897  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:11.281039  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:11.281860  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.283082  2604 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.282733  2681 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:11.287612  2650 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.336285  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:11.336973  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:11.341050  2604 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:11.348114  2681 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:11.343032  3378 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:11.354188  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.369818  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.371301  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:11.372042  2608 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.372112  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.342819  3420 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:11.375311  3389 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.375898  3389 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:11.377007  3389 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.380869  3387 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 14:51:11.381397  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.381537  3387 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:11.384330  2608 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:11.384508  3387 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:11.388089  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 317743us
I20260709 14:51:11.389971  2433 catalog_manager.cc:5697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:11.391189  3403 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.392212  3378 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:11.406102  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:11.429580  2754 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:11.400846  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:11.407075  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:11.406271  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.434473  2604 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.432313  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.415058  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:11.406649  3403 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:11.436362  2755 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.437474  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:11.438251  2608 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:11.440022  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:51:11.444214  2612 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.405689  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:11.445783  2758 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:11.446132  2679 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.424470  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:11.444274  3439 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.447566  2682 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.447938  3439 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:11.449043  3439 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.407972  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:11.441150  3420 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.455408  3420 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:11.456599  3420 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.467329  3290 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 14:51:11.459369  2757 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.473569  2608 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:11.475095  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.477596  3423 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.476025  3426 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 14:51:11.478276  3423 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.479542  3423 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:11.497210  2604 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:11.498534  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:11.505142  2650 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:11.510501  3378 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.513934  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.516533  3442 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.516827  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.519162  3378 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:11.520586  3378 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.517023  3442 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.523698  3426 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:11.524019  3442 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.524324  3426 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:11.524873  3426 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.530130  2682 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:11.533846  2755 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:11.551684  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.553166  3415 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:11.554492  3415 tablet_bootstrap.cc:492] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:11.553191  3387 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.555555  3387 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:11.557338  2757 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:11.563822  3415 ts_tablet_manager.cc:1403] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.406s	user 0.014s	sys 0.049s
I20260709 14:51:11.572873  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.575610  2608 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.574887  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:11.550355  3426 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.587034  3432 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.589361  3442 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.589947  3439 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:11.556696  3387 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.577302  3404 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.615973  2679 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.619825  3404 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.615145  3432 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.623500  3389 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
W20260709 14:51:11.685045  3445 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:51:11.731534  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:11.732383  2757 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.736907   431 ts_tablet_manager.cc:600] Registering tablets (29/45 complete)
I20260709 14:51:11.743892  3432 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:11.746199  3452 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 14:51:11.747292  3404 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:11.748188  3453 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 14:51:11.749547  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:11.750298  2758 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:11.763203  3426 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.769502  2757 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:11.773298  3415 raft_consensus.cc:359] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.774479  3415 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:11.773452  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.758646  3442 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:11.778993  3415 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:11.802522  3456 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.803326  3456 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.804388  3456 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:11.812968  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:11.836184  2758 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:11.841780  2679 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 14:51:11.843464  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:11.846197  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:11.865043  3460 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:11.865904  3460 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:11.867157  3460 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:11.887799  3464 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 14:51:11.906942  2679 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.929611  3434 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:11.904059  2724 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:11.942378  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:11.951480  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:11.951987  2679 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.954577  3429 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:11.970242  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 219726us
I20260709 14:51:11.974167  2608 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.978428  2430 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:11.979322  2608 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:11.996268  2430 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:12.008168  2576 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.009087  2576 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.009718  2576 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.012373  2430 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.015915  3415 ts_tablet_manager.cc:1434] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.451s	user 0.018s	sys 0.012s
I20260709 14:51:12.017361  3415 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:12.019752  2430 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.023739  2430 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.026170  2430 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:12.035349  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.036046  3462 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:12.041217  2724 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.050403  2608 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:12.068218  2608 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 14:51:12.069015  2576 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.013403  3460 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 14:51:12.063889  3383 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 14:51:12.093989  2724 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:12.104256  3482 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 14:51:12.121021  2724 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.130956  2576 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.131623  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61637.5us
W20260709 14:51:12.132282  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.132757  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.134546  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.137548  3418 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:12.139653  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:12.140452  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77778.4us
W20260709 14:51:12.141121  2724 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.143108  2650 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.150579  2650 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:12.151783  2650 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:51:12.152431  2650 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.160364  2611 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 14:51:12.168377  2650 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.178339  2650 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.176313  3434 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 14:51:12.196777  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.259121  2724 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.263949  2650 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:12.271275  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.272002  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.228199  2576 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.272594  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.274693  2650 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.275323  2650 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.273907  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45765.7us
I20260709 14:51:12.287317  2680 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 14:51:12.288724  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.289382  2650 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.291006  3385 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:12.295871  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.299669  3477 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:12.300343  3477 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:12.302233  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.309213  3415 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 14:51:12.351082  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.356191  2755 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:12.360607  2433 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.380757  2755 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:12.379889  2433 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.381386  3415 tablet_bootstrap.cc:492] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:12.382632  3415 ts_tablet_manager.cc:1403] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.366s	user 0.023s	sys 0.016s
I20260709 14:51:12.384414  3477 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:12.395694  2758 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 14:51:12.397184  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.397696  3460 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:51:12.401985  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:12.406819  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:51:12.408399  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.408178  3415 raft_consensus.cc:359] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:12.409348  3415 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:12.415369  3415 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:12.396539  3477 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:12.421146  3496 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 14:51:12.388021  3443 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 14:51:12.391729  3383 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 14:51:12.438058  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:51:12.442700  3415 ts_tablet_manager.cc:1434] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 14:51:12.443902  3415 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:12.421878  3477 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:12.465953  2758 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:12.465622  2435 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:12.476379  3477 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:12.475782  2435 catalog_manager.cc:5697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:12.484920  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:12.491820  3443 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:12.478336  2435 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.516749  2435 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.527661  2435 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:12.533105   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 14:51:12.533702   431 ts_tablet_manager.cc:595] Time spent register tablets: real 1.824s	user 0.374s	sys 0.029s
I20260709 14:51:12.529836  2435 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:12.580000  3415 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:12.599768  3415 tablet_bootstrap.cc:492] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:12.600704  3415 ts_tablet_manager.cc:1403] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.010s
I20260709 14:51:12.626638  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:12.653134  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:12.661087  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:12.662262  3477 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:12.662547  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:12.656584  3415 raft_consensus.cc:359] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:12.671362  3415 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:12.672274  3415 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:12.674369  3415 ts_tablet_manager.cc:1434] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.073s	user 0.002s	sys 0.003s
I20260709 14:51:12.680622  3477 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:12.681341  3477 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:12.681854  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:12.683933  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:12.685317  3499 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:12.686425  3477 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:12.689136  3415 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:12.688498  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:12.719136  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:12.721881  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:12.723315  3477 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:12.784327  3415 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:12.785650  3415 tablet_bootstrap.cc:492] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:12.786460  3415 ts_tablet_manager.cc:1403] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.000s
I20260709 14:51:12.789597  3415 raft_consensus.cc:359] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:12.791253  3415 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:12.792083  3415 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:12.794413  3415 ts_tablet_manager.cc:1434] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:51:12.795302  3415 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:12.914062  3415 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:12.916606  3415 tablet_bootstrap.cc:492] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:12.917654  3415 ts_tablet_manager.cc:1403] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.123s	user 0.035s	sys 0.026s
I20260709 14:51:12.920789  3415 raft_consensus.cc:359] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:12.921705  3415 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:12.922709  3415 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:12.926126  3415 ts_tablet_manager.cc:1434] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 14:51:12.927321  3415 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:12.981829  3415 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:12.983166  3415 tablet_bootstrap.cc:492] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:12.984365  3415 ts_tablet_manager.cc:1403] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.057s	user 0.035s	sys 0.005s
I20260709 14:51:12.987596  3415 raft_consensus.cc:359] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:12.988897  3415 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:12.989730  3415 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:12.992115  3415 ts_tablet_manager.cc:1434] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:51:12.993057  3415 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.035815  3415 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.036953  3415 tablet_bootstrap.cc:492] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.037710  3415 ts_tablet_manager.cc:1403] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.003s
I20260709 14:51:13.040772  3415 raft_consensus.cc:359] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.041610  3415 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.042629  3415 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.045713  3415 ts_tablet_manager.cc:1434] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:51:13.046871  3415 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.068441  3415 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.069885  3415 tablet_bootstrap.cc:492] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.070616  3415 ts_tablet_manager.cc:1403] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 14:51:13.073571  3415 raft_consensus.cc:359] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.074441  3415 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.076862  3415 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.085719  3415 ts_tablet_manager.cc:1434] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 14:51:13.088817  3415 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.237934  3477 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:13.238693  3477 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.245784  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:13.246368  3477 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:13.247174  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:13.249162  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:13.254480  3477 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:13.282052  3415 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.284030  3415 tablet_bootstrap.cc:492] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.284883  3415 ts_tablet_manager.cc:1403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.196s	user 0.019s	sys 0.007s
I20260709 14:51:13.288228  3415 raft_consensus.cc:359] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.289465  3415 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.290704  3415 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.293335  3415 ts_tablet_manager.cc:1434] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:51:13.294400  3415 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.371583  3415 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.372658  3415 tablet_bootstrap.cc:492] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.373382  3415 ts_tablet_manager.cc:1403] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.079s	user 0.038s	sys 0.023s
I20260709 14:51:13.376286  3415 raft_consensus.cc:359] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.377053  3415 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.377915  3415 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.380121  3415 ts_tablet_manager.cc:1434] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 14:51:13.382994  3415 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.412156  3415 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.413264  3415 tablet_bootstrap.cc:492] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.413969  3415 ts_tablet_manager.cc:1403] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.009s
I20260709 14:51:13.416844  3415 raft_consensus.cc:359] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.417582  3415 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.418517  3415 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.420970  3415 ts_tablet_manager.cc:1434] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 14:51:13.421833  3415 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.445688  3415 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.446805  3415 tablet_bootstrap.cc:492] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.447769  3415 ts_tablet_manager.cc:1403] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.003s
I20260709 14:51:13.450616  3415 raft_consensus.cc:359] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.451450  3415 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.452219  3415 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.454720  3415 ts_tablet_manager.cc:1434] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:51:13.455617  3415 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.581837  3415 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.582947  3415 tablet_bootstrap.cc:492] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.583683  3415 ts_tablet_manager.cc:1403] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.128s	user 0.046s	sys 0.053s
I20260709 14:51:13.586514  3415 raft_consensus.cc:359] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.587286  3415 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.593752  3415 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.596418  3415 ts_tablet_manager.cc:1434] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 14:51:13.597344  3415 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.618069  3415 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.619191  3415 tablet_bootstrap.cc:492] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.619884  3415 ts_tablet_manager.cc:1403] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 14:51:13.627300  3415 raft_consensus.cc:359] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.628422  3415 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.629408  3415 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.632341  3415 ts_tablet_manager.cc:1434] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 14:51:13.633210  3415 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.691861  3477 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:13.692749  3477 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.697206  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:13.697417  3477 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:13.706542  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:13.710558  3415 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.711449  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:13.711860  3415 tablet_bootstrap.cc:492] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.712438  3477 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:13.712688  3415 ts_tablet_manager.cc:1403] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.080s	user 0.022s	sys 0.000s
I20260709 14:51:13.719264   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:37831
I20260709 14:51:13.726987  3580 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:37831 every 8 connection(s)
I20260709 14:51:13.726751  3415 raft_consensus.cc:359] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.727869  3415 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.728811  3415 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:13.734022  3415 ts_tablet_manager.cc:1434] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 14:51:13.738633  3415 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.784104  3415 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.785301  3415 tablet_bootstrap.cc:492] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.786130  3415 ts_tablet_manager.cc:1403] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.048s	user 0.025s	sys 0.004s
I20260709 14:51:13.791178  3415 raft_consensus.cc:359] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.793675  3415 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.794461  3415 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:13.799855  3415 ts_tablet_manager.cc:1434] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 14:51:13.802937  3415 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:13.830271  3546 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:13.831729  3543 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:13.856679  3547 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:13.880281  3415 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:13.881525  3415 tablet_bootstrap.cc:492] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:13.882277  3415 ts_tablet_manager.cc:1403] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.080s	user 0.028s	sys 0.002s
I20260709 14:51:13.886462  3545 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:13.887482  3415 raft_consensus.cc:359] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.888329  3415 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:13.893837  3415 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:13.903221  3415 ts_tablet_manager.cc:1434] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.004s
I20260709 14:51:13.904224  3415 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
W20260709 14:51:13.994351  2650 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.043728  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.046964  3581 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39575
I20260709 14:51:14.047619  3581 heartbeater.cc:461] Registering TS with master...
I20260709 14:51:14.048681  3581 heartbeater.cc:507] Master 127.0.107.254:39575 requested a full tablet report, sending...
I20260709 14:51:14.058096  3415 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.059338  3415 tablet_bootstrap.cc:492] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.060176  3415 ts_tablet_manager.cc:1403] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.156s	user 0.013s	sys 0.014s
W20260709 14:51:14.068902  2650 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:14.073160  2435 ts_manager.cc:194] Re-registered known tserver with Master: c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:14.076155  3415 raft_consensus.cc:359] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.078616  3415 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.081902  3415 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.084858  3415 ts_tablet_manager.cc:1434] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 14:51:14.086004  3415 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.096259  3546 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:14.112715  3545 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:14.116272  3547 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:14.136116  2650 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.154670  3543 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:14.185204  2650 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:14.200358  3546 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:14.204530  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.234658  2435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52794
I20260709 14:51:14.241693  3581 heartbeater.cc:499] Master 127.0.107.254:39575 was elected leader, sending a full tablet report...
W20260709 14:51:14.261258  2576 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.294945  2576 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:51:14.340319  2724 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.356927  2650 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.363426  2576 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.432847  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.434680  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.439078  3415 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.445317  3415 tablet_bootstrap.cc:492] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.446350  3415 ts_tablet_manager.cc:1403] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.361s	user 0.028s	sys 0.008s
I20260709 14:51:14.449069  3415 raft_consensus.cc:359] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.449817  3415 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.450650  3415 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.453343  3415 ts_tablet_manager.cc:1434] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:51:14.454437  3415 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.571041  3415 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.572289  3415 tablet_bootstrap.cc:492] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.573441  3415 ts_tablet_manager.cc:1403] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.119s	user 0.019s	sys 0.006s
I20260709 14:51:14.578421  3415 raft_consensus.cc:359] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.580596  3415 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
W20260709 14:51:14.580905  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.581564  3415 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.583997  3415 ts_tablet_manager.cc:1434] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 14:51:14.585148  3415 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.637349  3542 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:14.644115  2724 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.683595  2650 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:51:14.739108  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.775215  3415 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.776348  3415 tablet_bootstrap.cc:492] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.777200  3415 ts_tablet_manager.cc:1403] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.192s	user 0.091s	sys 0.065s
I20260709 14:51:14.780393  3415 raft_consensus.cc:359] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:14.781121  3415 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.781855  3415 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:14.787489  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:51:14.788376  3415 ts_tablet_manager.cc:1434] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 14:51:14.789357  3415 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.829290  3415 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.830452  3415 tablet_bootstrap.cc:492] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.831245  3415 ts_tablet_manager.cc:1403] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.007s
I20260709 14:51:14.839359  3415 raft_consensus.cc:359] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.840320  3415 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.841110  3415 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.843538  3415 ts_tablet_manager.cc:1434] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 14:51:14.845494  3415 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.863945  3415 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.865222  3415 tablet_bootstrap.cc:492] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.866115  3415 ts_tablet_manager.cc:1403] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.006s
I20260709 14:51:14.868784  3415 raft_consensus.cc:359] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:14.869387  3415 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.870127  3415 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:14.872807  3415 ts_tablet_manager.cc:1434] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 14:51:14.873667  3415 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.891533  3415 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.892891  3415 tablet_bootstrap.cc:492] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.893707  3415 ts_tablet_manager.cc:1403] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
W20260709 14:51:14.895884  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:14.896955  3415 raft_consensus.cc:359] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.897801  3415 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.898607  3415 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.901096  3415 ts_tablet_manager.cc:1434] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:51:14.902182  3415 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.912742  3545 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:14.937994  3415 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 14:51:14.939397  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:51:14.940851  3415 tablet_bootstrap.cc:492] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.941746  3415 ts_tablet_manager.cc:1403] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.012s
I20260709 14:51:14.944818  3415 raft_consensus.cc:359] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.945469  3415 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.946216  3415 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:14.948696  3415 ts_tablet_manager.cc:1434] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:51:14.949548  3415 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:14.984793  3415 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:14.986099  3415 tablet_bootstrap.cc:492] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:14.987071  3415 ts_tablet_manager.cc:1403] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.014s
I20260709 14:51:14.992635  3415 raft_consensus.cc:359] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.993824  3415 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:14.994680  3415 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:14.996855  3415 ts_tablet_manager.cc:1434] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:51:15.000586  3415 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.001459  3545 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:15.050191  3415 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.051575  3415 tablet_bootstrap.cc:492] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.052486  3415 ts_tablet_manager.cc:1403] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.052s	user 0.025s	sys 0.021s
I20260709 14:51:15.055732  3415 raft_consensus.cc:359] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.056665  3415 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.057540  3415 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.060143  3415 ts_tablet_manager.cc:1434] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 14:51:15.061192  3415 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.081310  3415 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.082497  3415 tablet_bootstrap.cc:492] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.083290  3415 ts_tablet_manager.cc:1403] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 14:51:15.086354  3415 raft_consensus.cc:359] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.087322  3415 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.088280  3415 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.090502  3415 ts_tablet_manager.cc:1434] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:51:15.091447  3415 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.281870  3415 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.285319  3415 tablet_bootstrap.cc:492] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.286226  3415 ts_tablet_manager.cc:1403] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.195s	user 0.080s	sys 0.096s
I20260709 14:51:15.289275  3415 raft_consensus.cc:359] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.290123  3415 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.290972  3415 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.292706  3415 ts_tablet_manager.cc:1434] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 14:51:15.293610  3415 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.321528  3415 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.323561  3415 tablet_bootstrap.cc:492] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.324357  3415 ts_tablet_manager.cc:1403] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.008s
I20260709 14:51:15.327188  3415 raft_consensus.cc:359] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.327838  3415 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.329689  3415 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.332127  3415 ts_tablet_manager.cc:1434] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:51:15.334698  3415 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.369624  3415 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.370546  3415 tablet_bootstrap.cc:492] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.371243  3415 ts_tablet_manager.cc:1403] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.009s
I20260709 14:51:15.374339  3415 raft_consensus.cc:359] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:15.375037  3415 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.376315  3415 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:15.378962  3415 ts_tablet_manager.cc:1434] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:51:15.383034  3415 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.408960  3415 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.410271  3415 tablet_bootstrap.cc:492] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.411161  3415 ts_tablet_manager.cc:1403] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.008s
I20260709 14:51:15.416028  3415 raft_consensus.cc:359] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.416846  3415 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.418521  3415 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.420835  3415 ts_tablet_manager.cc:1434] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 14:51:15.421651  3415 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.449877  3415 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.451197  3415 tablet_bootstrap.cc:492] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.452178  3415 ts_tablet_manager.cc:1403] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.009s
I20260709 14:51:15.467163  3415 raft_consensus.cc:359] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.468101  3415 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.468928  3415 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.471743  3415 ts_tablet_manager.cc:1434] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 14:51:15.481426  3415 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.497965  3415 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.499010  3415 tablet_bootstrap.cc:492] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.499809  3415 ts_tablet_manager.cc:1403] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.003s
I20260709 14:51:15.503448  3415 raft_consensus.cc:359] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.504256  3415 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.505043  3415 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.508755  3415 ts_tablet_manager.cc:1434] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 14:51:15.510286  3415 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.528631  3415 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.529732  3415 tablet_bootstrap.cc:492] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.530519  3415 ts_tablet_manager.cc:1403] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 14:51:15.535562  3542 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:15.535259  3415 raft_consensus.cc:359] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.538311  3415 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.539268  3415 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.542088  3415 ts_tablet_manager.cc:1434] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 14:51:15.543076  3415 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.568045  3545 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:15.639746  3415 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.645753  3415 tablet_bootstrap.cc:492] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.646809  3415 ts_tablet_manager.cc:1403] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.104s	user 0.050s	sys 0.030s
I20260709 14:51:15.666733  3415 raft_consensus.cc:359] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.668128  3415 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.670455  3415 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.688607  3415 ts_tablet_manager.cc:1434] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 14:51:15.689692  3415 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.717814  3415 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.719506  3415 tablet_bootstrap.cc:492] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.720402  3415 ts_tablet_manager.cc:1403] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.012s
I20260709 14:51:15.723486  3415 raft_consensus.cc:359] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.724298  3415 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.725188  3415 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:15.727948  3415 ts_tablet_manager.cc:1434] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:51:15.728957  3415 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
W20260709 14:51:15.785241  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61097.4us
I20260709 14:51:15.818056  3415 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.819518  3415 tablet_bootstrap.cc:492] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.820390  3415 ts_tablet_manager.cc:1403] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.092s	user 0.019s	sys 0.005s
I20260709 14:51:15.826373  3415 raft_consensus.cc:359] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.829576  3415 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.865049  3415 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:15.867604  3415 ts_tablet_manager.cc:1434] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.001s
I20260709 14:51:15.868551  3415 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:15.888938  3545 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:15.914311  3415 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:15.915050  3547 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:15.919620  3415 tablet_bootstrap.cc:492] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:15.920609  3415 ts_tablet_manager.cc:1403] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.007s
I20260709 14:51:15.923586  3415 raft_consensus.cc:359] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:15.924357  3415 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:15.935664  3415 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:15.937503  3415 ts_tablet_manager.cc:1434] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 14:51:15.938709  3415 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:16.055460  3415 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:16.056704  3415 tablet_bootstrap.cc:492] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:16.057572  3415 ts_tablet_manager.cc:1403] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.119s	user 0.064s	sys 0.033s
I20260709 14:51:16.060745  3415 raft_consensus.cc:359] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:16.061569  3415 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:16.062316  3415 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:16.065341  3415 ts_tablet_manager.cc:1434] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:51:16.066257  3415 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:16.084954  3415 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:16.086005  3415 tablet_bootstrap.cc:492] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:16.086766  3415 ts_tablet_manager.cc:1403] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 14:51:16.089761  3415 raft_consensus.cc:359] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:16.090523  3415 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:16.095049  3415 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:16.097566  3415 ts_tablet_manager.cc:1434] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 14:51:16.098924  3415 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap starting.
I20260709 14:51:16.130663  3415 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:51:16.132076  3415 tablet_bootstrap.cc:492] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Bootstrap complete.
I20260709 14:51:16.132874  3415 ts_tablet_manager.cc:1403] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent bootstrapping tablet: real 0.034s	user 0.025s	sys 0.003s
I20260709 14:51:16.136010  3415 raft_consensus.cc:359] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:16.136757  3415 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Initialized, Role: FOLLOWER
I20260709 14:51:16.137584  3415 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:16.139880  3415 ts_tablet_manager.cc:1434] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:51:16.325590  3542 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.704825   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:19.777702  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:19.778628  2758 raft_consensus.cc:606] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:51:19.782122  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:19.782943  2758 raft_consensus.cc:606] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.788300  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:19.789119  2758 raft_consensus.cc:606] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.802872  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:19.803509  2679 raft_consensus.cc:606] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.806396  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:19.807014  2758 raft_consensus.cc:606] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.821269  2608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a8bd118baa7481399ce15d422463083"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:19.821540  3653 raft_consensus.cc:993] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.822001  2608 raft_consensus.cc:606] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.822067  3653 raft_consensus.cc:1081] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.824802  3545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:51:19.825659  3545 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:19.826169  3545 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.826288  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:19.827100  2758 raft_consensus.cc:606] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:19.829066   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 14:51:19.830718   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:19.834185  3545 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.836861  3545 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:19.837458  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:19.837116  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:19.838268  2608 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.838526  2758 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:19.838917  2758 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:19.839715  2758 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.841145  2758 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.846500  2608 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:19.847675  2758 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:19.847985  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:19.849818  3658 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:19.851500  3658 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:19.853062  3658 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.866688  3419 raft_consensus.cc:993] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:19.867583  3653 raft_consensus.cc:1081] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Signalling peer 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:19.867650  2435 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:19.869336  2608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:55874
I20260709 14:51:19.869946  2608 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:19.870296  2608 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.878566  2608 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:19.881978  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:19.882063  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:19.882686  2758 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:19.883024  2679 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.881439  2608 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:19.883075  2758 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:19.889346  2758 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:19.890905  2758 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:19.892988  3661 raft_consensus.cc:993] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.893604  3661 raft_consensus.cc:1081] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.895409  3545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:51:19.895989  3545 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:19.896371  3545 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.904947  2679 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:19.904985  3545 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:19.906553  2758 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:19.907974  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:19.908699  3545 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:19.909034  3649 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:19.910310  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:19.912806  2758 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:19.913098  2758 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:19.913756  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:19.914453  2680 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.915895  3653 raft_consensus.cc:993] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.916482  3661 raft_consensus.cc:1081] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:19.913805  2758 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:19.918437  3542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:51:19.919018  3542 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:19.919095  2758 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.919418  3542 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.924813  2680 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:19.926445  3649 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:19.926888  2758 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:19.926990  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:19.928154  3658 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:19.928071  3649 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:19.928784  3658 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:19.929267  3542 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.929787  3658 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:19.932323  3542 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:19.933825  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:19.934461  2608 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.935717  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:19.937552  2758 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:19.938482  2758 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:19.943953  2608 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:19.946991  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:19.946707  2758 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.948151  3658 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:19.948357  2758 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:19.947842  2435 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:19.948828  3658 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:19.955953  3658 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:19.951860  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:19.969434  2758 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:20.003486  3671 raft_consensus.cc:993] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:20.003790  3661 raft_consensus.cc:993] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:20.005330  3661 raft_consensus.cc:1081] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
W20260709 14:51:20.005527  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45671.9us
I20260709 14:51:20.005594  3672 raft_consensus.cc:1081] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:20.007656  3542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:51:20.008355  3542 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:20.008757  3542 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:20.018620  3545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:20.019469  3545 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:20.019865  3545 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.027369  3542 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:20.066385  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 108915us
I20260709 14:51:20.085105  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:20.087049  2758 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:20.089515  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:20.091898  2611 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:20.103900  3542 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:20.094367  2435 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:20.119129  2611 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:20.124526  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:20.127334  3658 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:20.128297  3658 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:20.130028  3658 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:20.087797  2758 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:20.029644  3545 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:20.147459  3545 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:20.152812  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:20.154493  2611 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.161161  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:20.162333  2679 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:20.162786  2679 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:20.164628  2679 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:20.170745  2679 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.148650  2758 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:20.179513  2758 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:20.198132  2611 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
W20260709 14:51:20.205169  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78524.8us
I20260709 14:51:20.205332  2679 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:20.215858  2758 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:20.227032  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:20.229298  3658 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:20.229908  3666 raft_consensus.cc:993] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:20.229997  3658 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:20.233024  3666 raft_consensus.cc:1081] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:20.233137  3658 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:20.255523  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48275.1us
I20260709 14:51:20.269505  3542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a8bd118baa7481399ce15d422463083"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37174
I20260709 14:51:20.270432  3542 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:20.270937  3542 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.347718  3542 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:20.354136  3542 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:20.359715  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:20.360543  2755 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.366544  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:20.367460  2608 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:20.367822  2608 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:20.368739  2608 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:20.370290  2608 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:20.376332  2680 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.378650  3659 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:20.380519  2755 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:20.381028  2435 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:20.385587  2435 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:20.390741  2611 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.395613  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:20.397630  3659 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:20.398270  3659 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:20.399425  3659 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:20.400745  2608 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:20.403398  3658 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:20.406690  2755 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.409121  3658 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:20.436661  2758 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.439258  3658 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.428818  2435 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:20.466027  2754 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.473479  3658 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:20.474023  2680 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.477113  3679 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.483381  2754 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.489794  2608 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.496178  3652 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.577561  3697 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:51:20.621748  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83768us
I20260709 14:51:20.637931  2754 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.639703  3697 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:20.655544  2608 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:20.657792  3697 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:20.693471  2679 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.696630  3700 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.722517  2611 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.725172  3700 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.832283   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:20.854473  2754 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.856263  3700 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:20.872960  2608 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:20.874866  3700 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:20.906054   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:20.955813  2680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:20.956478  2680 raft_consensus.cc:606] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:51:20.959266  2608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:20.959826  2608 raft_consensus.cc:606] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:20.962929  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:20.963639  2679 raft_consensus.cc:606] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:20.966284  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:20.966784  2679 raft_consensus.cc:606] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:20.969434  2680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:20.969990  2680 raft_consensus.cc:606] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:20.971057   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:51:20.972568   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:21.062695  3672 raft_consensus.cc:993] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:21.063549  3671 raft_consensus.cc:1081] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Signalling peer 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:21.065292  2608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:55870
I20260709 14:51:21.065766  3672 raft_consensus.cc:993] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:21.065955  2608 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:21.066380  3671 raft_consensus.cc:1081] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Signalling peer 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:21.066457  2608 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.068342  2754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:42502
I20260709 14:51:21.068979  2754 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:21.069319  2754 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.084468  3597 raft_consensus.cc:993] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:21.085134  3671 raft_consensus.cc:1081] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:21.087659  2754 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.094794  3545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:21.095486  3545 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:21.095839  3545 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.095901  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:21.096619  2611 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.099895  2754 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:21.092216  2608 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.100788  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:21.101473  2680 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:21.101794  2680 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:21.102742  2680 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.103931  2680 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.104156  2608 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:21.105670  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:21.106026  3545 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.106349  2754 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.109282  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:21.110132  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:21.110450  2608 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.110767  2682 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:21.111152  2682 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:21.111995  2682 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.112954  3545 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:21.113412  2682 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.118856  2608 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:21.119858  2754 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:21.120371  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:21.121831  3697 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:21.121984  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:21.122426  3697 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:21.122803  2681 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:21.123214  2681 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:21.123481  3697 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.124182  2681 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.125777  2681 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.129715  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:21.130991  3685 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:21.131776  3685 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:21.143188  2611 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:21.143188  2682 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:21.138751  2680 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:21.146328  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:21.147512  3419 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:21.139762  2681 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:21.148046  3419 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:21.149039  3419 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.132686  3685 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.195426  2435 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:21.194525  2430 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:21.204856  2433 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:21.245693  3652 raft_consensus.cc:993] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: : Instructing follower 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:21.246239  3685 raft_consensus.cc:1081] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Signalling peer 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:21.247977  2754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:42488
I20260709 14:51:21.248517  2754 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:21.248812  2754 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.255729  2754 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.258708  2754 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:21.259095  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:21.259867  2681 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.260999  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:21.261641  2611 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:21.261924  2611 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:21.262728  2611 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.264084  2611 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:21.268534  2681 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:21.270077  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:21.271636  3419 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:21.272127  3419 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:21.272953  3419 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:21.278072  2611 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:21.288447  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 2 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:21.388716  3672 raft_consensus.cc:993] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:21.389228  3597 raft_consensus.cc:1081] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:21.390892  3542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:21.391494  3542 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:21.391871  3542 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.397841  3542 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.399571  3542 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:21.400611  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:21.400722  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:21.401320  2611 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.401401  2681 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:21.401865  2681 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:21.402606  2681 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.403703  2681 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:21.407608  2611 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:21.409252  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:21.409480  2681 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:21.410375  3700 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:21.411062  3700 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:21.412000  3700 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:21.424226  2433 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:21.533437  2680 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.535640  3661 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.553016  2611 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.555756  3661 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.637606  2755 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:21.640195  3652 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.644676  2682 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:21.646963  3685 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.675153  2611 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.681881  3658 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:21.688464  2682 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.690399  3700 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.782923  2611 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:21.784276  3419 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:21.796023  2682 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:21.797813  3661 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:21.851188  2611 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.852514  3658 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:21.863593  2682 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:21.865146  3735 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:21.973517   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:22.027561   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:22.064749  2682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:22.065429  2682 raft_consensus.cc:606] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:22.068849  2755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:22.069453  2755 raft_consensus.cc:606] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:22.072837  2754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:22.073447  2754 raft_consensus.cc:606] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:51:22.076467  2608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:22.077078  2608 raft_consensus.cc:606] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:22.080508  2682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:22.081141  2682 raft_consensus.cc:606] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:22.082306   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:51:22.084306   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:22.084357  3419 raft_consensus.cc:993] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:22.084947  3661 raft_consensus.cc:1081] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Signalling peer 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:22.086675  2608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:55874
I20260709 14:51:22.087368  2608 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:22.087828  2608 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.095280  2608 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.097584  2608 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:22.098201  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:22.099562  2682 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.099647  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:22.100359  2754 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:22.100699  2754 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:22.101380  2754 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.102362  2754 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.105374  2682 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:22.106671  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:22.107486  3685 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:22.107824  2754 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:22.107918  3685 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:22.108989  3685 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.119776  2433 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:22.215230  3652 raft_consensus.cc:993] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: : Instructing follower 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:22.215798  3686 raft_consensus.cc:1081] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Signalling peer 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:22.217221  2754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:42488
I20260709 14:51:22.217747  2754 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:22.218088  2754 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.223510  2754 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.225276  2754 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:22.226227  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:22.226269  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:22.226995  2611 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:22.227044  2682 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.227299  2611 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:22.228128  2611 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.229389  2611 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.234668  2682 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:22.235917  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:22.236666  3419 raft_consensus.cc:993] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.237257  3731 raft_consensus.cc:1081] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.236933  3661 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:22.239701  2611 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:22.240617  3545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:51:22.241256  3545 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:22.241662  3545 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.239840  3661 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:22.243191  3661 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:22.243672  3733 raft_consensus.cc:993] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.244333  3672 raft_consensus.cc:1081] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.247062  3542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:22.248080  3542 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:22.248461  3542 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.252275  3545 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.254632  3545 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:22.256201  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:22.256942  2754 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:22.257249  2754 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:22.258417  2754 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.259701  2754 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.258515  3542 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.266503  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:22.267222  2682 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.267343  3542 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:22.269476  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:22.270125  2755 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.272372  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:22.273190  2681 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:22.273528  2681 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:22.265110  2433 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:22.280359  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:22.283974  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:22.285036  3658 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:22.285853  3658 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:22.274362  2681 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.288406  2754 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:22.290313  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:22.291293  3700 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:22.291431  2681 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:22.291879  3700 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:22.292925  3700 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.286818  3658 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:22.322351  3733 raft_consensus.cc:993] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.322966  3733 raft_consensus.cc:1081] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:22.324371  2682 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:22.325186  3543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:22.325810  3543 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:22.326326  3543 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.353331  2681 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:22.347945  3543 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:22.362569  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:22.363288  2611 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.364143  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:22.364825  3543 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:22.375311  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:22.375233  2433 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:22.376768  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:22.378221  3658 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:22.378974  3658 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:22.380250  3658 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:22.381322  2681 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:22.381798  2681 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:22.382661  2681 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:22.384166  2681 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:22.394059  2681 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:22.407372  2433 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:22.408846  2433 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:22.504653  2754 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:22.506436  3685 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:22.519574  2681 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:22.521731  3652 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:22.796128  2611 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.797318  3419 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:22.805020  2681 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.808650  3700 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:22.809158  2681 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.815552  3419 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:22.823354  2755 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.838419  3744 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:22.863808  2608 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.865718  3744 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:22.871176  2755 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:22.871666  2681 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:22.873167  3697 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:22.879120  3697 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:22.906900  2681 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:22.911942  3697 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:23.121244   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:24.034893  3759 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.035892  3759 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.040642  3759 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.051988  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.061728  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.089540  2430 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 153) took 1100 ms. Trace:
I20260709 14:51:24.084873  3733 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:24.096115  2433 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 80) took 1092 ms. Trace:
I20260709 14:51:24.098130  3762 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.101686  3759 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.103444  3759 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.096731  2433 rpcz_store.cc:276] 0709 14:51:23.004130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:23.004286 (+   156us) service_pool.cc:224] Handling call
0709 14:51:24.096030 (+1091744us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":976233,"reported_tablets":1}
I20260709 14:51:24.090368  2430 rpcz_store.cc:276] 0709 14:51:22.989827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:22.998309 (+  8482us) service_pool.cc:224] Handling call
0709 14:51:24.089420 (+1091111us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 14:51:24.117818  3764 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.118655  3764 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.126221  3764 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.096064  3733 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.107679  3762 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.170920  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:24.162571  2466 debug-util.cc:398] Leaking SignalData structure 0x7b0800218480 after lost signal to thread 434
W20260709 14:51:24.183091  2466 debug-util.cc:398] Leaking SignalData structure 0x7b080000e500 after lost signal to thread 2469
I20260709 14:51:24.189764  3652 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.175987  3686 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:24.211261  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce940 after lost signal to thread 2636
I20260709 14:51:24.211294  3760 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.203979  3764 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.218226  3760 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.219048  3765 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:24.219072  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9a00 after lost signal to thread 2710
I20260709 14:51:24.211648  3686 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.219084  3764 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.232795  3760 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.232421  3765 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.233559  3760 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.234350  3686 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.234160  3760 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.234752  3764 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.236822  3760 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.237375  3772 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:24.238153  3772 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.240988  3773 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.251585  3597 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.242192  3764 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.230015  3759 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.264169  3672 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.271862  3761 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.252380  3597 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.245975  3770 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.287644  3597 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:24.277240  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08002b3e20 after lost signal to thread 2784
I20260709 14:51:24.272583  3768 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.272653  3761 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.294469  3765 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.272111  3672 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.295789  3733 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.245209  3773 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.279865  3764 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.302228  3772 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.312808  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.303695  3743 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:24.314224  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 124818us
I20260709 14:51:24.317091  2608 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:24.321328  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.306661  3762 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:24.314698  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5200 after lost signal to thread 3580
I20260709 14:51:24.325498  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.331067  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.338042  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.301045  3768 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.348578  3768 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.349186  3768 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.349746  3768 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.289593  3770 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.355175   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:24.374452  3419 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.319633  3743 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.375391  3419 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.376973  3770 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.377121  3743 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.377422  3419 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.379665  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.379948  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.381217  3780 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.382762  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.384112  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.385169  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.385983  2756 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:24.364389  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.382045  3780 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.382089  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.404917  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.406649  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:24.408501  3419 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.409375  3419 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:24.422994  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 122017us
I20260709 14:51:24.427846  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.429256  3652 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:24.464133  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 95678.8us
I20260709 14:51:24.464928  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:24.463066  3686 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.474504  3419 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.472218  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.472195  3775 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.474947  3763 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.476064  3767 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.476347  3759 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.476763  3763 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.476895  3769 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.476680  3767 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.477458  3782 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.478585  3686 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.477026  3759 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:24.481976  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83310.4us
I20260709 14:51:24.487867  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.491863  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.471783  3761 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.481169  3686 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.476277  3775 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.478266  3785 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.478785  3672 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.502314  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.478919  3782 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.505635  3782 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.505771  3775 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.477377  3763 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.502756  3785 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.512574  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.512696  3784 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.514235  2605 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:24.514626  3784 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.490793  3764 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.517802  3784 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.519030  3784 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.513975  3790 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:24.531689  3779 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.524050  3784 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.532338  3779 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.524420  3790 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.534299  3790 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.534657  3784 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.539055  3794 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.478367  3769 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.513149  3761 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.515031  3791 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.541669  3771 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:24.545308  3785 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.546872  3791 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:24.530745  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86124.5us
I20260709 14:51:24.540930  3794 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.553500  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.555977  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.556036  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.556363  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.566282  2681 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:24.556627  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.557179  3597 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.567337  3597 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.558182  3793 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.572320  3769 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.559115  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.547675  3761 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.559597  3686 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.561522  3773 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.562611  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.578258  3686 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.563916  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.564682  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.580065  2679 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:24.572023  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.581076  3761 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.582000  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.582403  3686 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.550359  3788 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.571811  3768 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.572844  3793 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.544052  3780 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.564952  3759 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.585107  3783 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.583539  2680 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:24.587435  3767 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.612267  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.587782  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:24.591096  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104663us
I20260709 14:51:24.591670  3597 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.596131  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.555804  3763 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.599429  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.604314  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.604939  3773 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.583611  3771 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.607280  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.617424  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:24.612906  2611 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:24.620208  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:24.605127  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:24.616120  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.622783  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.628163  3794 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.623400  3773 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.630319  3773 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.623972  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.631994  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.622587  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.635049  3597 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:24.635509  3597 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.635854  3597 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.636500  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:24.610150  3783 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.650986  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.650385  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.631240  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.655944  3542 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:24.650089  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:24.653483  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.657663  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.659070  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.660641  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.661791  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.620189  3779 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.649412  3794 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.666105  3540 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:24.666276  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.651078  3764 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.658038  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.638785  3765 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:24.667150  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.616519  3790 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.682336  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.678365  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.669306  3784 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.684860  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.692885  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.693878  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.682361  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.699735  2605 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:24.679184  3764 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.702265  3764 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.687954  3791 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.692821  3784 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.705096  3784 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.676087  3773 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.674413  3769 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.708484  2681 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:24.697664  3780 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.719556  3780 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.720712  3771 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.721020  3545 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:24.707151  3762 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:24.721750  3791 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.722411  3791 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.722613  3769 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.676100  3794 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.679512  3765 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.724799  3775 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:24.724857  3765 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:24.725606  3775 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.728236  3773 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.682415  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.755273  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:24.755784  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43745.4us
W20260709 14:51:24.757397  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58295us
I20260709 14:51:24.757436  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:24.759680  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52096.9us
I20260709 14:51:24.762534  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:24.763339  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51071.7us
I20260709 14:51:24.766579  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.768790  3790 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.681385  3793 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.608043  3788 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.800840  3780 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.806808  2611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:24.807742  2611 raft_consensus.cc:606] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.811685  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:24.812336  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47569.8us
I20260709 14:51:24.813699  3775 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.813895  3788 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.814092  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.814395  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:24.818073  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.771317  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.819641  3792 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.823586  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.823207  3762 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.824944  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.826786  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.830516  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.834014  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.817742  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:24.835430  3783 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.812845  3791 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.839401  3769 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.835451  3597 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.832430  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.835760  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.849206  2678 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:24.839015  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.832963  3731 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 14:51:24.817706  3800 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.836244  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.813522  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.832523  3794 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.825800  3762 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.844517  2757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:24.856374  3762 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.856551  2757 raft_consensus.cc:606] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.858543  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:24.861630  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:24.863631  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.864500  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.846206  3784 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.822580  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.849200  3773 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.849766  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.875216  3784 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.875280  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.839129  3798 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.880199  3762 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.851430  3731 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:24.855165  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:24.855415  3790 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.883474  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.875372  3783 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.855239  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.858465  3597 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.871555  3796 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.878413  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.886058  3770 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:24.878906  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.881145  3731 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:24.872833  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.889321  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.890763  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.891310  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:24.887605  2678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:24.895839  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.896209  2678 raft_consensus.cc:606] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.897727  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.895740  3787 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:24.898535  3787 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:24.899207  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.899255  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:24.903378   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 14:51:24.905407   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:24.905438  3686 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.908340  3686 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.908772  3686 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.908931  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.910163  3743 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.910290  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:24.912011  3419 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:24.912287  3419 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:24.912664  3419 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:24.901186  3787 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.914384  3797 raft_consensus.cc:993] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:24.914067  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.916368  3797 raft_consensus.cc:1081] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:24.917487  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:24.918095  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.913239  3768 raft_consensus.cc:993] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:24.919800  3788 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.920271  3783 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:24.920421  3768 raft_consensus.cc:1081] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Signalling peer 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:51:24.920737  3783 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:24.921172  3783 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.915526  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.922752  3760 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.901476  3731 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:24.925613  3731 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.928381  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.929037  3787 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.932268  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.851435  3782 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.880846  3798 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.934674  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.937716  3788 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.937384  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.938691  3788 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.926381  3764 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.938699  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.942974  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.936280  3791 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.949245  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.946306  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.940308  2682 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:24.950701  3791 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.941862  3764 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.939193  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.943256  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:24.952646  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.952925  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.953299  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.954056  3791 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.955226  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.956924  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:24.957194  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:24.958364  3536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37174
I20260709 14:51:24.959120  3536 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:24.959626  3536 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.950718  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.960242  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.959439  3419 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.963281  3788 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.963393  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:24.963403  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:24.964349  3762 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:24.913772  3765 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:24.980782  3783 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:24.983053  3765 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.983426  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:24.984076  3791 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.984611  3773 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:24.966084  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.985697  3794 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:24.986094  3794 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:24.986931  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:24.988793  3770 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:24.989144  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:24.989513  3770 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:24.989995  3770 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:24.990361  3770 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:24.991015  3765 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:24.991544  3765 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:24.991952  3799 raft_consensus.cc:993] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:24.992255  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:24.992473  3799 raft_consensus.cc:1081] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:24.991997  3765 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:24.967685  3798 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:24.969792  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:24.994388  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:24.974968  3782 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:24.976073  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:24.996307  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.997555  3782 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:24.989230  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:24.975883  3686 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:24.978147  2682 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:24.979564  3419 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:24.986969  3794 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:25.000478  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.000593  3794 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.001818  3790 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.001706  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.003157  3790 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.968201  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.004489  3762 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.006117  3788 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:24.983072  3783 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:25.016870  3686 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:25.012197  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:25.012603  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.011314  3536 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.022006  3541 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:25.007751  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:25.016747  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106530us
I20260709 14:51:25.010274  3770 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.026191  3536 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:25.024312  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.028216  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:25.028223  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.030066  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:25.030519  3788 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.030725  2611 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:25.031258  2611 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:25.031544  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.030681  3762 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.032053  2611 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.033288  3764 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.033815  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.034629  3787 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.018748  3765 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.033311  2611 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:25.035317  3796 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.026747  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.041330  3765 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:25.035040  3770 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:25.042459  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.043929  3783 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.044719  3783 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.046590  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.050732  3731 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:25.055462  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:25.057036  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:25.059191  2612 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:25.037269  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.061466  3597 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.072091  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:25.061650  3764 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:25.073516  3764 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:25.073907  3764 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:25.074079  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:25.076505  2612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:55874
I20260709 14:51:25.077169  2612 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:25.077576  2612 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:25.085884  2611 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:25.090534  3783 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.090567  3788 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.095252  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:25.099048  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:25.076308  3764 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:25.101045  3764 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.112167  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.113098  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.113814  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.114683  2759 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:25.113127  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.117305  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.119130  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:25.112146  3798 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:25.120745  3801 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:25.120935  3541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37160
I20260709 14:51:25.121883  3541 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:25.122478  3541 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.127872  3798 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.129149  3798 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.129805  3790 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:25.131112  3790 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:25.131522  3790 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:25.131783  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:25.132402  3790 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.133894  3790 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.143316  2759 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:25.147442  2605 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:25.148514  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.149722  3789 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.147593  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:25.143374  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:25.158531  2753 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:25.165388  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.169276  3790 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.170931  3798 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:25.155752  2612 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:25.172240  2610 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:25.172305  3788 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:25.172837  3788 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.172384  3541 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.173766  3788 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.174777  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.177608  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:25.178278  2682 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:25.178745  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:25.179499  2753 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:25.175135  3788 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.176286  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:25.179849  2753 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:25.180472  2686 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:25.180892  2686 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:25.181789  2686 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.183637  2686 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.183460  2753 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:25.184993  2753 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:25.187882  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:25.189594  3798 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.191942  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:25.176818  2612 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:25.195483  3764 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.206118  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.208094  3787 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.217103  3541 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:25.222956  2682 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:25.226408  2686 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:25.227300  2680 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:25.228300  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.229189  3801 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.229663  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:25.231143  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:25.231961  3797 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:25.232522  3797 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:25.233418  3797 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:25.238361  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50652.9us
I20260709 14:51:25.241271  2430 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:25.245169  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.247680  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.248831  3765 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 14:51:25.057869  3686 consensus_queue.cc:1175] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c1b0e692e4294ee3b8694bbada174ba1" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:51:25.252388  3794 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:25.256407  2753 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:25.253896  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 4, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:25.273953  2608 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 2: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:25.266789  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.267024  3798 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.270586  3547 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:25.283582  3788 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.278545  3794 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.285475  3765 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:25.285677  3794 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.286758  3765 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:25.287176  3765 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:25.288007  3765 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.289398  3765 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:25.291107  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.294391  3788 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:25.296077  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:25.297829  3794 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.299633  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:25.300668  3794 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.290937  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:25.303534  3794 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:25.304795  3788 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:25.305210  3788 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.305488  3765 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:25.306124  3788 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.310986  3788 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.320891  2433 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:25.324435  2753 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:25.329290  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:25.330475  2753 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:25.333630  2759 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:25.333865  3794 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:25.334908  3798 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:25.335314  3798 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.336048  3785 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:25.336238  3798 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:25.337414  3785 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:25.337750  3798 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:25.337808  3785 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:25.340031  3785 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:25.341523  3785 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:25.341872  2753 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:25.536106  2753 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:25.537643  3785 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:25.569842  2611 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:25.571799  3785 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:25.686990  2753 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:25.689736  3797 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:25.703590  2686 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:25.707868  3686 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:25.714449  2753 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:25.724646  3794 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:25.769418  2680 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:25.773697  3788 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:25.906536   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:25.970212   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:26.016892  2611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:26.017606  2611 raft_consensus.cc:606] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:26.022709   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 14:51:26.024873   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:26.062863  3760 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:26.063649  3760 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:26.071355  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:26.071857  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.072321  2682 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:26.072696  2759 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:26.074045  3760 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:26.074046  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:26.075227  3760 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:26.075675  3760 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:26.076187  3760 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:26.092092  3760 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:26.094552  3760 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:26.095305  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:26.096201  2682 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:26.096320  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:26.105662  2759 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:26.107228  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:26.108193  3686 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:26.108724  3686 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:26.109612  3686 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:26.125213  2430 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:26.144111  3760 raft_consensus.cc:993] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:26.144711  3760 raft_consensus.cc:1081] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:51:26.146620  3536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37174
I20260709 14:51:26.147305  3536 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:26.147778  3536 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:26.157550  3536 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.160241  3536 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:26.161204  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:26.161259  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:26.162075  2759 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:26.162511  2608 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:26.162858  2608 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:26.163688  2608 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.164866  2608 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:26.184535  2608 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:26.187817  2759 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:26.189222  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.190627  3788 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:26.191220  3788 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:26.192193  3788 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.207010  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:26.480813  3733 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.481482  3733 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.485013  3733 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.485221  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.486631  2753 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:26.487118  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:26.487964  3540 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:26.488188  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:26.489267  3789 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:26.489748  3789 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:26.490224  3789 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.499673  3789 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.501937  3789 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.502931  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:26.503634  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:26.504621  3540 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.504598  2753 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.514360  3540 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:26.516296  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.517261  3789 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:26.518947  2753 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:26.527388  3789 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:26.528667  3789 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.538880  2686 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:26.540689  3794 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.540885  3686 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:26.541535  3794 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.545084  3794 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:26.547788  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:26.548583  2682 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:26.547833  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:26.552150  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:26.546228  2430 catalog_manager.cc:5697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:26.554229  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.555222  3794 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:26.555642  3794 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:26.556109  3794 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.545878  3825 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.568113  3825 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.568792  3733 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.569456  3733 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:26.593027  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:26.594163  2608 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:26.595124  2759 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:26.595721  3788 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:26.604506  3815 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:26.605183  3825 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.606666  3788 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.607266  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:26.608006  3536 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:26.609212  3788 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:26.609789  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.611243  3789 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:26.611608  3789 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.611984  3789 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.613570  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:26.614317  2682 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:26.615983  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:26.616827  2611 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:26.618726  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.622031  3794 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.580334  3733 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:26.630565  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:26.631376  2611 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.640930  3815 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:26.641557  3815 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:26.642066  3815 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.640050  3789 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.596136  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:26.596954  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.645411  2757 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:26.647948  3825 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:26.648495  3825 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:26.648881  3825 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.652915  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:26.654881  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.656092  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:26.645328  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49337.2us
I20260709 14:51:26.660112  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.659936  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:26.660980  2757 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:26.662276  3788 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:26.672101  3733 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.672838  3733 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.682457  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.690056  2757 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:26.690598  2753 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:26.692605  3760 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:26.726740  3825 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:26.715260  3794 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:26.755352  3788 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:26.753119  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69486us
I20260709 14:51:26.724947  3789 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:26.762540  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63142us
I20260709 14:51:26.771631  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:26.772564  2611 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.780350  2611 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:26.759132  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:26.758478  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 100825us
I20260709 14:51:26.768548  3733 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.786547  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:26.801415  2682 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.807354  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:26.808229  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:26.813165  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:26.814567  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:26.808749  3788 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.854694  2682 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:26.803001  3782 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.807574  3825 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:26.873776  3801 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.874747  3801 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.854332  3815 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.877458  3815 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:26.863055  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48567.6us
I20260709 14:51:26.859540  3760 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:26.880851  3760 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.881754  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:26.863734  3842 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:26.883244  3842 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.883531  3801 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:26.883872  3842 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.863752  3782 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.889271  3782 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.854164  3835 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:26.870893  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:26.891268  3835 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:26.892398  3835 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:26.892904  2759 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.898319  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:26.880208  3815 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.900343  3815 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.913496  2611 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:26.918682  3760 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.919011  3547 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:26.920316  3815 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:26.921703  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.922746  3787 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:26.924551  3787 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:26.925073  3787 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.927242  2759 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:26.932709  3843 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.933697  3787 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.935214  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:26.944352  2757 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:26.944005  3843 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.945705  3787 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:26.945705  3760 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.946815  3843 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:26.946938  3760 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.947041  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:26.948930  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.949335  3760 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:26.936744  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:26.950189  2611 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:26.952781  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:26.953584  3536 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:26.954769  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.955833  3843 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:26.956231  3843 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:26.956601  3843 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:26.956838  3850 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:26.956902  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:26.958112  2608 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:26.957957  3850 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:26.968926  3547 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:26.969900  3844 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:26.971969  3844 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:26.972442  3844 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:26.990046  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:26.991154  3787 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:26.991707  3787 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:26.992589  3787 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:26.995453  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:27.003276  3831 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:27.004127  3850 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:27.010202  3801 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:27.010949  3801 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.014470  3801 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.016474  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.017412  3547 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:27.019254  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:27.022259  2433 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:27.024317  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:27.025171  2608 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:27.026291   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:27.027325  3801 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:27.027880  3801 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:27.028297  3801 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:27.037516  3842 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.040359  3842 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.041410  2682 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:27.043232  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:51:27.044091  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66778.3us
I20260709 14:51:27.043798  3844 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:27.044946  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:27.045066  3788 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:27.046922  3853 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:27.047070  3788 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:27.047611  3853 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:27.048262  3844 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:27.048601  3853 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:27.048337  3788 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:27.054322  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:27.058378  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.059342  3547 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:27.062239  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:27.061494  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:27.064036  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.064411  2608 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:27.066077  3541 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:27.063624  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.074151  3540 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:27.078137  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:27.094775  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:27.095528  2612 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:27.101333  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:27.105624  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59337.3us
I20260709 14:51:27.102418  2759 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:27.109225  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:27.115727  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:27.120990  3859 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:27.121878  3859 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.125022  2608 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:27.126924  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:27.116431  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:27.132699  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:27.144323  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:27.145910  2759 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:27.138026  3686 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:27.158651  2430 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:27.155253  3843 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:27.169323  3843 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:27.170069  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 393152us
I20260709 14:51:27.170547  3801 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.172871  3801 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.139807  3844 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:27.175433  2754 raft_consensus.cc:2368] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:27.140946  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:27.181149  3858 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 14:51:27.156788  3841 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:27.200603  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:27.193789  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86792.6us
I20260709 14:51:27.207607  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:27.208298  2680 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:27.162024  3686 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:27.248651  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:27.204088  3844 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:27.254482  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:51:27.281122  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 148519us
I20260709 14:51:27.255388  3686 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:27.299877  3859 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.490638  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:27.491462  3547 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:27.519143  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:27.474411  3537 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:27.565622  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:27.566411  2611 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 3
W20260709 14:51:27.521080  3805 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 14:51:27.566627  3547 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
W20260709 14:51:27.585429  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 66398.7us
I20260709 14:51:27.567225  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.598591  2611 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:27.505139  3844 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:27.602489  3866 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.261s	user 0.009s	sys 0.000s
W20260709 14:51:27.641400  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54531.9us
I20260709 14:51:27.641723  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:27.600893  3540 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:27.600241  3874 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:27.736003  3686 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:27.739827  3686 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.752013  2686 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:27.720613  3874 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:27.777638  3874 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:27.726908  3860 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:27.779241  3860 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:27.787766  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46222.6us
I20260709 14:51:27.787400  3882 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:27.782665  3860 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:27.843120  3876 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:27.843201  3832 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:27.843852  3876 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.846784  3876 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:27.846066  3874 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.863337  3874 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:27.865365  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 158394us
I20260709 14:51:27.867448  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:27.868724  3836 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:27.872313  3836 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:27.873413  3836 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.876379  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:27.877087  3536 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:27.879581  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:27.880283  3547 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:27.880414  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:27.881196  3537 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:27.889454  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:27.891237  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:27.895787   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:27.897578  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:27.900722  3864 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:27.901202  3864 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:27.901583  3864 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:27.907882  3536 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:27.910737  3864 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:27.916642  3864 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:27.913077  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:27.921945  3885 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:27.922565  3885 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:27.923514  3885 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.927354  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:27.931726  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 279002us
I20260709 14:51:27.938966  2610 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:27.938027  3536 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:27.940800  3815 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:27.941533  3815 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.943410  3547 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:27.943877  3815 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:27.944216  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 485039us
I20260709 14:51:27.940865  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:27.945909  3885 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:27.946440  3885 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:27.947409  3885 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:27.948625  3894 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:27.949225  3894 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.950353  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:27.934911  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.007s
W20260709 14:51:27.952834  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.006s	sys 0.007s
I20260709 14:51:27.952713  2611 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:27.954051  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:27.954202  2433 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:27.954936  3888 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:27.954703  3894 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:27.956383  3833 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:27.964766  3833 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:27.964553  3888 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:27.965863  3833 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.967978  3888 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:27.957051  3853 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:27.959194  2756 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:27.973436  3536 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:27.974064  3815 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:27.974994  3815 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.977370  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:27.978178  2758 raft_consensus.cc:2368] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:27.980166  2611 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:27.982606  3861 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:27.984035  3815 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:27.984826  3815 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:27.985378  3815 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:27.990029  2682 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:27.992444  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:28.016873  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:28.047293  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:28.047871  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:28.049328  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:28.048281  2612 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:28.050979  2611 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:28.057181  2577 leader_election.cc:400] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:28.057585  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:28.054178  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:28.063059  2577 leader_election.cc:403] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:28.044416  3844 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:28.083078  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:28.086979  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62299.6us
I20260709 14:51:28.092406  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:28.072662  3863 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:28.062923  3889 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 14:51:28.103438  3538 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:28.114998  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:28.118873  2577 leader_election.cc:400] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:28.119297  2577 leader_election.cc:403] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 14:51:28.095924  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:28.098167  3891 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:28.061398  3894 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 14:51:28.165426  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:28.167850  3898 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 14:51:28.097867  3844 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:28.123261  3863 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:28.140110  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:28.145015  3888 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:28.200385  3788 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:28.205135  3844 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:28.158591  3794 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 14:51:28.208339  2612 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:28.210592  3883 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:28.123055  3837 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 14:51:28.183859  3894 consensus_queue.cc:1243] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 14:51:28.227223  3895 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:51:28.229104  3873 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 14:51:28.232882  3815 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:28.216069  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 99170.4us
I20260709 14:51:28.213109  3879 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 14:51:28.201548  3888 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:28.252652  3888 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:28.207280  3863 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:28.324774  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:28.325919  3541 raft_consensus.cc:2368] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:28.332312  3874 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:28.333017  3874 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:28.353060  2431 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:51:28.356491  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 249453us
W20260709 14:51:28.360307  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 260791us
I20260709 14:51:28.356230  2431 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:28.369841  2431 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 4, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:28.334195  3874 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:28.398602  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 272854us
W20260709 14:51:28.405133  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:28.400929  3836 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.020s	sys 0.038s
W20260709 14:51:28.412779  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 299744us
I20260709 14:51:28.417639  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:28.426312  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:28.433132  2759 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
W20260709 14:51:28.437145  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 315718us
I20260709 14:51:28.418391  3874 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:28.465631  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:28.462241  2753 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:28.440047  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:28.445099  3837 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:28.488950  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48971.4us
I20260709 14:51:28.432591  3835 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 14:51:28.477566  3882 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:28.499109  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 400622us
I20260709 14:51:28.514045  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:28.490782  3886 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:28.513268  3882 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:28.537389  3882 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:28.547358  2576 leader_election.cc:400] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
W20260709 14:51:28.580655  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91034us
I20260709 14:51:28.589871  3547 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:28.540007  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:28.606146  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 165504us
I20260709 14:51:28.618584  3545 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:28.602635  3859 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 14:51:28.621349  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:28.632836  3896 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 14:51:28.622197  3788 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:28.675805  3547 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 14:51:28.639290  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 116395us
I20260709 14:51:28.680065  3886 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 14:51:28.686481  3845 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:28.687196  3845 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:28.635169  3836 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 14:51:28.699023  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:28.700687  3831 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:28.700704  3845 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:28.736905  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:28.741566  3910 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 14:51:28.743122  3903 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:28.743701  3903 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:28.746464  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 305617us
I20260709 14:51:28.754112  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:28.754936  3543 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:28.760368  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 270162us
I20260709 14:51:28.772281  2755 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:28.801582  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:28.821148  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:28.803975  3905 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:28.834934  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:28.763693  3845 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 14:51:28.852807  3902 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:28.744764  3903 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:28.880239  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:28.897819  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:28.922727  3846 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:28.854317  3733 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 14:51:28.842270  3905 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:28.939615  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 95157.1us
I20260709 14:51:28.943723  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:28.943995  3919 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:28.927789  3903 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:28.943832  3733 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 14:51:28.895491  3906 consensus_queue.cc:1175] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 4 status { last_received { term: 4 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 14:51:29.030761  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 221317us
I20260709 14:51:28.947067  3861 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 14:51:28.969978  3919 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.021891  3733 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:29.048511  3874 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.004s
I20260709 14:51:29.048372  3861 consensus_queue.cc:1243] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 14:51:29.091508  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 330603us
I20260709 14:51:29.099465  2756 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 14:51:29.100208  2756 pending_rounds.cc:85] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 2
I20260709 14:51:29.100654  2756 pending_rounds.cc:107] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 14:51:29.101058  2756 raft_consensus.cc:2889] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:51:29.102552  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 226159us
I20260709 14:51:29.107440  3844 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.108158  3844 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.111712  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:29.115933  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:29.107455  3905 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:29.128746  3924 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 14:51:29.128559  3844 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:29.139263  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46294.9us
I20260709 14:51:29.298028  3929 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.299166  3929 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:29.302534  3919 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:29.363036  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:29.368376  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:29.369511  3905 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:29.370311  3905 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:29.369344  3929 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:29.463464  3542 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 14:51:29.540932  3923 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.586s	user 0.003s	sys 0.001s
I20260709 14:51:29.569178  3877 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.569918  3877 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.570271  3935 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.569264  3882 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:29.570915  3935 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.571194  3882 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.572336  3877 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:29.573076  3882 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:29.573488  3935 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.582598  3686 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.583276  3686 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.585682  3686 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.590765  2686 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 14:51:29.590580  3934 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.591311  2686 pending_rounds.cc:85] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 2
I20260709 14:51:29.591594  2686 pending_rounds.cc:107] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 14:51:29.592005  2686 raft_consensus.cc:2889] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:29.591516  3934 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.594064  3934 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:29.594307  3686 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.595027  3686 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:29.595983  3863 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.596527  3863 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.597252  3686 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:29.599031  3863 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:29.596439  3887 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.600190  3887 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.603684  3887 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:29.605736  3850 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.606420  3850 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.609035  3850 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.628410  3733 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:29.629806  3733 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:29.639642  3541 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:29.639724  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:29.648136  3537 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:29.652295  3864 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:29.653764  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:29.655047  3873 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:29.655530  3873 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.658772  3873 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:29.667148  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:29.671145  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 130) took 1089 ms. Trace:
I20260709 14:51:29.671723  2611 rpcz_store.cc:276] 0709 14:51:28.581501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.581993 (+   492us) service_pool.cc:224] Handling call
0709 14:51:28.582484 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:28.583047 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:28.583092 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:28.602510 (+ 19418us) log.cc:844] Serialized 26 byte log entry
0709 14:51:28.954158 (+351648us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:28.954219 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:28.954312 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:29.540188 (+585876us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:29.670612 (+130424us) raft_consensus.cc:1685] finished
0709 14:51:29.670658 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:29.671043 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":176,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":12910,"spinlock_wait_cycles":30224640,"thread_start_us":335762,"threads_started":1,"wal-append.queue_time_us":341469}
I20260709 14:51:29.677163  3927 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.677933  3927 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:29.681005  3927 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:29.682392  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.041s
W20260709 14:51:29.682863  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.010s	sys 0.042s
I20260709 14:51:29.688735  3931 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:29.689383  3931 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:29.691094  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:29.691272  3931 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:29.691979  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:29.722285  3862 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.723176  3862 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.726965  3862 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:29.727670  3862 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.728235  3862 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.728857  3883 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.729477  3883 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.730693  3862 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.733152  3883 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:29.735970  3927 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.737165  3927 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.740648  3927 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:29.745045  3883 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:29.745903  3883 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:29.749636  3883 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:29.758332  2605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:29.759315  2605 raft_consensus.cc:606] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:29.761165  3864 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:29.761816  3864 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.765203  3864 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:29.767104  2608 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:29.769446  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:29.771546  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:29.773509  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:29.775408  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:29.776188  2612 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:29.777413   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 14:51:29.778751  3782 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:29.779371  3782 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.781682  3782 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.788198  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:29.790038  2608 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:29.789983  3863 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:29.791520  3864 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:29.791965  3864 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:29.792301  3864 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:29.793358  3863 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:29.802628  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:29.822853  3788 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:29.824654  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 122) took 1407 ms. Trace:
I20260709 14:51:29.824997  3847 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:29.825232  2756 rpcz_store.cc:276] 0709 14:51:28.417074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.423969 (+  6895us) service_pool.cc:224] Handling call
0709 14:51:28.453143 (+ 29174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:29.101776 (+648633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:29.824311 (+722535us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:29.824559 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458349824}
I20260709 14:51:29.825703  3847 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:29.830812  3847 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:29.834053  3934 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:29.843987  3911 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.844677  3911 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.891731  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:29.893667  3875 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.894345  3875 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.897104  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 616) took 1112 ms. Trace:
I20260709 14:51:29.897743  2686 rpcz_store.cc:276] 0709 14:51:28.784742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.784858 (+   116us) service_pool.cc:224] Handling call
0709 14:51:28.785126 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:29.592508 (+807382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:29.896685 (+304177us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:29.896981 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851528960}
I20260709 14:51:29.919734  2678 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 14:51:29.920337  2678 pending_rounds.cc:85] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 3
I20260709 14:51:29.920635  2678 pending_rounds.cc:107] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 14:51:29.920938  2678 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:29.922622  3864 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.924176  3835 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:29.924839  3835 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:29.928596  3860 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.929289  3860 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:29.930433  3867 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.931044  3867 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:29.937681  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:29.947163  3733 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:29.948144  3733 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:29.962637   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:29.996728  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 444) took 1055 ms. Trace:
I20260709 14:51:29.997452  3538 rpcz_store.cc:276] 0709 14:51:28.941517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.945091 (+  3574us) service_pool.cc:224] Handling call
0709 14:51:29.995501 (+1050410us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521e9c 00007fa16b521a5b 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:29.995830 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:29.996126 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:29.996164 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:29.996230 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:29.996289 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:29.996385 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:29.996597 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14355200}
I20260709 14:51:29.999905  3927 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.001735  2435 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 3 to 4, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:30.002951  2433 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 156) took 2053 ms. Trace:
I20260709 14:51:30.008466  3733 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:30.008803  2430 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 157) took 1936 ms. Trace:
I20260709 14:51:30.000810  3927 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:30.014098  2430 rpcz_store.cc:276] 0709 14:51:28.076191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.080812 (+  4621us) service_pool.cc:224] Handling call
0709 14:51:30.008729 (+1927917us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210497,"reported_tablets":2}
I20260709 14:51:30.019666  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:30.034858  2756 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:30.003371  2433 rpcz_store.cc:276] 0709 14:51:27.949984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:27.950172 (+   188us) service_pool.cc:224] Handling call
0709 14:51:30.002875 (+2052703us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:28.087536 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:28.088172 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:28.088213 (+    41us) write_op.cc:435] Acquired schema lock
0709 14:51:28.088230 (+    17us) tablet.cc:662] Decoding operations
0709 14:51:28.088583 (+   353us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:28.088658 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:28.088689 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:51:28.088878 (+   189us) tablet.cc:701] Row locks acquired
0709 14:51:28.088900 (+    22us) write_op.cc:260] PREPARE: finished
0709 14:51:28.089066 (+   166us) write_op.cc:270] Start()
0709 14:51:28.089187 (+   121us) write_op.cc:276] Timestamp: P: 1783608688089016 usec, L: 0
0709 14:51:28.089209 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 14:51:28.089976 (+   767us) log.cc:844] Serialized 540 byte log entry
0709 14:51:28.484795 (+394819us) op_driver.cc:464] REPLICATION: finished
0709 14:51:28.845727 (+360932us) write_op.cc:301] APPLY: starting
0709 14:51:29.489469 (+643742us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b600024e1c8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa171fb0161 00007fa17211c14b 00007fa17211e774 00007fa1720936db 00007fa171f90b93 00007fa171f92de8 00007fa172091e34 00007fa17207d97c 00007fa172081d61 00007fa172081d19 00007fa172081ca9
0709 14:51:29.489601 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:29.489713 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:29.489727 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:29.490019 (+   292us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:29.490240 (+   221us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 14:51:29.490297 (+    57us) write_op.cc:312] APPLY: finished
0709 14:51:29.490579 (+   282us) log.cc:844] Serialized 22 byte log entry
0709 14:51:29.490886 (+   307us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:29.490984 (+    98us) write_op.cc:454] Released schema lock
0709 14:51:29.491030 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:29.491469 (+   439us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:29.491506 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":326217,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":336488,"apply.run_cpu_time_us":6166,"apply.run_wall_time_us":652861,"num_ops":1,"prepare.queue_time_us":119117,"prepare.run_cpu_time_us":2748,"prepare.run_wall_time_us":2748,"replication_time_us":385749,"spinlock_wait_cycles":9979136,"thread_start_us":331442,"threads_started":2}]]}
I20260709 14:51:30.057366  3911 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.060279  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:30.057878  3927 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:30.061898  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:30.071480  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51946.9us
I20260709 14:51:30.072595  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:30.075071  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.080960  2431 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 162) took 1780 ms. Trace:
I20260709 14:51:30.090523  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:30.110668  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:30.119890  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:30.122166  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:30.232620  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.429352  3852 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:30.233165  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:30.267033  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:30.224447  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:30.246044  2610 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:30.255812  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.438298  2757 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:30.133857  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:30.334524  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.057s	sys 0.047s
W20260709 14:51:30.443051  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.059s	sys 0.055s
I20260709 14:51:30.443243  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.132700  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.085610  3864 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:30.205895  3885 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:30.088481  3931 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.091264  3878 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.222213  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:30.249449  3836 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:30.453574  3835 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:30.278427  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:30.137741  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:30.431083  3883 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:30.126855  3934 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.431362  3875 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:30.213100  3862 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.407442  3840 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:30.377723  3788 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:30.392421  3846 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.205574  3937 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.232313  3861 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.453344  3908 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.401973  3867 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:30.454187  3852 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:30.474399  3942 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 14:51:30.465883  3879 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.470636  3837 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.475744  3911 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.781975  3937 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.782251  3846 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:30.781318  3934 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.665654  3860 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:30.789206  3883 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.792513  3864 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.774605  3840 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.771930  3862 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:30.566890  3931 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.566632  3878 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:30.818660  3863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.094s	sys 0.152s
I20260709 14:51:30.779299  3861 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:30.778321  3908 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:30.849318  3951 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 14:51:30.851934  3908 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:30.843204  3953 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:30.853318  3953 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.855676  3953 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.856262  3953 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:30.856769  3953 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:30.861373  3953 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:30.866554  3931 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.867641  3934 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.869072  3937 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.086799  2431 rpcz_store.cc:276] 0709 14:51:28.300491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.300675 (+   184us) service_pool.cc:224] Handling call
0709 14:51:30.080850 (+1780175us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:28.458944 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:28.546340 (+ 87396us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:28.548781 (+  2441us) write_op.cc:435] Acquired schema lock
0709 14:51:28.553618 (+  4837us) tablet.cc:662] Decoding operations
0709 14:51:28.563067 (+  9449us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:28.563132 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:28.563168 (+    36us) tablet.cc:685] Acquiring locks for 3 operations
0709 14:51:28.563570 (+   402us) tablet.cc:701] Row locks acquired
0709 14:51:28.563596 (+    26us) write_op.cc:260] PREPARE: finished
0709 14:51:28.563776 (+   180us) write_op.cc:270] Start()
0709 14:51:28.563918 (+   142us) write_op.cc:276] Timestamp: P: 1783608688563718 usec, L: 0
0709 14:51:28.563942 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 14:51:28.817129 (+253187us) log.cc:844] Serialized 1316 byte log entry
0709 14:51:28.842077 (+ 24948us) op_driver.cc:464] REPLICATION: finished
0709 14:51:29.039071 (+196994us) write_op.cc:301] APPLY: starting
0709 14:51:29.039195 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:29.039344 (+   149us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:29.094234 (+ 54890us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:29.094966 (+   732us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:29.095235 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 14:51:29.095307 (+    72us) write_op.cc:312] APPLY: finished
0709 14:51:29.095634 (+   327us) log.cc:844] Serialized 34 byte log entry
0709 14:51:29.095879 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:29.096017 (+   138us) write_op.cc:454] Released schema lock
0709 14:51:29.096081 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:29.595361 (+499280us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54001130d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e641e28 00007fa17207c581 00007fa17207bd58 00007fa1720809f1 00007fa1720809a9 00007fa172080939 00007fa172080901 00007fa17207fb2d 0000000000396ad4 0000000000396919
0709 14:51:29.595605 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:29.595710 (+   105us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150858,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":192900,"apply.run_cpu_time_us":3255,"apply.run_wall_time_us":57535,"num_ops":3,"prepare.queue_time_us":328,"prepare.run_cpu_time_us":14001,"prepare.run_wall_time_us":1158380,"replication_time_us":274074,"spinlock_wait_cycles":416802560,"thread_start_us":114745,"threads_started":1}]]}
W20260709 14:51:30.901329  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.002s	sys 0.007s
W20260709 14:51:30.902137  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.003s	sys 0.007s
W20260709 14:51:30.904286  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.000s	sys 0.019s
W20260709 14:51:30.905157  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.019s
I20260709 14:51:30.138384  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:30.908447  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.004s	sys 0.006s
W20260709 14:51:30.909210  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.004s	sys 0.006s
I20260709 14:51:30.911492  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:30.912535  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:30.914916  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:30.916812  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 654) took 1118 ms. Trace:
I20260709 14:51:30.917461  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:30.919432  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:30.917695  2608 rpcz_store.cc:276] 0709 14:51:29.798607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:29.798777 (+   170us) service_pool.cc:224] Handling call
0709 14:51:29.799211 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:29.800028 (+   817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:29.800065 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:29.800634 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 14:51:30.443689 (+643055us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:30.443755 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:30.443861 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:30.915951 (+472090us) raft_consensus.cc:1685] finished
0709 14:51:30.916024 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:30.916525 (+   501us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":642502,"threads_started":1,"wal-append.queue_time_us":892555}
I20260709 14:51:30.920320  2604 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:30.914779  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:30.939603  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 485586us
W20260709 14:51:30.946331  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:30.775099  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.032s	sys 0.039s
I20260709 14:51:30.648303  2752 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
W20260709 14:51:30.951584  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.032s	sys 0.040s
I20260709 14:51:30.952284  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:30.668298  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08002422a0 after lost signal to thread 434
W20260709 14:51:30.954749  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.015s	sys 0.023s
W20260709 14:51:30.955497  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.016s	sys 0.023s
W20260709 14:51:30.956080  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 843891us
W20260709 14:51:30.956696  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08000ada80 after lost signal to thread 2469
W20260709 14:51:30.960177  2781 debug-util.cc:398] Leaking SignalData structure 0x7b0800305a20 after lost signal to thread 2636
I20260709 14:51:30.820978  3837 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:30.977499  3852 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:30.977927  3958 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 14:51:30.779875  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.011s	sys 0.026s
I20260709 14:51:30.978206  3861 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:30.977988  3835 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:30.978359  3853 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:30.997624  2604 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:31.003398  3782 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.007479  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 346) took 1120 ms. Trace:
I20260709 14:51:31.007777  3782 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:30.995792  3835 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.008181  3545 rpcz_store.cc:276] 0709 14:51:29.886535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:29.889219 (+  2684us) service_pool.cc:224] Handling call
0709 14:51:29.889801 (+   582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:29.890298 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:29.890341 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:30.064662 (+174321us) log.cc:844] Serialized 26 byte log entry
0709 14:51:30.466073 (+401411us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:30.466125 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:30.466201 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:30.980363 (+514162us) raft_consensus.cc:1685] finished
0709 14:51:30.983868 (+  3505us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.007158 (+ 23290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":191,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":134,"thread_start_us":355599,"threads_started":1,"wal-append.queue_time_us":614827}
I20260709 14:51:31.012042  3835 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:30.996608  3853 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:30.828857  3911 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.019883  3911 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:31.020745  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.011s	sys 0.026s
W20260709 14:51:31.021457  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 796998us
W20260709 14:51:30.843694  3863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.095s	sys 0.153s
W20260709 14:51:31.028702  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44728.1us
I20260709 14:51:30.994710  3840 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:30.789911  3788 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.040895  3875 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.042124  3825 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.043164  3958 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:30.837857  3864 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.057546  3883 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:31.038354  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50701.5us
I20260709 14:51:30.997339  3862 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:30.819764  3879 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.067656  3788 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.067935  3878 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:31.070792  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111112us
I20260709 14:51:31.075013  3837 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.077543  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.079763  3825 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.080592  3875 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.084184  3860 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.059298  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.085364  3860 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.090739  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.062913  3846 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.058703  3864 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.086426  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.099090  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.099145  3977 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.098868  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.104928  3546 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:31.106909  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.093634  3861 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.108425  3861 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.107172  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.112089  3545 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:31.104480  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.122721  3966 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:31.123620  3966 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.114034  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.102749  3963 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:31.115399  2577 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.532s (SENT) [suppressed 362 similar messages]
I20260709 14:51:31.104480  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.116710  3543 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:31.094630  3853 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.116976  3908 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.117748  3894 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.117691  3934 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:31.114059  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.124356  3966 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.129573  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.134325  3879 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.136245  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.145139  3875 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.145615  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.145989  3835 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.147121  3864 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.148010  3894 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.149827  3861 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:31.142875  2612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.001s	sys 0.016s
I20260709 14:51:31.165382  3885 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.141760  3825 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.139184  3963 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:31.174993  2612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.002s	sys 0.016s
I20260709 14:51:31.155994  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.178429  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.179822  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 656) took 1359 ms. Trace:
I20260709 14:51:31.179431  3864 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.180455  3863 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.180953  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.181242  3863 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.180305  2612 rpcz_store.cc:276] 0709 14:51:29.820632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:29.821234 (+   602us) service_pool.cc:224] Handling call
0709 14:51:29.821546 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:29.821763 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:51:30.269535 (+447772us) log.cc:844] Serialized 14 byte log entry
0709 14:51:31.179203 (+909668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:31.179355 (+   152us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:31.179506 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:31.179597 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.179764 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93085,"prepare.queue_time_us":998467,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":169406,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":6701,"thread_start_us":1260641,"threads_started":2}
W20260709 14:51:31.183941  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76977.1us
I20260709 14:51:31.183051  3836 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.182536  3864 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:31.193032  3873 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.514s	user 0.023s	sys 0.015s
I20260709 14:51:31.195840  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 458) took 1055 ms. Trace:
I20260709 14:51:31.196466  3538 rpcz_store.cc:276] 0709 14:51:30.140702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.205273 (+ 64571us) service_pool.cc:224] Handling call
0709 14:51:31.135487 (+930214us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:31.195720 (+ 60233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10836992}
I20260709 14:51:31.197625  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.198665  3860 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.199690  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.202127  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:31.203547  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52865.5us
I20260709 14:51:31.173607  3958 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.203727  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 351) took 1144 ms. Trace:
W20260709 14:51:31.245388  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 130067us
I20260709 14:51:31.187692  3977 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.193280  3835 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.278546  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.188835  3908 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.195896  3873 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.280653  3836 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.289439  3863 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:31.287834  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53968.6us
I20260709 14:51:31.297574  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:31.303153  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.303587  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:31.303907  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 110106us
I20260709 14:51:31.231087  3885 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.305083  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:31.296746  2686 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:31.306502  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.297995  2759 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:31.300196  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:31.283525  3934 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.235414  3537 rpcz_store.cc:276] 0709 14:51:30.059141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.063985 (+  4844us) service_pool.cc:224] Handling call
0709 14:51:31.202393 (+1138408us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:31.202671 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:31.203112 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:31.203155 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:31.203240 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:31.203318 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:31.203395 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.203542 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11200384}
I20260709 14:51:31.318866  3965 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 14:51:31.324059  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.324299  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.328375  3887 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.329017  3887 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.329556  3887 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.329988  3835 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.319677  3863 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.330956  3863 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.331116  3966 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.331476  3863 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.337779  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.339584  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.302582  3957 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:31.318846  3936 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.347226  3957 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:31.333446  3963 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.334095  3965 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:31.345547  3789 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.340399  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:31.286355  3958 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.316357  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.364647  3875 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.385661  3860 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.383582  3789 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.400425  3789 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.411191  3958 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:31.411972  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 90098.9us
I20260709 14:51:31.413265  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.415151  3963 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.347440  3936 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.383344  3965 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:31.427210  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43720.1us
W20260709 14:51:31.431612  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47656.6us
I20260709 14:51:31.328440  3934 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.428242  3988 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.416699  3885 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.435604  3836 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.427290  3537 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:31.437920  3908 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.439136  3875 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.440702  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.440829  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.419966  3977 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.450464  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.454288  3965 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.459172  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.459443  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.442337  3957 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.464321  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.467115  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.447191  3958 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.463033  3988 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.474866  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.477830  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.480176  3537 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:51:31.485203  3934 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.485723  3894 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.489810  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:31.447259  3860 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.516244  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.470860  2610 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:31.496320  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.486143  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:31.504733  3836 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.008069   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:31.513309  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.513942  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:31.514312  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08003107a0 after lost signal to thread 2710
I20260709 14:51:31.514914  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.520876  3860 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.489809  3875 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.495289  3958 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.534574  3965 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:31.566473  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.567392  2758 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:31.568714  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.569918  3789 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.569989  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.571089  2751 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:31.530968  3836 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:31.572379  3985 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 14:51:31.576930  3988 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.578893  3836 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.579847  3863 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.582437  3863 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:31.571642  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85692.5us
I20260709 14:51:31.572322  3972 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.531606  3835 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.588356  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.588748  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.589481  2609 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:31.589810  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:51:31.590276  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:31.591127  3873 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.591557  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.593149  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.593837  2758 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:31.594815  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.596467  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.603363  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:31.604529  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:31.608170  3835 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:31.608592  3835 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.609002  3835 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:31.608943  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.573004  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 627) took 1506 ms. Trace:
I20260709 14:51:31.619241  2685 rpcz_store.cc:276] 0709 14:51:30.075986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.083001 (+  7015us) service_pool.cc:224] Handling call
0709 14:51:30.688780 (+605779us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:30.689178 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:30.956028 (+266850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:30.956106 (+    78us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:30.956190 (+    84us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:30.956269 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:31.572426 (+616157us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:31.572492 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.572726 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628708480}
I20260709 14:51:31.621057  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.621882  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.610172  3963 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.623397  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:31.624019  2781 debug-util.cc:398] Leaking SignalData structure 0x7b080032d180 after lost signal to thread 2784
I20260709 14:51:31.623600  3963 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:31.624538  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 515) took 1684 ms (client timeout 1532 ms). Trace:
I20260709 14:51:31.639191  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.640621  2757 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:31.642139  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.644050  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.602088  3936 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.643101  2751 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:31.651289  3958 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.653613  3782 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.654392  3977 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:31.654786  3733 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.655269  3977 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.655880  3733 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.659660  3977 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.660454  3733 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.668298  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.716128  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:31.625077  2678 rpcz_store.cc:269] 0709 14:51:30.058480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.083053 (+ 24573us) service_pool.cc:224] Handling call
0709 14:51:31.623084 (+1540031us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:31.624451 (+  1367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18325248}
I20260709 14:51:31.644047  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.731601  2608 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:31.655270  3852 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:31.737115  3919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.014s	sys 0.016s
I20260709 14:51:31.744203  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:31.751001  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.748956  2757 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:31.754990  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.756506  3847 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.757053  3847 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:31.758268  3989 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 14:51:31.760815  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 93239.7us
I20260709 14:51:31.761871  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:31.761726  3835 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.763109  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.764794  3972 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:31.765021  3875 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:31.765116  3835 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.765394  3972 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.765846  3972 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:31.765939  3835 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:31.766412  3835 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.766798  3835 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.767495  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.768841  3867 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.769529  3867 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.771323  3887 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.772131  3867 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.773761  3887 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.775324  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.776561  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.776737  3835 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:31.777819  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.739516  3861 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.779335  3861 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.779697  3835 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:31.779989  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.781512  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.781787  3861 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.782313  3861 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:31.782461  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.782778  3861 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:31.783517  3902 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.783756  3789 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.783972  3902 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.784417  3902 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.764849  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:31.762037  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.757436  3847 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:31.753420  3919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.472s	user 0.014s	sys 0.018s
I20260709 14:51:31.788998  2757 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:31.750555  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.792079  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:31.796212  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:31.787918  3852 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.801604  3963 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:31.802201  3957 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:31.793099  3991 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.803689  3852 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.806001  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.806906  2760 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:31.807364  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.813603  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:31.802707  3957 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.812474  3875 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.817180  3957 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.818547  3875 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.819129  3875 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.815886  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:31.812819  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08001688c0 after lost signal to thread 3580
W20260709 14:51:31.822350  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67512.8us
I20260709 14:51:31.808380  3782 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 14:51:31.824225  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 131) took 2155 ms (client timeout 1785 ms). Trace:
I20260709 14:51:31.824425  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.825691  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:31.817915  3959 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:31.831315  2755 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:31.832146  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.833170  2607 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:31.831034  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.821900  3981 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:31.824724  2759 rpcz_store.cc:269] 0709 14:51:29.666605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:29.666759 (+   154us) service_pool.cc:224] Handling call
0709 14:51:31.817761 (+2151002us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:31.821794 (+  4033us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3725240704}
W20260709 14:51:31.825964  2781 debug-util.cc:398] Leaking SignalData structure 0x7b0800168760 after lost signal to thread 3581
I20260709 14:51:31.834275  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.829185  3952 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:31.838560  3850 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:31.834393  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.842676  2604 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.848364  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.849157  2606 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:31.835320  3959 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.854246  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.842221  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:31.825791  3782 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.841738  2760 raft_consensus.cc:1217] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Deduplicated request from leader. Original: 3.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 14:51:31.842219  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.841444  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.853972  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.857512  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 132) took 1644 ms. Trace:
I20260709 14:51:31.858150  3927 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:31.859261  3979 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:31.905323  3979 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:31.905790  3979 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:31.846495  3850 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:31.859566  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.911087  3850 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.863029  3959 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:31.863999  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 356) took 1792 ms. Trace:
I20260709 14:51:31.845086  3952 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.868438  3852 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:31.872884  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.896955  2606 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:31.898176  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:31.923380  3993 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 14:51:31.877689  2760 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 14:51:31.903223  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.904222  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.908017  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.908138  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.907074  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.900180  2611 rpcz_store.cc:276] 0709 14:51:30.213427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.213576 (+   149us) service_pool.cc:224] Handling call
0709 14:51:30.213975 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:30.916405 (+702430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:30.916462 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:30.916526 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:30.916600 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:31.854930 (+938330us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:31.854979 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.857305 (+  2326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613730688}
I20260709 14:51:31.914970  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.917543  3847 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.919247  2604 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:31.899777  3782 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.926342  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:31.913626  3864 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.921355  3952 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:31.919222  3908 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:31.931054  3864 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:31.934363  3908 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:31.935672  3852 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:31.936204  3852 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:31.936597  3852 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:31.936605  3995 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:31.937439  3864 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:31.939527  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.925252  3902 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:31.947120  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70685.8us
I20260709 14:51:31.948486  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:31.948841  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.951516  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:31.953912  3981 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.954981  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.956225  3927 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:31.960451  3847 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:31.937527  3995 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:31.966797  3957 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:31.964289  3961 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 14:51:31.973650  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:31.974020  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:31.974375  3961 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:31.974503  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:31.975015  3961 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:51:31.975430  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 675) took 1697 ms (client timeout 1704 ms). Trace:
I20260709 14:51:31.981472  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.983475  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.915541  3540 rpcz_store.cc:276] 0709 14:51:30.071859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.083459 (+ 11600us) service_pool.cc:224] Handling call
0709 14:51:31.441596 (+1358137us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:31.840535 (+398939us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521e9c 00007fa16b521a5b 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:31.840809 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:31.841084 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:31.841113 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:31.854810 (+ 13697us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:31.863568 (+  8758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:31.863650 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:31.863810 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40551040}
I20260709 14:51:31.991113  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.995067  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:31.995893  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:31.981261  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:31.975906  2753 rpcz_store.cc:269] 0709 14:51:30.278007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:31.344013 (+1066006us) service_pool.cc:224] Handling call
0709 14:51:31.974189 (+630176us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:31.975359 (+  1170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9881984}
I20260709 14:51:32.002403  3878 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:32.004513  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.004540  2686 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:31.965634  3972 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.013381  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.013878  2605 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:31.975978  3961 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.003520  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:32.021972  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 461) took 1776 ms (client timeout 1926 ms). Trace:
I20260709 14:51:32.026942  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:32.028960  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 97519.8us
W20260709 14:51:32.022508  3541 rpcz_store.cc:269] 0709 14:51:30.245964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.249455 (+  3491us) service_pool.cc:224] Handling call
0709 14:51:31.145475 (+896020us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:31.999560 (+854085us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521e9c 00007fa16b521a5b 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:32.021888 (+ 22328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51084800}
I20260709 14:51:32.041026  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.036840  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 133) took 1813 ms. Trace:
I20260709 14:51:32.053596  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.055047  3902 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.056180  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.057838  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:32.044231  3862 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:32.057801  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:32.064543  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 663) took 2201 ms. Trace:
I20260709 14:51:32.049642  3961 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.065239  2755 rpcz_store.cc:276] 0709 14:51:30.059145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.071250 (+ 12105us) service_pool.cc:224] Handling call
0709 14:51:30.222636 (+151386us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:31.830924 (+1608288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:31.831805 (+   881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:32.053326 (+221521us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54002e22d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:32.058518 (+  5192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3833128704}
I20260709 14:51:32.070340  3957 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:32.072604  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:32.073439  2611 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.073884  3850 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.048415  2605 rpcz_store.cc:276] 0709 14:51:30.221997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.233043 (+ 11046us) service_pool.cc:224] Handling call
0709 14:51:32.013194 (+1780151us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:32.013546 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:32.019400 (+  5854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:32.035160 (+ 15760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10220928}
I20260709 14:51:32.081357  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.083269  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.083521  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.086445  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.058032  3965 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:32.084205  2682 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:32.036109  3995 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.089949  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.068473  3862 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.099176  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.103593  3547 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:32.109062  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.113183  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.116815  2435 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 94) took 4000 ms. Trace:
I20260709 14:51:32.119673  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.117251  2435 rpcz_store.cc:276] 0709 14:51:28.091949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:28.092062 (+   113us) service_pool.cc:224] Handling call
0709 14:51:32.085492 (+3993430us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:30.006816 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:30.007425 (+   609us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:30.007481 (+    56us) write_op.cc:435] Acquired schema lock
0709 14:51:30.007504 (+    23us) tablet.cc:662] Decoding operations
0709 14:51:30.007900 (+   396us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:30.007965 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:30.007994 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:51:30.008202 (+   208us) tablet.cc:701] Row locks acquired
0709 14:51:30.008220 (+    18us) write_op.cc:260] PREPARE: finished
0709 14:51:30.008378 (+   158us) write_op.cc:270] Start()
0709 14:51:30.008516 (+   138us) write_op.cc:276] Timestamp: P: 1783608690008327 usec, L: 0
0709 14:51:30.008542 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 14:51:30.009403 (+   861us) log.cc:844] Serialized 540 byte log entry
0709 14:51:30.266177 (+256774us) op_driver.cc:464] REPLICATION: finished
0709 14:51:30.837818 (+571641us) write_op.cc:301] APPLY: starting
0709 14:51:30.840418 (+  2600us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:30.846036 (+  5618us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:30.851074 (+  5038us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:30.851347 (+   273us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:30.851567 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 14:51:30.851615 (+    48us) write_op.cc:312] APPLY: finished
0709 14:51:30.851829 (+   214us) log.cc:844] Serialized 22 byte log entry
0709 14:51:30.852025 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:30.852123 (+    98us) write_op.cc:454] Released schema lock
0709 14:51:30.852168 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:30.852544 (+   376us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:30.852576 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14586,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":556957,"apply.run_cpu_time_us":2137,"apply.run_wall_time_us":19686,"num_ops":1,"prepare.queue_time_us":136,"prepare.run_cpu_time_us":3006,"prepare.run_wall_time_us":5651,"replication_time_us":249014,"thread_start_us":211345,"threads_started":1}]]}
I20260709 14:51:32.120194  3850 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:32.101004  3952 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.116829  3883 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 14:51:32.123829  3883 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.124185  3883 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.124297  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.125037  3542 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:32.125121  3883 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.125347  3875 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.126469  3883 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.127020  3862 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:32.127701  2607 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.130051  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:32.131129  3927 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:32.131646  3927 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:32.131937  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.132649  3927 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.133440  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.133682  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.137619  3908 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:32.106681  3987 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:32.138550  3979 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.140034  2686 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:32.141186  3979 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.143824  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:32.145313  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.146698  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.147833  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.148423  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.149755  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.145983  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.151467  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.152220  2683 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:32.152649  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.137419  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.143656  3959 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.138732  3987 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.158735  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.134654  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.115434  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:32.164855  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:32.177091  2607 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:32.178959  3972 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:32.180071  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.107990  3965 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.186057  3965 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.113478  3934 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:32.187376  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78701.9us
I20260709 14:51:32.187728  3934 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.188282  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.142400  3979 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:32.210214  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.203791  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.210283  2684 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.203274  2608 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:32.214998  2611 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:32.216150  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.220199  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.216813  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:32.203192  3972 raft_consensus.cc:2764] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:51:32.220506  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:32.239054  3859 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:32.240476  4002 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:32.241199  4002 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.243002  3787 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 14:51:32.246606  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:32.202752  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.248358  3987 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:32.248880  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.219872  3541 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:32.232988  3958 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:32.254398  3958 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:32.249186  3987 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:32.262538  3958 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.265806  3959 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.267944  2603 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:32.269560  3733 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.258272  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.271654  2686 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:32.277762  3733 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:32.277160  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.279863  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.280323  2760 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.254496  3965 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:32.273669  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54074us
I20260709 14:51:32.282182  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.236863  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.294305  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.292447  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.295794  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:32.297889  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77493.6us
I20260709 14:51:32.274035  2435 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 5, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:51:32.299335  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:32.303895  3997 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 14:51:32.304203  3952 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:32.305454  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.305828  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 112) took 1836 ms. Trace:
I20260709 14:51:32.307798  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.308486  2759 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:32.304234  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.309514  2757 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.310891  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.314003  2435 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:51:32.285682  3995 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.319907  3967 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.306262  2679 rpcz_store.cc:276] 0709 14:51:30.469292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:30.469462 (+   170us) service_pool.cc:224] Handling call
0709 14:51:30.469923 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:30.470394 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:30.470441 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:30.487100 (+ 16659us) log.cc:844] Serialized 26 byte log entry
0709 14:51:31.148264 (+661164us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b4800541158. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:51:31.959505 (+811241us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18002a6a00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:51:32.060510 (+101005us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:32.060561 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:32.060659 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:32.305379 (+244720us) raft_consensus.cc:1685] finished
0709 14:51:32.305409 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:32.305590 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200807,"raft.run_cpu_time_us":3555,"raft.run_wall_time_us":456514,"spinlock_wait_cycles":10903680,"thread_start_us":66283,"threads_started":1,"wal-append.queue_time_us":1397678}
I20260709 14:51:32.323107  3733 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.330633  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.331545  2758 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.315564  3887 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 14:51:32.333739  3801 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:32.334434  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.317829  3955 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:32.329519  2683 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:32.331728  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.316175  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.343407  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.344921  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.346557  3733 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.284819  3825 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 14:51:32.350618  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.354887  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.303045  3852 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.337275  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.303376  3965 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.298965  3987 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.334439  3801 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:32.360658  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:51:32.342011  3998 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 14:51:32.393059  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:32.328276  3782 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.372021  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.338909  3934 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:32.374009  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:32.379537  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.381726  3878 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:32.384033  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.398818  2681 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:32.387686  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.392431  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:32.339579  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.334810  3887 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.341939  4002 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:32.331399  3875 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.395498  2678 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:32.352885  3861 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:32.416193  3538 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:32.416428  3861 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.417560  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.400942  3782 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:32.391950  3801 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.422045  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.402159  3878 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.423130  3878 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.424180  3878 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.406262  3965 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.425629  3878 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.395943  4004 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.428372  3861 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:32.407188  2760 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:32.407100  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:32.408129  2755 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.366768  3955 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:32.408063  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77542.4us
I20260709 14:51:32.389289  3979 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:32.439623  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79114.5us
I20260709 14:51:32.441198  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:32.442730  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.443934  3540 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:51:32.444895  3873 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:32.445078  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.445565  3873 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:32.446084  3873 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:32.447502  4008 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 14:51:32.410589  3887 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:32.449391  3887 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.394054  3927 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:32.455386  3927 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:32.456054  2682 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
W20260709 14:51:32.457000  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69467.8us
I20260709 14:51:32.460060  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:32.457077  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:32.463166  2759 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:32.463830  2754 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:32.465095  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.458313  3955 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.466117  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:32.465226  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.468958  2760 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:32.465878  3860 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.471807  4007 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:32.472486  4007 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:32.473521  4007 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.474304  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.475463  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.476884  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.478278  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.480024  2607 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.482328  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.484133  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.450735  3887 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.486290  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.458510  3850 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:32.531219  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44286.2us
I20260709 14:51:32.522432  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.522698  4005 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:32.523039  3967 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:32.523494  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.524061  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:32.536870  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.524664  2604 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:32.524559  3801 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.539150  3787 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:32.539580  3787 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:32.539973  3787 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:32.541543  2759 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.466369  3972 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 14:51:32.542984  3972 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.543368  3972 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.485239  3957 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:32.544504  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:32.522490  3965 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
W20260709 14:51:32.519948  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55928.1us
I20260709 14:51:32.529017  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.488040  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:32.487273  2610 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:32.521060  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.458532  3982 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:32.535056  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 68771.8us
I20260709 14:51:32.519412  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.545778  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.554119  2611 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:32.482703  2603 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:32.567715  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:32.491386  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:32.487274  3981 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.567330  3787 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.570910  3959 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:32.571200  3987 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:32.571514  3959 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:32.550163  3801 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:32.556195  3934 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:32.572242  3965 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:32.544710  3860 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 14:51:32.572666  3965 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:32.572885  3999 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:32.573352  3860 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:32.573881  3860 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:32.574064  3999 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:32.573722  3965 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.574517  3999 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.575433  4005 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:32.584937  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.574981  3860 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.569655  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:32.576421  3952 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:51:32.579689  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 91733.2us
I20260709 14:51:32.579838  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:32.580293  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:32.583199  4005 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:32.599033  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:32.571943  3959 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.604815  3902 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 14:51:32.607774  3952 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:32.594161  3850 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:32.608125  3902 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:32.608269  3952 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:32.608651  3902 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:32.609421  3952 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.609632  3902 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:32.611198  4005 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.612653  4005 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:32.591790  3875 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:32.605223  4004 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 14:51:32.616189  3873 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.616773  4002 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:32.605504  3934 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:32.617573  3860 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:32.617254  4002 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:32.636802  4002 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.618877  3852 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:32.621276  3936 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:32.621272  3902 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:32.621869  3934 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.628274  3883 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.603436  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:32.630748  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:32.621587  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:32.647070  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43698.3us
I20260709 14:51:32.647984  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:32.544224  3972 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.650315  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.654428  2610 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 14:51:32.640761  3961 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.005s
I20260709 14:51:32.629926  3873 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:32.665098  4002 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.666126  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:32.667474  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:32.668205  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.638038  3965 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:32.670667  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 67313.3us
I20260709 14:51:32.638125  3850 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.651695  3952 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.580092  3927 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.661329  3967 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:32.681594  2759 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.678758  2604 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
W20260709 14:51:32.687681  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 102803us
I20260709 14:51:32.695946  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.655635  3987 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
W20260709 14:51:32.703373  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110859us
I20260709 14:51:32.706135  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.678618  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:32.679082  3961 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 14:51:32.695271  3977 consensus_queue.cc:1175] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:51:32.697643  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:32.680116  2605 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:32.675005  3979 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 14:51:32.683547  3967 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:32.728914  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:32.682816  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 271125us
I20260709 14:51:32.681007  3875 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.739603  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.747301  3875 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:32.751008  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.674289  3999 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.756724  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.759187  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.760432  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.761909  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.762136  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.742151  3967 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:32.765164  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.734144  3852 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 14:51:32.768940  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.759637  3801 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.763756  3859 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.773645  3850 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:32.764134  3919 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:32.724702  4015 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 14:51:32.767606  3825 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.799894  3999 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:32.770591  3908 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.766414  3864 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.770551  3957 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.769495  3847 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.798575  4016 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:32.811437  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:32.866796  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 127307us
I20260709 14:51:32.877475  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.856669  3537 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 14:51:32.857362  3878 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
W20260709 14:51:32.913705  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 199324us
I20260709 14:51:32.886410  3537 pending_rounds.cc:85] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
W20260709 14:51:32.873009  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 125021us
I20260709 14:51:32.879532  3959 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:32.908876  3787 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:32.927311  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:32.926515  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:32.929872  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52528.1us
I20260709 14:51:32.930706  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.923030  3961 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:32.921906  3537 pending_rounds.cc:107] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:51:32.934161  3537 raft_consensus.cc:2889] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:32.934974  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:32.954511  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:32.926008  4017 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:32.967739  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 195978us
W20260709 14:51:32.981542  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 212020us
W20260709 14:51:32.982496  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 238116us
I20260709 14:51:32.983903  2682 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:32.958915  4005 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:32.989758  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:32.992869  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 62269.1us
I20260709 14:51:32.993803  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.993803  3547 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:32.997612  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.994645  4029 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.000134  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:32.959595  3961 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:33.005267  2752 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:33.006088  3859 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:32.958062  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:33.010234  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52337.8us
I20260709 14:51:33.011235  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.002327  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.017304  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:33.017583  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.018170  3733 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:33.018540  3733 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:33.018940  3733 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.018729  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:33.018883  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.023058  4023 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.024677  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.000965  4017 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:33.024704  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:33.025646  3847 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.022156  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.025820  4034 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:33.016288  3825 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:33.019500  3875 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.043650  3801 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:33.021257  3987 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.028697  4030 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 14:51:33.027962  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.038021  3860 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.055941  4026 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.057621  4025 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:33.059202  4016 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:33.059654  4016 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:33.060034  4016 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:33.063957  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:33.066287  3733 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.039642  4034 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:33.068292  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43707.6us
I20260709 14:51:33.069319  4026 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:33.071647  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48877.9us
I20260709 14:51:33.072569  3543 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:33.095240  4026 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:33.096872  3733 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:33.076608  3959 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:33.082417  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:33.096405  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74341.2us
I20260709 14:51:33.076723  4034 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.086556  3972 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.097043  4026 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:51:33.105196  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.105248  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.115972  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.116593  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:33.126528  3878 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.126691  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.131927  2604 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:33.135206  2609 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:33.132910  3847 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 14:51:33.142975  2607 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:33.144114  4007 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:33.144609  4007 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:33.145520  4007 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.147960  3860 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:33.151706  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.153048  4024 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:33.153496  4024 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:33.156515  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:33.158365  2608 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:33.159655  4024 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:33.161248  4005 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:33.157065  3847 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:33.162420  4005 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:33.163216  4002 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 14:51:33.167557  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45994.1us
I20260709 14:51:33.170647  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:33.171463  2679 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.146672  4007 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:33.185789  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.187359  2751 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:33.183692  4000 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:33.167924  4024 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.168051  4005 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:33.196815  3936 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:33.151654  3538 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
W20260709 14:51:33.195465  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74272.6us
I20260709 14:51:33.205840  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.209589  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.166942  3847 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.223531  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.201725  3999 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:33.214180  3541 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:33.204623  3936 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:33.224647  4030 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.213130  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.206359  4000 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:33.231801  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.216216  3902 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.214375  3850 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 14:51:33.240609  4018 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.367s	user 0.002s	sys 0.002s
I20260709 14:51:33.240902  4000 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.243978  4000 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.240635  3999 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:33.246961  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.248346  3999 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:33.248852  3543 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:33.248915  3999 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.261119  3967 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.263055  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.266069  2679 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:33.268661  3967 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:33.245468  4005 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.272640  2610 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:33.287750  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:33.289255  4034 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.294001  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:33.292477  4034 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:33.315291  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:33.340662  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:33.351800  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.353350  4004 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.354068  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:33.355103  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:33.374262  2607 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 14:51:33.362522  4037 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 14:51:33.377866  3936 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:33.378703  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.381783  3972 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.384199  3958 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:33.370642  2757 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.375272  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.391435  3952 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.393193  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.379698  3999 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.390197  2606 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:33.397722  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:33.398721  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 212956us
I20260709 14:51:33.406775  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:33.406867  3537 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:33.404666  3952 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:33.398891  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.406553  3847 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:33.414047  3936 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.415768  2757 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:33.417076  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.420717  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.421805  3958 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.421918  2680 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:33.422853  3934 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:33.417320  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 144) took 1136 ms. Trace:
I20260709 14:51:33.423281  3934 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:33.423519  2605 rpcz_store.cc:276] 0709 14:51:32.281344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:32.297622 (+ 16278us) service_pool.cc:224] Handling call
0709 14:51:33.414769 (+1117147us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400135454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:33.417227 (+  2458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756483328}
I20260709 14:51:33.424289  3934 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.397424  3859 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 14:51:33.409966  4034 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.420244  3908 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:33.428503  2757 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:33.431883  3864 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 14:51:33.436611  3859 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:33.418219  3883 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 14:51:33.437083  3859 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:33.433578  3972 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 14:51:33.422025  3875 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 14:51:33.438097  3859 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.441538  3864 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:33.442148  3859 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.450510  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:33.447085  3934 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.455703  4019 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:33.457108  4019 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:33.456694  3801 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 14:51:33.458209  4019 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.463757  3801 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:33.464442  3801 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:33.459753  3875 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:33.469180  3875 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.471465  3875 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:33.466598  4039 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
I20260709 14:51:33.472606  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:33.476639  2757 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:33.477438  3999 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:33.479213  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:33.480278  3999 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:33.480760  3999 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:33.481194  3999 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.483619  2757 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:33.481244  3542 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:33.485885  3952 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:33.491060  4016 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.492321  3952 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:33.492656  2611 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:33.493012  3952 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.494431  4016 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:33.496647  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:33.502170  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.503358  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:33.512029  3927 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:33.512173  3847 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:33.506297  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:33.468097  3801 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.531081  3733 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:33.519577  3936 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:33.539397  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:33.564635  2610 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:33.552192  4035 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:33.559154  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.561250  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:33.562814  3936 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:33.537065  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.569075  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:33.591017  4035 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:33.622584  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:33.623625  2755 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:33.592057  3801 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.643602  3787 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:33.648227  3787 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:33.603446  3981 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:33.625770  4035 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.648062  3952 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:33.646693  4002 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:33.648857  3936 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:33.657899  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 129173us
I20260709 14:51:33.689134  3787 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:33.663615  3981 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:33.687641  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 148349us
I20260709 14:51:33.691300  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:33.708341  2612 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:33.714214  4048 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 14:51:33.714571  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.716327  4050 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:33.717000  4050 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:33.719388  4050 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:33.720132  4005 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:33.720490  2680 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:33.725056  4049 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:33.695253  3825 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 14:51:33.727643  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:33.727746  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.729626  2680 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:33.733309  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.724540  3981 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:33.734441  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:33.735103  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:33.743283  2677 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:33.762328  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.765020  4025 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 14:51:33.753422  3979 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.709885  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:33.760394  3902 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:33.773851  3961 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.793860  4025 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:33.795239  4025 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.795437  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:33.786926  4019 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:33.790405  4007 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:33.783521  3982 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 14:51:33.833796  2684 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:33.787845  3902 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:33.836274  3982 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:33.823064  4002 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:33.840181  4051 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 14:51:33.829347  3538 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
W20260709 14:51:33.821679  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 116376us
I20260709 14:51:33.828334  4019 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:33.855835  4019 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:33.844851  3982 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:33.860272  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:33.861259  2681 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:33.863266  4025 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:33.864912  4035 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:33.861362  3982 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.849645  4002 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:33.868348  3982 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:33.869459  4002 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:33.872366  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:33.874928  3902 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:33.891279  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:33.904464  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.906702  3547 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:33.914324  2751 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:33.915419  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:33.919070  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:33.919773  3541 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:33.921124  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:33.924443  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:33.914139  3995 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:33.931092  3999 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:33.933596  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:33.940631  3875 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:33.928186  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:33.947886  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:33.948421  2759 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:33.948721  3545 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:33.952989  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:33.953820  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:33.965677  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:33.920348  2751 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
W20260709 14:51:33.974803  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46402.3us
W20260709 14:51:33.965273  4062 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 14:51:34.000658  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:34.001525  2755 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.002575  4035 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.003901  2605 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:33.914701  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:34.004593  3999 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:34.004948  4053 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.357s	user 0.004s	sys 0.000s
W20260709 14:51:34.005117  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111724us
I20260709 14:51:34.007035  4002 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:34.007104  3952 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:34.008364  3952 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:34.008745  3952 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:34.009049  3927 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:34.009647  3927 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.012382  4026 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:33.985322  3859 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.012395  3927 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.015151  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:33.974036  3861 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.000919  3977 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 14:51:34.015513  4040 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.648s	user 0.007s	sys 0.000s
I20260709 14:51:33.978591  4013 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.023733  2757 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:33.964600  3961 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 14:51:34.009637  3952 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:34.016351  3861 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:33.994515  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.022250  3977 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:34.025779  4013 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.050204  2757 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:34.066356  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.067147  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:34.067878  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.044536  3861 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.068665  2754 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:34.072130  2752 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:34.074724  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:34.078289  3908 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:34.078991  3908 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.079664  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:34.067296  2760 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:34.045982  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:34.091838  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45450.1us
I20260709 14:51:34.055080  4013 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.102620  3952 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:34.116056  3547 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:34.099813  4035 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:34.117864  2755 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:34.117988  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78360.4us
I20260709 14:51:34.120442  2435 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 158) took 1976 ms. Trace:
W20260709 14:51:34.118449  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 119592us
I20260709 14:51:34.123133  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:34.124882  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:34.121414  2603 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:34.134783  2431 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 3 to 4, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:34.121079  2435 rpcz_store.cc:276] 0709 14:51:32.143383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:32.149016 (+  5633us) service_pool.cc:224] Handling call
0709 14:51:34.119559 (+1970543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:32.637863 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:32.848233 (+210370us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:32.866049 (+ 17816us) write_op.cc:435] Acquired schema lock
0709 14:51:32.867668 (+  1619us) tablet.cc:662] Decoding operations
0709 14:51:32.936770 (+ 69102us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:32.936896 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:32.936927 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 14:51:32.951156 (+ 14229us) tablet.cc:701] Row locks acquired
0709 14:51:32.951210 (+    54us) write_op.cc:260] PREPARE: finished
0709 14:51:32.951455 (+   245us) write_op.cc:270] Start()
0709 14:51:32.951649 (+   194us) write_op.cc:276] Timestamp: P: 1783608692951359 usec, L: 0
0709 14:51:32.951689 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 14:51:32.993296 (+ 41607us) log.cc:844] Serialized 928 byte log entry
0709 14:51:33.243981 (+250685us) op_driver.cc:464] REPLICATION: finished
0709 14:51:33.474705 (+230724us) write_op.cc:301] APPLY: starting
0709 14:51:33.474845 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:33.474998 (+   153us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:33.499199 (+ 24201us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:33.512280 (+ 13081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:33.512552 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 14:51:33.512615 (+    63us) write_op.cc:312] APPLY: finished
0709 14:51:33.512924 (+   309us) log.cc:844] Serialized 28 byte log entry
0709 14:51:33.518931 (+  6007us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:33.525524 (+  6593us) write_op.cc:454] Released schema lock
0709 14:51:33.533219 (+  7695us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:33.587203 (+ 53984us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:33.587261 (+    58us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":230383,"apply.run_cpu_time_us":3606,"apply.run_wall_time_us":99693,"num_ops":2,"prepare.queue_time_us":256547,"prepare.run_cpu_time_us":27577,"prepare.run_wall_time_us":431251,"raft.queue_time_us":73963,"raft.run_cpu_time_us":10028,"raft.run_wall_time_us":189253,"replication_time_us":292118,"thread_start_us":369706,"threads_started":4,"wal-append.queue_time_us":137396}]]}
I20260709 14:51:34.138190  4075 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:34.077493  3977 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:34.141305  3908 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:34.144115  3859 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.145720  3957 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:34.105890  3936 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 14:51:34.167613  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:34.168669  2759 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:34.173955   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:34.181717  2612 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:34.185984  3733 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:34.187605  2431 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 4 to 5, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:34.210621  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:34.211015  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56439.3us
I20260709 14:51:34.162036  3859 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.150480  4075 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:34.163914  3927 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:34.211624  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84077.6us
I20260709 14:51:34.208614  2431 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 4 to 5, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:34.234603  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:34.186697  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:34.250881  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.258258  2611 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:34.162472  3875 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 14:51:34.193524  3957 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:34.234125  3825 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:34.233456  4075 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:34.265693  4062 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 14:51:34.215348  4076 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 14:51:34.257143  4080 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.267182  3538 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.244385  3927 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.294274  3875 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 14:51:34.315547  3957 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:34.316613  4064 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 14:51:34.316913  4049 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.317546  4049 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.318994  3999 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:34.318710  3861 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.319466  3999 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:34.319968  4049 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.320501  3999 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.321161  3861 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.321803  3999 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:34.334281  3927 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:34.337247  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46903us
I20260709 14:51:34.347675  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.348078  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:34.348515  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:34.348714  2759 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.354719  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:34.364171  4055 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.365177  4055 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.368319  4055 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.371155  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.374933  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.378214  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.399250  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:34.413823  4075 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.404481  4049 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.420733  2757 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.395289  2609 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:34.355234  4080 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:34.432832  2604 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 14:51:34.433328  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:34.444136  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 207156us
I20260709 14:51:34.432132  3825 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:34.456073  2759 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:34.459657  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:34.507184  2757 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:34.502797  3902 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:34.516896  4075 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.431053  4049 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.525502  3538 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:34.520159  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:34.531276  2680 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:34.515051  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:34.455855  4031 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 14:51:34.514134  3801 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.587184  3801 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.544911  3733 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:34.601600  3825 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:34.605090  4080 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:34.611653  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.612548  2752 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:34.613492  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:34.614315  3547 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.562096  4013 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:34.619988  2606 raft_consensus.cc:2368] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:34.620986  4013 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.560480  4049 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.563764  3977 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:34.538731  4073 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:34.574724  3952 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:34.577666  3859 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.526319  3902 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:34.580760  2605 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.003s
I20260709 14:51:34.625926  3952 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.627352  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:34.692770  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:34.694460  4089 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.695173  4089 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:34.697738  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70475.1us
W20260709 14:51:34.698805  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69936.2us
I20260709 14:51:34.701015  3847 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.701469  3847 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.701818  3847 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.702980  4084 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:34.703588  4084 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:34.703871  4089 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:34.709764  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.704998  3902 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.712700  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.693537  2605 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:34.714118  2677 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:34.707617  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:34.718756  3542 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:34.715129  3801 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.720839  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:34.722091  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.704546  3859 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:34.723057  4007 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:34.723650  4007 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:34.704530  4084 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:34.724669  4007 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.718307  4055 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.723913  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:34.732257  3547 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:34.746122  3547 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:34.728644  2604 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.745328  2609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:34.761260  4082 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 14:51:34.764932  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:34.766068  3541 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.767815  3825 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.768425  3825 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.775008  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.775803  2755 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:34.745908  4035 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.780328  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:34.785401  2681 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.770658  3825 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.789229  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.779493  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:34.794190  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.802794  4055 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.799458  3982 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:34.805290  3982 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.805728  3982 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:34.808537  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:34.809175  3952 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:34.809337  2608 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.810098  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:34.811002  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:34.780725  3859 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.798811  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:34.812870  3955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.031s	sys 0.084s
I20260709 14:51:34.790014  4075 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:34.787693  3999 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 14:51:34.832911  3955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.032s	sys 0.084s
I20260709 14:51:34.817097  3787 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:34.837893  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:34.789567  4035 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:34.859338  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.860132  2611 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:34.838310  4075 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:34.837728  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.863225  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:34.872740  3543 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:34.875010  2608 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:34.876410  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:34.869863  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 163) took 1005 ms. Trace:
I20260709 14:51:34.877581  2606 rpcz_store.cc:276] 0709 14:51:33.856501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:33.895287 (+ 38786us) service_pool.cc:224] Handling call
0709 14:51:34.847274 (+951987us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54002ff3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:34.861570 (+ 14296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325569280}
I20260709 14:51:34.830600  3859 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.886317  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:34.892999  2682 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:34.825225  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:34.855430  2752 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.902980  3859 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.903903  3908 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:34.874509  3847 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:34.907711  3958 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:34.908176  4074 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:34.908699  4074 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:34.909482  3412 leader_election.cc:400] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:34.894675  4026 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:34.910015  4026 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.910379  4026 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:34.911231  3412 leader_election.cc:403] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:34.911586  2680 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:34.912796  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:34.921453  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:34.922472  3542 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.925133  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:34.908212  3958 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:34.878685  3977 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:34.944480  3977 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:34.945732  3977 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.906555  3847 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:34.933115  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:34.933367  3958 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:34.934116  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.933439  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.934906  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:34.913794  4074 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:34.935359  3541 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:34.939211  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:34.941723  3982 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:34.942651  4024 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:34.981945  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:34.868654  4035 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:34.958088  2682 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:34.962491  2611 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:34.899187  4085 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:34.933091  4075 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:34.974488  3952 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:34.990958  4082 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:34.976286  3543 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:34.975865  3537 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:34.979839  2607 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
W20260709 14:51:35.121600  2610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.005s	sys 0.023s
W20260709 14:51:35.194774  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.023s
I20260709 14:51:35.127317  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:35.128278  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:34.992422  4024 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:34.991317  4088 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.141s
I20260709 14:51:35.145218  2609 raft_consensus.cc:606] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:35.210903  2609 raft_consensus.cc:614] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:35.025463  4085 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:35.139256  4035 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:35.203634  2677 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:35.145766  4082 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:35.251533  4082 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:35.257731  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:35.259033  4094 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:35.259598  4094 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:35.260011  4094 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:35.260246  4024 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.270093  3982 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:35.271451  4095 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 14:51:35.279695  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:35.280853  2604 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:35.284111  2681 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:35.295809  2757 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:35.300109  4080 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.296165  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:35.439529  4091 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.635s	user 0.006s	sys 0.000s
I20260709 14:51:35.442041  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 586) took 1238 ms. Trace:
I20260709 14:51:35.282894  4035 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:35.306007  4094 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.402757  3999 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.423115  4075 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.442553  2684 rpcz_store.cc:276] 0709 14:51:34.203308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.203515 (+   207us) service_pool.cc:224] Handling call
0709 14:51:34.204026 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:34.335548 (+131522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:34.335580 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:34.336094 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 14:51:34.537917 (+201823us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:34.543074 (+  5157us) log.cc:844] Serialized 14 byte log entry
0709 14:51:34.543382 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:34.768479 (+225097us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:34.768545 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:35.249068 (+480523us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:35.441727 (+192659us) raft_consensus.cc:1685] finished
0709 14:51:35.441756 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:35.441961 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6147,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":5133,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":207,"spinlock_wait_cycles":319632512,"thread_start_us":201417,"threads_started":1,"wal-append.queue_time_us":460017}
I20260709 14:51:35.414469  4080 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:35.480999  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.058s	sys 0.096s
W20260709 14:51:35.482278  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.058s	sys 0.096s
W20260709 14:51:35.488109  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.030s	sys 0.061s
W20260709 14:51:35.488722  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.031s	sys 0.062s
I20260709 14:51:35.464295  3999 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:35.489113  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.24021e+06us
I20260709 14:51:35.470927  4075 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:35.409919  4092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.014s	sys 0.013s
W20260709 14:51:35.490131  3902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.040s
I20260709 14:51:35.519078  3733 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.549844  3733 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.520941  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:35.528620  4094 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:35.531715  4098 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 14:51:35.533192  4026 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.004s
I20260709 14:51:35.535781  4075 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:35.542536  3847 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.538367  4099 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:35.504359  4031 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:35.549503  3902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.015s	sys 0.041s
W20260709 14:51:35.563563  3542 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.005s
W20260709 14:51:35.552749  4093 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.350s	user 0.002s	sys 0.000s
I20260709 14:51:35.605396  3733 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:35.563416  4085 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:35.547884  4092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.015s	sys 0.013s
I20260709 14:51:35.632468  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.559989  4087 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.579447  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:35.562273  2605 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 5
W20260709 14:51:35.642854  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.016s	sys 0.041s
I20260709 14:51:35.643234  3542 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:35.565151  4084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.009s	sys 0.073s
W20260709 14:51:35.608829  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.012s	sys 0.042s
W20260709 14:51:35.658262  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.017s	sys 0.041s
W20260709 14:51:35.658442  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.012s	sys 0.042s
I20260709 14:51:35.564536  3957 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.577536  3859 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.654207  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:35.608486  3847 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.664892  2604 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:35.664809  4085 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.663093  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 372) took 1055 ms. Trace:
I20260709 14:51:35.564661  4026 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:35.663818  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84516.2us
W20260709 14:51:35.659413  4084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.012s	sys 0.074s
I20260709 14:51:35.661012  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.672010  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:35.610649  4099 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.675634  3542 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:51:35.677476  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.018s	sys 0.050s
W20260709 14:51:35.678714  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.019s	sys 0.050s
I20260709 14:51:35.625623  4031 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.669348  4007 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:35.688422  4096 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
I20260709 14:51:35.656319  4035 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:35.673812  4085 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.672082  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:35.685897  3955 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:35.731076  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 212596us
I20260709 14:51:35.737772  3545 raft_consensus.cc:2368] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4.
W20260709 14:51:35.741694  3958 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.768s	user 0.014s	sys 0.000s
I20260709 14:51:35.742254  3958 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:35.743624  4113 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:35.744258  2755 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:35.744237  4113 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.745311  3958 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:35.737474  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.023s
I20260709 14:51:35.665855  3957 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.668569  3859 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:35.659101  4087 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:35.749357  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.005s	sys 0.023s
W20260709 14:51:35.791859  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 280368us
W20260709 14:51:35.772349  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100355us
I20260709 14:51:35.789330  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:35.799853  2608 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:35.800153  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 167) took 1645 ms. Trace:
I20260709 14:51:35.796721  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:35.804381  2604 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:35.795291  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.777753  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:35.806991  3547 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:35.754091  4126 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:35.757299  3847 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:35.737522  4007 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.817760  4087 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:35.750005  3958 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:35.800778  2610 rpcz_store.cc:276] 0709 14:51:34.155281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.155529 (+   248us) service_pool.cc:224] Handling call
0709 14:51:34.156070 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:34.447401 (+291331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:34.447437 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:34.458376 (+ 10939us) log.cc:844] Serialized 26 byte log entry
0709 14:51:35.195432 (+737056us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:35.678168 (+482736us) log.cc:844] Serialized 14 byte log entry
0709 14:51:35.679152 (+   984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:35.679368 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:35.799900 (+120532us) raft_consensus.cc:1685] finished
0709 14:51:35.799925 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:35.800083 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4614,"prepare.queue_time_us":557205,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":111092,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":10212,"thread_start_us":1374300,"threads_started":3,"wal-append.queue_time_us":946647}
I20260709 14:51:35.755791  4031 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:35.758193  3999 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:35.836869  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 97422.3us
I20260709 14:51:35.820232  3995 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.820473  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.795210  4122 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 14:51:35.750907  4104 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
I20260709 14:51:35.763662  4099 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:35.766640  4127 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.785218  3902 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.822263  4055 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.822542  4085 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:35.767843  4026 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:35.824015  2604 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:35.772176  3955 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.824077  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 599) took 1047 ms. Trace:
W20260709 14:51:35.826478  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48908.1us
I20260709 14:51:35.764258  4125 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.784191  4112 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:35.832767  3847 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.761149  3957 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:35.833922  4119 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:35.838974  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:35.825608  4126 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:35.903228  4126 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:35.833308  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 680) took 1056 ms. Trace:
I20260709 14:51:35.904577  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:35.838105  3958 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:35.865443  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:35.867985  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:35.839303  4035 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:35.841257  3995 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.879155  4129 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.867535  4127 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.883067  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:35.936604  2758 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:35.887663  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:35.891100  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:35.891533  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.899399  2759 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:35.851511  4122 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:35.854038  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:35.852370  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:35.848727  3538 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:35.877177  3543 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:35.870661  2683 rpcz_store.cc:276] 0709 14:51:34.776891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.777015 (+   124us) service_pool.cc:224] Handling call
0709 14:51:34.777323 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:34.778960 (+  1637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:34.778991 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:34.796318 (+ 17327us) log.cc:844] Serialized 26 byte log entry
0709 14:51:35.105518 (+309200us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b18000d1860. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:51:35.659541 (+554023us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:35.659583 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:35.659649 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:35.823703 (+164054us) raft_consensus.cc:1685] finished
0709 14:51:35.823730 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:35.824006 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1245,"raft.queue_time_us":3047,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":12481,"spinlock_wait_cycles":22813952,"thread_start_us":553043,"threads_started":1,"wal-append.queue_time_us":1010350}
I20260709 14:51:35.961048  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:35.962630  4108 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.905714  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:35.910180  3958 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:35.935439  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67563.6us
I20260709 14:51:35.904767  2677 rpcz_store.cc:276] 0709 14:51:34.774268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.779399 (+  5131us) service_pool.cc:224] Handling call
0709 14:51:35.818720 (+1039321us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:35.830301 (+ 11581us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590018432}
I20260709 14:51:35.945032  4122 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:35.947248  4109 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.967451  3547 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:35.867858  3902 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.876348  4055 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.935691  2756 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:35.894302  4113 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:35.974642  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:35.976133  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:35.899603  4112 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.929754  2608 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:35.971853  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:35.965158  4136 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:35.905596  3847 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:35.991590  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:35.894111  4125 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:35.984440  2603 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:35.989264  2683 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:35.671167  3546 rpcz_store.cc:276] 0709 14:51:34.607626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.625425 (+ 17799us) service_pool.cc:224] Handling call
0709 14:51:34.700551 (+ 75126us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:34.700767 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:34.701197 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:34.702054 (+   857us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:34.702619 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 14:51:35.108039 (+405420us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:35.115271 (+  7232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:35.135919 (+ 20648us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:35.651491 (+515572us) raft_consensus.cc:1685] finished
0709 14:51:35.654683 (+  3192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:35.662983 (+  8300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":9916672,"thread_start_us":280611,"threads_started":1,"wal-append.queue_time_us":678034}
I20260709 14:51:35.953050  4035 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:35.997046  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:35.997195  2681 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:36.008906  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.013235  2686 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:35.981935  2759 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:36.001447  4032 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:35.998339  4136 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:36.002408  4123 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:36.005369  3847 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:35.971091  3955 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.017691  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.022312  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.021111  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 6) took 1274 ms. Trace:
I20260709 14:51:36.033965  2686 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:35.967432  4129 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:35.980786  4109 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.054003  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.054925  2686 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:36.032582  2609 rpcz_store.cc:276] 0709 14:51:34.725762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.731329 (+  5567us) service_pool.cc:224] Handling call
0709 14:51:35.999424 (+1268095us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:35.999620 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904862976}
I20260709 14:51:36.056344  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.057313  2752 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:36.059046  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:36.055584  2605 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:36.055683  3955 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:36.014933  4082 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:36.065130  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44297.6us
W20260709 14:51:36.063715  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 221532us
I20260709 14:51:36.068179  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:36.066625  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:36.030203  4136 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:36.075911  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 88794.6us
I20260709 14:51:36.091770  3542 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 14:51:36.092787  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65145.4us
I20260709 14:51:36.061113  4007 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.069042  4113 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:35.985714  4108 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.095142  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:36.062458  4123 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.049381  4032 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.065513  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.078795  4026 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:36.069954  3955 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.121313  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:36.104844  4113 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.123468  2753 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:36.117260  4007 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.130296  3995 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.110519  4075 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:36.071310  4082 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.140517  2679 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:36.074451  4119 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 14:51:36.130587  4108 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.123219  4135 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.141004  3883 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:36.147879  4113 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:36.177927  2758 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:36.198248  4132 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:36.199172  4132 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.182803  3883 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:36.160892  4007 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.179106  4148 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:36.183972  3902 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.182065  4108 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.213596  4129 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.208211  4087 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.215276  4035 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.220259  4125 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.221418  3883 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:36.225981  2724 leader_election.cc:400] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4.
I20260709 14:51:36.257254  2724 leader_election.cc:403] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:36.195130  4055 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.222385  4127 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.268275  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:36.251709  4082 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:36.255272  4101 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.711s	user 0.003s	sys 0.003s
I20260709 14:51:36.255038  4087 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:36.255932  4139 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 14:51:36.238555  4120 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:36.246577  4132 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.226613  4148 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:36.184891  4135 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.249868  4108 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.274639  4087 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.269657  4125 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:36.280871  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.240571  4112 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.253929  4109 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.283262  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:36.283942  4118 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.631s	user 0.005s	sys 0.001s
I20260709 14:51:36.300736  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 380) took 1023 ms. Trace:
I20260709 14:51:36.301414  3536 rpcz_store.cc:276] 0709 14:51:35.277571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:35.293318 (+ 15747us) service_pool.cc:224] Handling call
0709 14:51:36.299859 (+1006541us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:36.300137 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:36.300359 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:36.300387 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:36.300434 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:36.300477 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:36.300542 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:36.300659 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24428416}
I20260709 14:51:36.308631  3955 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.258644  4123 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.268253  4075 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.334017  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 668) took 1408 ms. Trace:
I20260709 14:51:36.285214  4148 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:36.334618  2755 rpcz_store.cc:276] 0709 14:51:34.924605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.924843 (+   238us) service_pool.cc:224] Handling call
0709 14:51:36.332966 (+1408123us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:36.333161 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881455872}
I20260709 14:51:36.304316  4082 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:36.335202  4007 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.277374  4120 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.351949  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.359108  4108 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:36.344305  4075 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:36.366724  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 92136.4us
I20260709 14:51:36.344666  4082 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:36.373642  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 273799us
I20260709 14:51:36.374485  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.375465  2760 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:36.375612  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.380055  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.380980  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:36.383648  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:36.334647  4032 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.362685  4075 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.403702  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:36.391700  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:36.384277  4108 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.371373  4087 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.374296  4082 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:36.382926  4084 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.377756  3999 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.387152  4007 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.392755  3952 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:36.392459  4099 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.449409  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:36.412161  4144 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:36.423132  3801 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.503934  3801 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.439100  4108 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.489833  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:36.439541  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 400697us
W20260709 14:51:36.509509  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 412968us
I20260709 14:51:36.512498  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.451066  4033 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:36.445952  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:36.524689  3825 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.535127  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:36.547642  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.507012  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:36.512742  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:36.536113  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:36.474587  4144 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.541596  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:36.560313  2679 raft_consensus.cc:606] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:36.472105  4007 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.452023  3999 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.488467  3952 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.488004  4099 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.569967  4114 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:36.444705  4087 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.570798  4153 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:36.593866  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 498874us
I20260709 14:51:36.566176  4144 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:36.585345  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:36.565120  3958 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.005s
W20260709 14:51:36.570199  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57580.2us
I20260709 14:51:36.566016  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.558840  3825 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.587507  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:36.578603  4114 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:36.584956  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.619598  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:36.539357  4033 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.612720  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:36.673569  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.678715  2754 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:36.674356  3936 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:36.675463  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.623556  4114 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:51:36.682427  4126 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.779s	user 0.003s	sys 0.020s
I20260709 14:51:36.447631  4084 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.627130  3801 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.707808  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:36.636787  4151 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:36.708768  2679 raft_consensus.cc:606] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:36.709208  2679 raft_consensus.cc:614] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:36.634614  4135 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.629904  4026 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:36.609077  3958 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.625844  4035 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.702924  4126 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.640867  4125 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4.
I20260709 14:51:36.679328  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:36.709041  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 114700us
I20260709 14:51:36.726218  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:36.729233  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 121906us
I20260709 14:51:36.731182  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.691119  4158 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 14:51:36.723847  4113 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.526s	user 0.015s	sys 0.001s
I20260709 14:51:36.712858  4151 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.744745  2755 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:36.745293  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.746660  2758 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:36.748039  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.748662  2681 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:36.745095   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 14:51:36.750130  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:36.751860   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:36.760990  4158 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:36.746127  4113 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.735339  4075 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:36.734134  4099 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.763087  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.728807  3883 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.726531  4035 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.745968  4151 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:36.742110  2606 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:36.776623  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.778278  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:36.753573  4120 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.767539  3541 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:36.798391  2611 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:36.801039  3825 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.801326  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:36.804595  4156 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 14:51:36.807184  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.809044  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:36.801757  4080 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.812337  4084 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.813090  4084 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:36.813458  4084 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:36.813772  4084 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:36.817679  3883 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.819149  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:36.820092  2603 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:36.780191  4108 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:36.784081  4099 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.785773  3999 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.786087  4007 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.792191  4033 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:36.827651  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77583.5us
I20260709 14:51:36.833367  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:36.834250  2679 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 14:51:36.820979  4148 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.826869  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:36.830801  4126 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:36.829555  4136 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.003s
I20260709 14:51:36.826274  4159 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:36.776705  4026 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:36.831439  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 429) took 1247 ms. Trace:
I20260709 14:51:36.829380  4087 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:36.847651  3540 rpcz_store.cc:276] 0709 14:51:35.584537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:35.605500 (+ 20963us) service_pool.cc:224] Handling call
0709 14:51:36.830511 (+1225011us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:36.830768 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:36.831032 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:36.831069 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:36.831120 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:36.831162 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:36.831226 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:36.831362 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7335296}
I20260709 14:51:36.832129  4120 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:36.857784  2759 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:36.885493  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.888434  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.888535  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.889918  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.868520  2681 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:36.832407  4099 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:36.880204  4161 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 14:51:36.884819  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:36.895371  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:36.872205  2752 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:36.877833  4031 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:36.883780  4035 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.848095  4136 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.915539  2678 raft_consensus.cc:2368] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:36.874612  2756 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:36.902792  4127 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:36.867398  4120 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:36.903129  4110 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 14:51:36.909268  4031 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:36.857457  4159 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.011423  4032 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:36.913911  4099 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:36.925580  4035 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.081786  3883 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:37.082578  3883 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.085584  3883 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:37.084508  4121 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:36.929962  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:36.922183  4144 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:36.986697  4148 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:37.009639  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81229.4us
I20260709 14:51:37.014778  4031 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.017230  2677 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:36.858579  4158 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.146319  4121 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.181102  4120 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.187682  4116 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:36.893913  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:37.219256  4116 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.219956  4116 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.148393  4035 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.220420  4121 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:37.182219  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 252310us
I20260709 14:51:37.224049  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.222703  2681 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:37.197448  4136 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:37.212589  4082 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.761s	user 0.010s	sys 0.003s
I20260709 14:51:37.201041  4007 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.202672  4108 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.228909  4159 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.256740  2679 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:37.258586  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:37.268414  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.260385  4073 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:37.270479  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.281121  2679 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:37.274505  4111 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:37.270785  4033 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:37.283380  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 206399us
I20260709 14:51:37.264832  4007 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.274417  4035 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.266149  4108 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.267761  3787 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:37.265225  4082 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:37.287530  4154 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.009s	sys 0.023s
W20260709 14:51:37.295060  4154 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.009s	sys 0.023s
I20260709 14:51:37.297039  4120 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:37.297163  4169 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 14:51:37.297847  4120 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.286901  4167 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:37.301707  4108 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.302271  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.303941  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.304100  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.289903  4121 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 14:51:37.307308  4007 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.308378  3536 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.307058  4167 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.312814  3540 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:37.314882  4113 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.315973  4120 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:37.291682  3787 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.316112  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.317094  3546 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:37.318215  3787 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:37.308049  4084 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.318632  4144 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.321687  4084 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.320333  4167 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:37.324034  4151 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.010s
I20260709 14:51:37.328992  3859 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.330050  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 437) took 1443 ms. Trace:
I20260709 14:51:37.330471  3547 rpcz_store.cc:276] 0709 14:51:35.886900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:35.887231 (+   331us) service_pool.cc:224] Handling call
0709 14:51:37.329817 (+1442586us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b5400200154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:37.329968 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68361600}
I20260709 14:51:37.332167  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.332547  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.333813  4152 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:37.333915  4174 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:37.334432  3883 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.334627  4174 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.335871  3883 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.337889  4174 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.334591  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.339291  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.340018  2607 raft_consensus.cc:2368] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6.
I20260709 14:51:37.334726  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.335032  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.340262  2606 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:37.335481  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.343050  4152 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.326303  4082 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:37.352223  4114 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.003s
I20260709 14:51:37.354727  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.355376  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:37.356058  2751 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:37.359716  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.346524  3787 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.371412  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.374197  3536 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:37.375255  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:37.375751  3733 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.376684  2609 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:37.339197  3883 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.377228  3733 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.382650  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.383049  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.382628  3787 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.350683  4152 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.388984  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:37.394357  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.381230  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.395865  3927 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.396512  3927 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.398371  2753 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:37.348796  4151 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.402842  3733 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.395900  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:37.405187  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.723s (SENT) [suppressed 15 similar messages]
W20260709 14:51:37.407414  4149 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.866s	user 0.006s	sys 0.009s
I20260709 14:51:37.381422  4136 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.409619  4136 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.413038  4116 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.413504  4136 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.376117  4114 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.415362  2684 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:37.416352  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 606) took 1558 ms (client timeout 2041 ms). Trace:
W20260709 14:51:37.416756  2678 rpcz_store.cc:269] 0709 14:51:35.857559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:35.863524 (+  5965us) service_pool.cc:224] Handling call
0709 14:51:37.416086 (+1552562us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:37.416250 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174882688}
I20260709 14:51:37.418390  4114 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.419471  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.423204  2758 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:37.420305  2760 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:37.428754  2609 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:37.431211  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 605) took 1574 ms. Trace:
I20260709 14:51:37.440864  2682 rpcz_store.cc:276] 0709 14:51:35.857258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:35.860449 (+  3191us) service_pool.cc:224] Handling call
0709 14:51:35.886193 (+ 25744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:36.107150 (+220957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:36.107185 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:36.107231 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:36.107271 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:37.430951 (+1323680us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:37.431004 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:37.431146 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507503488}
I20260709 14:51:37.431908  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.444346  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.446456  4111 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:37.447055  4111 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.447602  4111 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:37.451997  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.453042  4088 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:37.453702  4088 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:37.454180  4088 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.432201  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:37.462522  2652 leader_election.cc:400] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6.
I20260709 14:51:37.463001  2652 leader_election.cc:403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:37.463881  4114 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.464778  4114 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6.
I20260709 14:51:37.467092  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:37.474023  4123 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:37.476068  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:37.484158  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 736) took 1084 ms. Trace:
W20260709 14:51:37.481997  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100142us
I20260709 14:51:37.484691  2610 rpcz_store.cc:276] 0709 14:51:36.399484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.400182 (+   698us) service_pool.cc:224] Handling call
0709 14:51:37.483327 (+1083145us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:37.483570 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:37.483784 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:37.483809 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:37.483867 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:37.483901 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:37.483964 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:37.484094 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8667520}
I20260709 14:51:37.415515  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.418480  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.401023  3927 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:37.493484  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:37.495602  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.498761  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.491547  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.505239  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.483536  4123 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:37.517906  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:37.488530  4150 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:37.519063  4144 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:37.519487  4150 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.522179  4150 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.523818  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:37.524575  2611 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.531196  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:37.532195  2610 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.534520  2611 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:37.536711  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.538018  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 735) took 1149 ms. Trace:
I20260709 14:51:37.538515  2608 rpcz_store.cc:276] 0709 14:51:36.388608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.388781 (+   173us) service_pool.cc:224] Handling call
0709 14:51:37.491284 (+1102503us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:37.537949 (+ 46665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28431616}
I20260709 14:51:37.543380  4033 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.543658  4151 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.544147  4033 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.512319  4157 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.550504  2751 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:37.550192  2610 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:37.500437  4127 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:37.494678  4087 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:37.556911  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51863.4us
W20260709 14:51:37.557117  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 151988us
I20260709 14:51:37.492748  4084 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 14:51:37.558964  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.567075  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:37.555306  2607 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:37.552006  4085 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:37.553153  4073 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.556727  4127 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.546295  4157 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.573998  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.575295  3545 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:37.581719  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.582801  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.583191  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.584242  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.573455  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:37.584790  4144 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:37.584307  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.592494  4116 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.594005  3547 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:37.602130  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.570617  4150 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:37.607256  3982 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.608819  2607 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:37.595813  4033 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:37.587531  4127 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:37.610167  2611 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:37.612679  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.613544  2604 raft_consensus.cc:2368] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4.
I20260709 14:51:37.613942  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 732) took 1245 ms. Trace:
I20260709 14:51:37.617461  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:37.617780  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.618273  2751 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:37.614276  2604 rpcz_store.cc:276] 0709 14:51:36.368518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.368657 (+   139us) service_pool.cc:224] Handling call
0709 14:51:37.612484 (+1243827us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:37.613880 (+  1396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24757504}
I20260709 14:51:37.564667  4087 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.634199  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.630366  2751 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:37.644080  3952 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.647500  4111 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.648097  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.650429  4111 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.652606  4087 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.654893  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 730) took 1287 ms. Trace:
I20260709 14:51:37.654989  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:37.576321  4113 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 14:51:37.654623  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.659708  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:37.646420  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 558) took 1065 ms. Trace:
I20260709 14:51:37.663652  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:37.666003  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.625427  3825 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:37.683303  2603 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:37.683811  3825 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.686169  3825 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.664358  2605 rpcz_store.cc:276] 0709 14:51:36.367592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.369107 (+  1515us) service_pool.cc:224] Handling call
0709 14:51:37.654029 (+1284922us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:37.654235 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:37.654469 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:37.654505 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:37.654571 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:37.654605 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:37.654671 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:37.654800 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32645504}
I20260709 14:51:37.685701  4127 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.674527  3536 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:37.688367  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.689617  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.671494  2678 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.690486  4127 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.655907  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.681180  4113 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 14:51:37.694664  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 173) took 1343 ms (client timeout 1723 ms). Trace:
I20260709 14:51:37.690614  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:37.701470  2755 rpcz_store.cc:269] 0709 14:51:36.351224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.351561 (+   337us) service_pool.cc:224] Handling call
0709 14:51:37.694345 (+1342784us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:37.694568 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879498624}
I20260709 14:51:37.701918  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.677891  3541 rpcz_store.cc:276] 0709 14:51:36.580940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.585854 (+  4914us) service_pool.cc:224] Handling call
0709 14:51:37.646161 (+1060307us) spinlock_profiling.cc:243] Waited 86.2 ms on lock 0x7b5400003554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:37.646332 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198262528}
I20260709 14:51:37.661144  4116 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:37.704373  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.706728  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.706724  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.709553  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.691394  3902 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:37.662279  4089 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 14:51:37.713056  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:37.719841  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.721516  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.722339  3541 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:37.724941  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.705120  4084 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:37.725965  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 687) took 1049 ms. Trace:
I20260709 14:51:37.724996  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:37.713521  4082 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:37.710683  4163 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:37.727180  3927 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:37.727627  3927 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:37.728005  3927 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.703538  4167 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.716837  4170 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:37.727779  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:37.725180  4114 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:37.729889  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.673437  4087 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 14:51:37.701113  4113 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:37.726362  2754 rpcz_store.cc:276] 0709 14:51:36.676370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.746371 (+ 70001us) service_pool.cc:224] Handling call
0709 14:51:37.724752 (+978381us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:37.725897 (+  1145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6359808}
I20260709 14:51:37.711424  4148 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:37.741854  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.742949  4151 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.743605  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.743525  4151 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.718089  3902 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:37.752825  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:37.749075  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.735201  4084 raft_consensus.cc:2764] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 14:51:37.760689  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 265145us
I20260709 14:51:37.739948  4150 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:37.764784  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:37.756390  3902 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.751753  3883 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.754892  4099 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.757534  4082 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:37.754666  4007 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.756124  4148 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:37.737687  4114 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.762890  4026 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:37.757673  4167 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:37.782617  4007 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.772400  4080 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:37.786971  4080 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:37.787613  4148 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.787967  4080 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.772107  4151 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.764137  4088 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:37.792869  4088 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.793439  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.775301  4150 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.795123  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.781985  4026 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:37.784205  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.779470  4099 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.796746  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.786810  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.774722  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.800030  3542 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:37.804394  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.804991  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.807924  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:37.808817  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:37.803289  3847 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.811775  3847 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.812086  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.817287  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.819664  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:37.820456  2758 raft_consensus.cc:2368] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:37.806305  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.822755  3538 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:37.822815  4026 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.825291  4144 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:37.813488  4087 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:37.825906  4144 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.806784  4157 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.827373  3958 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.828245  4144 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:37.831918  4099 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.834210  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.835258  2606 raft_consensus.cc:2368] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:37.836506  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.838740  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:37.843681  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:37.854351  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.854633  4150 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:37.857396  4094 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:37.858009  4094 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.858409  4094 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.828315  4114 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:37.825927  4113 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:37.860081  4113 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:37.863153  4121 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:37.863688  4121 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:37.864588  4121 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:37.864737  3902 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:37.865967  4121 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:37.850346  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:37.855064  3847 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.984956  3546 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:37.985220  3902 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:37.985670  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:37.986902  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 119037us
I20260709 14:51:37.986464  2755 raft_consensus.cc:2376] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4. Re-sending same reply.
I20260709 14:51:37.995635  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:37.999024  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.001395  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:37.995589  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.004063  2682 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:38.007385  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.010645  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.011503  2685 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:38.014465  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.015349  2612 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:38.017143  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.021126  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 698) took 1138 ms. Trace:
I20260709 14:51:38.021649  2677 rpcz_store.cc:276] 0709 14:51:36.882241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.884464 (+  2223us) service_pool.cc:224] Handling call
0709 14:51:38.020922 (+1136458us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:38.021055 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264328960}
I20260709 14:51:38.018808  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.027379  4167 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:38.005318  4055 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:38.028028  4167 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.028337  4055 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:38.029814  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:37.986486  4150 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.037868  2611 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:38.030655  3541 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:38.010818  2678 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:37.996518  3927 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:38.045799  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.030561  4167 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:38.031771  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.057622  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:38.059499  3927 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:38.030750  4055 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:38.056311  2753 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:38.066069  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.072654  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:38.057641  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:38.071259  2755 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:38.073920  3547 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:38.074771  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.079003  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.079583  3547 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:38.039959  4150 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:38.147931  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:38.072000  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.070916  3546 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:38.144472  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.196990  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.199399  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.200274  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.202589  2760 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:38.147619  4094 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:38.206934  2611 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:38.222246  4150 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:38.206593  4094 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:38.185595  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:38.199524  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:38.186264  4113 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:38.304450  4026 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:38.334169  4026 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:38.217572  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.412287  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.413102  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.413612  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.414088  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:38.414934  2683 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:38.414868  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:38.415769  2684 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:38.413192  2677 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:38.413002  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:38.418704  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:38.420177  2606 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:38.423475  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:38.426384  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.428361  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.431213  3547 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:38.432080  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 444319us
I20260709 14:51:38.440763  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:38.443099  3411 leader_election.cc:400] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:38.443950  3411 leader_election.cc:403] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 14:51:38.447962  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.460264  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:38.461829  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:38.444248  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.005s	sys 0.032s
I20260709 14:51:38.480993  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.483184  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:38.489650  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.515380  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.519102  2608 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:38.485862  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.006s	sys 0.033s
W20260709 14:51:38.525462  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 222441us
I20260709 14:51:38.481956  3847 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:38.479008  4055 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:38.369029  4113 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:38.515895  4112 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:38.577613  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.523664  4049 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.523049  4115 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.520279  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.543183  4151 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:38.498446  3825 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 14:51:38.532680  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.572949  4182 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:38.565213  4094 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:38.573997  4112 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:38.582234  4055 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:38.603434  4182 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.604885  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.582707  4026 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:38.617453  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 181650us
I20260709 14:51:38.606318  4112 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:38.622212  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:38.620564  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 139706us
I20260709 14:51:38.624444  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:38.622889  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.624898  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.634505  3859 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:38.634239  4055 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:38.635432  3859 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:38.636816  3859 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:38.643458  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.645000  4115 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.605542  4094 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.654425  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.625768  4121 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:38.659818  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 200255us
I20260709 14:51:38.631319  4151 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.685053  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:38.694759  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.673641  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 698) took 1053 ms. Trace:
I20260709 14:51:38.706054  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.704270  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.704353  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.713413  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.715430  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.716660  2609 raft_consensus.cc:2368] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:38.721007  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.665743  4121 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.728988  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.724422  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.734953  2608 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:38.692258  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 572) took 1068 ms. Trace:
I20260709 14:51:38.722960  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.725934  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:38.738428  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:38.741593  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113827us
I20260709 14:51:38.743806  2610 raft_consensus.cc:2368] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:38.745087  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.679528  4115 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 14:51:38.710583  4152 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:38.751099  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.753141  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.757140  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:38.758323  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.698400  4151 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 14:51:38.759909  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.727231  4113 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:38.761524  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:38.763629  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 274048us
I20260709 14:51:38.727097  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:38.766825  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.767148  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.769887  3859 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:38.769965  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.770190  3825 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 14:51:38.771237  4195 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 14:51:38.771413  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:38.741379  3540 rpcz_store.cc:276] 0709 14:51:37.617244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.614107 (+996863us) service_pool.cc:224] Handling call
0709 14:51:38.628812 (+ 14705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:38.654971 (+ 26159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.655824 (+   853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:38.662428 (+  6604us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:38.665302 (+  2874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.673207 (+  7905us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:38.685374 (+ 12167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:38.772925  4049 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:38.774298  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 400) took 1141 ms. Trace:
I20260709 14:51:38.755231  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.742264  4026 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.778214  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:38.776263  4193 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260709 14:51:38.758735  4150 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:38.771302  3825 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:38.726890  4200 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 14:51:38.784613  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:38.785250  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.784583  3825 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.791092  3825 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:38.777326  4152 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:38.793428  4152 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.790004  2604 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:38.775645  4205 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.797257  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.777994  4136 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:38.800832  3412 leader_election.cc:400] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:38.801733  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.803618  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.804775  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:38.805470  4200 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:38.806069  3412 leader_election.cc:400] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:38.806247  4200 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.807036  4026 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:38.807479  4026 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.807935  4026 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:38.807305  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.809490  3847 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.810792  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.811818  3847 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:38.812114  4121 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:38.812652  3847 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.813174  3847 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:38.812671  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.817224  4200 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.819002  3902 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:38.783370  4113 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:38.820199  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:38.820971  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 127852us
I20260709 14:51:38.782225  4150 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.831794  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.847961  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.798605  4049 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:38.856899  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46197.4us
I20260709 14:51:38.854063  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.802295  4136 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.798735  4199 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.846263  4080 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:38.822118  4113 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.844264  4150 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:38.847262  4200 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.807166  4207 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 14:51:38.863924  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.855077  4205 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:38.856031  4094 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:38.870748  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 92672.8us
I20260709 14:51:38.855394  4182 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:38.864635  4136 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:38.774791  3540 rpcz_store.cc:276] 0709 14:51:37.632433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.773255 (+1140822us) service_pool.cc:224] Handling call
0709 14:51:38.773676 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:38.773910 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.773937 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:38.773988 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:38.774020 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.774081 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:38.774233 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:51:38.872643  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 93671.4us
I20260709 14:51:38.873462  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.875102  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.706547  2752 rpcz_store.cc:276] 0709 14:51:37.617643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:37.625095 (+  7452us) service_pool.cc:224] Handling call
0709 14:51:37.685785 (+ 60690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:37.686331 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:37.686370 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:37.687007 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 14:51:38.073771 (+386764us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16b53c0ac 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b
0709 14:51:38.566522 (+492751us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:38.570689 (+  4167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.583989 (+ 13300us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:38.645073 (+ 61084us) raft_consensus.cc:1685] finished
0709 14:51:38.645119 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:38.670994 (+ 25875us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4298,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"spinlock_wait_cycles":27532672,"thread_start_us":837757,"threads_started":1,"wal-append.queue_time_us":897057}
I20260709 14:51:38.882400  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.884066  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.885303  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:38.885999  2753 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:38.889557  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.894030  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.895516  2753 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:38.868081  4113 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:38.853708  4112 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:38.873937  4200 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.874361  2605 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:38.871420  3902 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:38.868906  4080 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:38.917819  4205 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:38.915321  3927 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:38.913564  4151 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:38.921780  4182 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:38.918807  4200 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:38.936887  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:38.915825  3825 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:38.942566  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 94707.7us
I20260709 14:51:38.925351  3860 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.949481  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:38.948907  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63732.9us
I20260709 14:51:38.951722  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:38.954603  4205 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:38.955077  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 555) took 2378 ms. Trace:
I20260709 14:51:38.930385  4211 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:38.940424  4007 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:38.948873  4210 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:38.972580  4200 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.972784  4199 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:38.973242  4200 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:38.973568  4112 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:38.980244  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.982199  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:38.987757  4140 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:38.988808  4213 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.989697  4080 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:38.989435  4213 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:38.989933  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:38.991052  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:38.991786  4213 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:38.992835  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 456) took 1167 ms. Trace:
I20260709 14:51:38.993253  3825 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:38.967396  4205 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 14:51:38.993361  3546 rpcz_store.cc:276] 0709 14:51:37.825030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.991556 (+1166526us) service_pool.cc:224] Handling call
0709 14:51:38.992009 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:38.992286 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.992331 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:38.992418 (+    87us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:38.992473 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.992565 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:38.992752 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:38.995173  4007 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:38.995831  4007 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.003314  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:39.004392  2612 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:38.942728  4026 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:38.955476  4151 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 14:51:38.956166  2752 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:38.955691  3537 rpcz_store.cc:276] 0709 14:51:36.576984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:36.587623 (+ 10639us) service_pool.cc:224] Handling call
0709 14:51:37.563583 (+975960us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:37.572542 (+  8959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:37.660652 (+ 88110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:37.664623 (+  3971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:37.672269 (+  7646us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:37.678906 (+  6637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.954640 (+1275734us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:38.954719 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:38.954958 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152626304}
W20260709 14:51:38.971163  3541 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.912s	user 0.016s	sys 0.000s
I20260709 14:51:38.950042  4182 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:38.959822  4211 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:38.938872  4217 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 14:51:39.017840  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 163857us
I20260709 14:51:39.031263  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.031538  2612 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:38.971930  4210 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:38.955904  3860 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.036185  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:39.019270  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:39.028903  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.032774  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.017277  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:39.061803  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:39.062022  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:39.065339  3787 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.027755  3541 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
W20260709 14:51:39.190459  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 161654us
W20260709 14:51:39.188874  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 124005us
I20260709 14:51:39.033499  4211 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:39.200850  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:39.201690  3787 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:37.769169   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:39.237731  3787 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.038244  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 562) took 1459 ms. Trace:
I20260709 14:51:39.065521  4212 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.062163  3927 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:39.056928  4223 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.067154  4151 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:39.243096  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 223955us
I20260709 14:51:39.063706  4007 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.187901  4215 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.203943  3825 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.263043  3787 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:39.277122  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 75951.5us
I20260709 14:51:39.290030  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.189430  4218 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.237177  4205 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:39.237138  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 457) took 1206 ms. Trace:
I20260709 14:51:39.241497  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 568) took 1636 ms. Trace:
I20260709 14:51:39.244985  4204 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.299311  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.301100  4218 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.270112  4210 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.259982  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 412) took 1259 ms. Trace:
W20260709 14:51:39.302733  4222 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 14:51:39.292626  4151 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.293066  4205 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.305784  4235 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.306959  4157 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.296439  3537 rpcz_store.cc:276] 0709 14:51:37.998960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.021648 (+1022688us) service_pool.cc:224] Handling call
0709 14:51:39.033605 (+ 11957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.060060 (+ 26455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.063115 (+  3055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.065237 (+  2122us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.065294 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.065393 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.205194 (+139801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.312261  4201 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:39.312990  3860 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.252696  3859 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:39.343820  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44668.8us
I20260709 14:51:39.409192  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:39.280990  4109 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:39.297026  4080 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.322708  4202 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:39.401078  4150 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.529s	user 0.003s	sys 0.010s
I20260709 14:51:39.321820  4231 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.340240  4198 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:39.384408  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 143885us
I20260709 14:51:39.296749  3825 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.294036  4223 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.358392  4210 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:39.322897  3541 rpcz_store.cc:276] 0709 14:51:37.601241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.029469 (+428228us) service_pool.cc:224] Handling call
0709 14:51:39.237309 (+1207840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.359467  4200 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:39.403904  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 467080us
W20260709 14:51:39.408044  4220 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 14:51:39.418460  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.363456  4236 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.320801  4237 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.387739  4205 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:39.421518  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:39.434475  4080 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.436908  4212 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.418444  3859 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.401185  4201 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:39.384019  4235 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.250273  3543 rpcz_store.cc:276] 0709 14:51:37.577835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:37.593073 (+ 15238us) service_pool.cc:224] Handling call
0709 14:51:38.953364 (+1360291us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:38.964281 (+ 10917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:38.978131 (+ 13850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.000623 (+ 22492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.013005 (+ 12382us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.016488 (+  3483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.020091 (+  3603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.036677 (+ 16586us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65393024}
I20260709 14:51:39.487648  4202 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.527271  4202 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.528570  4240 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.496555  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:39.529214  4240 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.438987  4182 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.530716  4182 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:39.531270  4182 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:39.532392  3825 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.532232  4182 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.437700  4230 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.420243  4216 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
W20260709 14:51:39.450752  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 204868us
I20260709 14:51:39.395943  3927 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.346174  4204 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.446660  4242 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:39.349565  3546 rpcz_store.cc:276] 0709 14:51:37.992356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.017530 (+1025174us) service_pool.cc:224] Handling call
0709 14:51:39.033745 (+ 16215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.060558 (+ 26813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.063376 (+  2818us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.188136 (+124760us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.198919 (+ 10783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.240633 (+ 41714us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.252070 (+ 11437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11648}
I20260709 14:51:39.574879  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 411) took 1772 ms. Trace:
I20260709 14:51:39.447067  4150 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.532519  3859 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:39.577603  3536 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:39.575416  3540 rpcz_store.cc:276] 0709 14:51:37.802290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.917556 (+1115266us) service_pool.cc:224] Handling call
0709 14:51:38.936893 (+ 19337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:38.963476 (+ 26583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.970687 (+  7211us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:38.998051 (+ 27364us) log.cc:844] Serialized 26 byte log entry
0709 14:51:39.490240 (+492189us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.497976 (+  7736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.508055 (+ 10079us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:39.574590 (+ 66535us) raft_consensus.cc:1685] finished
0709 14:51:39.574622 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.574785 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":455,"raft.queue_time_us":16145,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":155,"thread_start_us":440779,"threads_started":1,"wal-append.queue_time_us":516985}
I20260709 14:51:39.402351  4157 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:39.514122  4055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.042s	sys 0.036s
I20260709 14:51:39.566493  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.513176  4237 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.510706  4112 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.518544  3537 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:39.569715  3825 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.508563  4236 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.591236  4202 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.592227  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 458) took 1593 ms. Trace:
I20260709 14:51:39.592597  4236 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.592912  4157 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.450572  4231 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.585830  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 410) took 1784 ms. Trace:
I20260709 14:51:39.593771  4109 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 14:51:39.573825  3847 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.760s	user 0.011s	sys 0.002s
I20260709 14:51:39.592734  3537 rpcz_store.cc:276] 0709 14:51:37.999335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.442549 (+1443214us) service_pool.cc:224] Handling call
0709 14:51:39.503269 (+ 60720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.590231 (+ 86962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.592160 (+  1929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.594063  3542 rpcz_store.cc:276] 0709 14:51:37.801426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.857971 (+1056545us) service_pool.cc:224] Handling call
0709 14:51:38.859645 (+  1674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:38.860011 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.860169 (+   158us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:38.860903 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 14:51:38.919991 (+ 59088us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:38.920060 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:38.920170 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:39.585571 (+665401us) raft_consensus.cc:1685] finished
0709 14:51:39.585598 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.585761 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":58563,"threads_started":1,"wal-append.queue_time_us":200907}
I20260709 14:51:39.600907  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:39.601189  4231 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:39.585127  4055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.043s	sys 0.037s
I20260709 14:51:39.574380  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 462) took 1562 ms. Trace:
I20260709 14:51:39.589221  4250 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:39.571241  4245 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.623848  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:39.582036  4239 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.591352  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 460) took 1591 ms. Trace:
I20260709 14:51:39.609982  3847 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.637470  4080 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:39.643213  3542 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:39.644368  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 463) took 1632 ms. Trace:
I20260709 14:51:39.644846  3542 rpcz_store.cc:276] 0709 14:51:38.012182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.639782 (+1627600us) service_pool.cc:224] Handling call
0709 14:51:39.642906 (+  3124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.643938 (+  1032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.644284 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.597277  4201 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.647626  4201 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:39.649694  4246 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 14:51:39.585704  4242 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.653129  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:39.603600  4218 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.674810  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 464) took 1662 ms. Trace:
I20260709 14:51:39.677412  3542 rpcz_store.cc:276] 0709 14:51:38.034755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.655212 (+1620457us) service_pool.cc:224] Handling call
0709 14:51:39.674208 (+ 18996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.674429 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.674465 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.674549 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:39.674581 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.674637 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.674757 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.679706  4223 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.680595  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:39.639693  3541 rpcz_store.cc:276] 0709 14:51:37.999987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.516019 (+1516032us) service_pool.cc:224] Handling call
0709 14:51:39.568819 (+ 52800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.586265 (+ 17446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.586928 (+   663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.589877 (+  2949us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:39.589952 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.590057 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.591287 (+  1230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.684017  4204 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:39.569638  4230 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.684839  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 467) took 1649 ms. Trace:
I20260709 14:51:39.685292  3541 rpcz_store.cc:276] 0709 14:51:38.035676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.683889 (+1648213us) service_pool.cc:224] Handling call
0709 14:51:39.684242 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.684448 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.684502 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.684552 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:39.684591 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.684660 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.684777 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.625481  3543 rpcz_store.cc:276] 0709 14:51:38.011884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.565484 (+1553600us) service_pool.cc:224] Handling call
0709 14:51:39.573720 (+  8236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.573950 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.573988 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.574046 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:39.574088 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.574157 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.574312 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.687242  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 757) took 1485 ms. Trace:
I20260709 14:51:39.688350  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 413) took 1623 ms. Trace:
I20260709 14:51:39.688820  3543 rpcz_store.cc:276] 0709 14:51:38.065370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.687386 (+1622016us) service_pool.cc:224] Handling call
0709 14:51:39.687777 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.687989 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.688020 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.688066 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:39.688105 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.688170 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.688285 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.689951  3847 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.691107  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.691545  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 585) took 1459 ms. Trace:
I20260709 14:51:39.692266  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.599526  4112 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.628439  4151 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.694284  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 465) took 1659 ms. Trace:
I20260709 14:51:39.602011  3825 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.702678  3542 rpcz_store.cc:276] 0709 14:51:38.035195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.678538 (+1643343us) service_pool.cc:224] Handling call
0709 14:51:39.693770 (+ 15232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.693961 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.693986 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.694030 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.694074 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.694127 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.694222 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.700775  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.641412  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 566) took 2062 ms. Trace:
I20260709 14:51:39.710376  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:39.710558  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.707294  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.677079  4235 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.722560  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:39.708250  4113 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.005s
I20260709 14:51:39.723523  3542 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:39.687767  2603 rpcz_store.cc:276] 0709 14:51:38.201936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.202327 (+   391us) service_pool.cc:224] Handling call
0709 14:51:39.627514 (+1425187us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:39.641349 (+ 13835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.686872 (+ 45523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.686913 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.686961 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:39.687004 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.687070 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.687180 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47446784}
I20260709 14:51:39.721724  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 469) took 1682 ms. Trace:
I20260709 14:51:39.733642  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.734071  3541 rpcz_store.cc:276] 0709 14:51:38.037063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.686348 (+1649285us) service_pool.cc:224] Handling call
0709 14:51:39.686647 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.693419 (+  6772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.699748 (+  6329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.704570 (+  4822us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.706544 (+  1974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.706634 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.719353 (+ 12719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.640893  4239 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.735545  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:39.736311  3541 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:39.737066  4239 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.738143  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 586) took 1174 ms. Trace:
I20260709 14:51:39.739565  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.740988  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.743441  4236 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:39.744102  3859 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:39.745618  3859 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:39.745803  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.746057  3859 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:39.746313  4236 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:39.634569  4245 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.745872  3847 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.747985  3847 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:39.749847  4245 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.679740  4257 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.753062  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 399) took 2122 ms. Trace:
I20260709 14:51:39.760524  3536 rpcz_store.cc:276] 0709 14:51:37.632152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.775580 (+1143428us) service_pool.cc:224] Handling call
0709 14:51:38.783049 (+  7469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.578048 (+794999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.752773 (+174725us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:39.752992 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826753280}
I20260709 14:51:39.707237  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 582) took 1618 ms. Trace:
I20260709 14:51:39.821843  3825 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.738581  3541 rpcz_store.cc:276] 0709 14:51:38.564324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.737221 (+1172897us) service_pool.cc:224] Handling call
0709 14:51:39.737592 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.737797 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.737826 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.737880 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.737909 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.737966 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.738084 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.820976  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:39.821203  4230 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.824615  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 415) took 1407 ms. Trace:
I20260709 14:51:39.825199  3542 rpcz_store.cc:276] 0709 14:51:38.417142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.795163 (+1378021us) service_pool.cc:224] Handling call
0709 14:51:39.824053 (+ 28890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.824269 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.824296 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.824343 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.824373 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.824442 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.824555 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.827243  3536 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:39.826745  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.691939  3546 rpcz_store.cc:276] 0709 14:51:38.232551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.690657 (+1458106us) service_pool.cc:224] Handling call
0709 14:51:39.690999 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.691205 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.691234 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.691279 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.691317 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.691372 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.691486 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:51:39.830153  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 85564.5us
I20260709 14:51:39.714562  4240 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:39.848356  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:39.853338  3536 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:39.746721  4236 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:39.860430  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.821694  3543 rpcz_store.cc:276] 0709 14:51:38.088328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.689888 (+1601560us) service_pool.cc:224] Handling call
0709 14:51:39.690231 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.690434 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.690467 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:39.690519 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:39.690565 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.690630 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.706318 (+ 15688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.766508  4113 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.865511  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:39.858127  4221 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 14:51:39.882527  3536 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 14:51:39.883249  3536 pending_rounds.cc:85] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 1
I20260709 14:51:39.883565  3536 pending_rounds.cc:107] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:51:39.883886  3536 raft_consensus.cc:2889] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:51:39.866109  4136 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.949s	user 0.011s	sys 0.004s
W20260709 14:51:39.923272  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57652.7us
I20260709 14:51:39.868678  4115 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.868949  4080 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.869694  3860 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:39.829238  4257 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:39.924510  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:39.859517  3859 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.958937  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.959337  3859 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:39.959635  4250 consensus_queue.cc:1175] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260709 14:51:39.960633  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.962139  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.963076  4267 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:39.963390  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.964875  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.966251  3927 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:39.967145  3927 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.969767  3927 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:39.970485  4113 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.971122  2686 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:39.967141  4242 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:39.975984  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 769) took 1091 ms. Trace:
W20260709 14:51:39.976586  4211 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.003s
I20260709 14:51:39.981658  3547 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:39.981976  4211 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.983281  3542 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:39.983647  4208 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:39.984123  4208 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:39.982544  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 571) took 2365 ms. Trace:
I20260709 14:51:39.985021  3547 rpcz_store.cc:276] 0709 14:51:37.616913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.432168 (+815255us) service_pool.cc:224] Handling call
0709 14:51:39.980967 (+1548799us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:39.981223 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.982225 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.982479 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13149568}
I20260709 14:51:39.924834  3541 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:39.987460  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 470) took 1328 ms. Trace:
I20260709 14:51:39.988044  3541 rpcz_store.cc:276] 0709 14:51:38.659527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.826731 (+1167204us) service_pool.cc:224] Handling call
0709 14:51:39.865929 (+ 39198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.987071 (+121142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.987371 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:39.894039  4258 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 14:51:39.946882  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:39.930244  4115 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.710882  3538 rpcz_store.cc:276] 0709 14:51:37.579123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.201700 (+622577us) service_pool.cc:224] Handling call
0709 14:51:38.439295 (+237595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:38.666645 (+227350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.666706 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:39.331681 (+664975us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b2c000e3d48. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:51:39.332442 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 14:51:39.535856 (+203414us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:39.568053 (+ 32197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:39.570382 (+  2329us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:39.615296 (+ 44914us) raft_consensus.cc:1685] finished
0709 14:51:39.615329 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:39.639000 (+ 23671us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1391,"raft.queue_time_us":101647,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":8028,"spinlock_wait_cycles":9429760,"thread_start_us":322715,"threads_started":2,"wal-append.queue_time_us":268745}
I20260709 14:51:39.936169  3860 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:39.994019  4115 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:39.939447  4080 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.994042  4205 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.995982  3537 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:39.867739  4261 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:39.924870  4236 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:39.996609  4080 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:39.950058  4256 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 14:51:39.997239  4236 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:39.997967  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:39.997822  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:39.895848  4136 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.001075  4136 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.001577  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.003144  4233 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:39.951016  4112 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.003726  4233 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.004223  4233 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:40.004312  2677 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:40.004776  2754 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:39.952266  4257 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.006867  4211 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.009562  2755 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:39.932897  2760 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:39.955029  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:40.013559  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 595156us
I20260709 14:51:39.892042  4204 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:39.958177  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.015357  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.015524  4204 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.016054  4204 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.016662  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:39.956100  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:39.978109  4158 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.018271  4158 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.020677  4158 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.021296  4237 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.021733  4198 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:40.023550  4198 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:40.024233  4198 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 14:51:40.024863  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 603495us
W20260709 14:51:40.025949  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:40.028429  4238 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:40.028884  4238 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.029220  4238 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:40.031037  4152 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:39.985080  4208 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:39.976608  2752 rpcz_store.cc:276] 0709 14:51:38.884340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.889081 (+  4741us) service_pool.cc:224] Handling call
0709 14:51:39.975730 (+1086649us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:39.975896 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132588288}
I20260709 14:51:40.039806  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.040186  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.033015  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.031708  4261 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.044694  4261 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.044847  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.040398  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.046115  4026 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.046754  4211 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:40.047359  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.047515  4243 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:40.047734  4202 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.048118  4243 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.048314  4202 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.050513  4243 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:39.955825  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.041090  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.055439  2751 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
W20260709 14:51:40.055601  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 99849us
I20260709 14:51:40.056013  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.056418  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.057591  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.057818  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.057858  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.059168  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.039263  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:39.994513  3860 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.061244  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.052016  4215 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:40.022082  4233 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.037441  2431 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 157) took 7631 ms. Trace:
I20260709 14:51:40.067198  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.068790  4233 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.069536  4233 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:40.072461  2677 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:40.066305  2431 rpcz_store.cc:276] 0709 14:51:32.403239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:32.410561 (+  7322us) service_pool.cc:224] Handling call
0709 14:51:40.034223 (+7623662us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:34.841165 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:34.841823 (+   658us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:34.841862 (+    39us) write_op.cc:435] Acquired schema lock
0709 14:51:34.841881 (+    19us) tablet.cc:662] Decoding operations
0709 14:51:34.842538 (+   657us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:34.842637 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:34.842670 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 14:51:34.843059 (+   389us) tablet.cc:701] Row locks acquired
0709 14:51:34.843078 (+    19us) write_op.cc:260] PREPARE: finished
0709 14:51:34.843213 (+   135us) write_op.cc:270] Start()
0709 14:51:34.843341 (+   128us) write_op.cc:276] Timestamp: P: 1783608694843176 usec, L: 0
0709 14:51:34.843365 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 14:51:34.844151 (+   786us) log.cc:844] Serialized 1316 byte log entry
0709 14:51:36.615877 (+1771726us) op_driver.cc:464] REPLICATION: finished
0709 14:51:36.925032 (+309155us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16b53c0ac 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa17207cef0
0709 14:51:37.392781 (+467749us) write_op.cc:301] APPLY: starting
0709 14:51:37.422099 (+ 29318us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:37.430389 (+  8290us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:37.430422 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:37.518263 (+ 87841us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:37.518536 (+   273us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 14:51:37.518612 (+    76us) write_op.cc:312] APPLY: finished
0709 14:51:37.549136 (+ 30524us) log.cc:844] Serialized 34 byte log entry
0709 14:51:37.713048 (+163912us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:37.718697 (+  5649us) write_op.cc:454] Released schema lock
0709 14:51:37.730986 (+ 12289us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:37.754201 (+ 23215us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:37.754253 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":188108,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":751107,"apply.run_cpu_time_us":66060,"apply.run_wall_time_us":359272,"num_ops":3,"prepare.queue_time_us":562912,"prepare.run_cpu_time_us":36129,"prepare.run_wall_time_us":818464,"raft.queue_time_us":614179,"raft.run_cpu_time_us":34225,"raft.run_wall_time_us":726847,"replication_time_us":1741602,"spinlock_wait_cycles":16053504,"thread_start_us":2305264,"threads_started":5,"wal-append.queue_time_us":931946,"wal-append.run_cpu_time_us":37297,"wal-append.run_wall_time_us":2922137}]]}
I20260709 14:51:40.064745  4215 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:40.074549  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.075096  4233 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.075462  4150 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.075575  4215 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.077018  4215 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:40.052170  4213 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:40.077893  4213 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:40.067256  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.079178  4158 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:40.075659  4233 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.060667  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.081456  4211 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:40.050619  4026 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:40.062731  4080 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:40.082569  4233 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.083068  4202 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.070456  4261 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.054682  4207 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:40.034456  4152 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.075425  3540 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:40.085772  4261 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.087365  2753 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:40.087606  3860 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:40.089171  4080 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:40.089552  4080 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:40.054601  2760 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:40.062248  2755 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:40.063041  3787 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:40.091486  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 577) took 2414 ms. Trace:
I20260709 14:51:40.076035  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.078876  4213 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.053491  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.092386  4213 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:40.092490  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.091929  3545 rpcz_store.cc:276] 0709 14:51:37.677070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.816659 (+1139589us) service_pool.cc:224] Handling call
0709 14:51:38.817069 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:38.869974 (+ 52905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.870016 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:38.870072 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:38.870105 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.091231 (+1221126us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400224cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:40.091285 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:40.091426 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118581120}
I20260709 14:51:40.094802  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.096283  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:40.061993  2681 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:40.054333  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 777) took 1251 ms. Trace:
I20260709 14:51:40.097285  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.099007  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.085017  2686 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:40.102169  3787 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:40.104029  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.108639  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.110741  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:40.099596  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.114125  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:40.114037  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:40.167264  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48844.5us
I20260709 14:51:40.110275  3545 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
W20260709 14:51:40.168642  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:40.116381  3787 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 14:51:40.170383  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.090467  4080 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.166867  4266 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.173534  4266 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.175015  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.097237  2604 rpcz_store.cc:276] 0709 14:51:38.802456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.802581 (+   125us) service_pool.cc:224] Handling call
0709 14:51:38.802994 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:38.853965 (+ 50971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:38.853999 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:38.861148 (+  7149us) log.cc:844] Serialized 26 byte log entry
0709 14:51:39.699886 (+838738us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b180027a9c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1
0709 14:51:39.700106 (+   220us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.711943 (+ 11837us) log.cc:844] Serialized 14 byte log entry
0709 14:51:40.052225 (+340282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.052325 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:40.052378 (+    53us) raft_consensus.cc:1685] finished
0709 14:51:40.054089 (+  1711us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:40.054269 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350675,"prepare.queue_time_us":11659,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":46,"raft.queue_time_us":416801,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":15434,"spinlock_wait_cycles":3621760,"thread_start_us":115449,"threads_started":2,"wal-append.queue_time_us":101561}
I20260709 14:51:40.180383  4055 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.181511  4055 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:40.113162  4123 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.030s	sys 0.019s
W20260709 14:51:40.183111  4123 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.030s	sys 0.019s
I20260709 14:51:40.189226  4205 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.189607  4266 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.192958  4055 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.196265  4087 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.196977  4087 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.199034  4198 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:40.199677  4198 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.202301  4198 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.203917  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.208108  3541 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:40.208397  2682 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:40.209031  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.209554  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.217844  4204 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.176180  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.223939  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.223235  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.233248  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:40.231750  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.229404  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.235785  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.236794  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.243372  2754 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:40.231463  2757 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
W20260709 14:51:40.244985  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 67571.1us
I20260709 14:51:40.250494  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.100632  4152 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.108629  4233 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:40.118760  4026 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:40.114495  4201 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:40.266287  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.267146  2759 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:40.268532  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 92448.4us
W20260709 14:51:40.227200  4227 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.978s	user 0.007s	sys 0.005s
I20260709 14:51:40.279408  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.224648  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.328315  4049 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:40.328955  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.260221  4216 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:40.322065  2680 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:40.281890   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:40.323165  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:40.337128  2682 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:40.269161  3927 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:40.335705  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:40.345054  4049 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:40.345659  4049 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.346069  4049 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.345132  4140 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:40.349098  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.350687  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.309808  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.280205  4055 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 14:51:40.352196  4140 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:40.365957  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.268783  4202 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 14:51:40.364933  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.310212  4258 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 14:51:40.339555  4216 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.280117  4198 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 14:51:40.365482  2433 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:40.356037  4236 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:40.333220  4270 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 14:51:40.332208  4158 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 14:51:40.394073  4094 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:40.365906  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.361346  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.365623  2605 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:40.395954  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.394448  4267 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:40.334034  4026 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:40.418776  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 75357.4us
I20260709 14:51:40.363143  4237 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:40.417819  2755 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:40.376489  4216 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:40.426709  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.430274  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.392549  4201 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.470482  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:40.445147  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48353.6us
I20260709 14:51:40.411760  4257 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:40.413005  4158 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.410445  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.397787  4094 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:40.409123  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.438277  4210 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.401384  4152 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
W20260709 14:51:40.403121  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 197) took 1621 ms (client timeout 1736 ms). Trace:
W20260709 14:51:40.481168  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53735.2us
I20260709 14:51:40.439280  4026 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:40.486729  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:40.487226  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 151599us
I20260709 14:51:40.464265  2603 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:40.412002  3860 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.476171  4182 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:40.536842  4182 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:40.537353  4182 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:40.407778  4140 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.539598  4140 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.540233  4080 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.491168  4094 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:40.543777  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.544736  2753 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:51:40.496847  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.519352  4211 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.497448  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.476130  4201 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.498180  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.476397  4257 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:40.472743  4167 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 14:51:40.532250  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.533898  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.534526  3860 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.534937  4233 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.525233  4150 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:40.497155  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:40.548569  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:40.598891  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.481215  4158 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.599391  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.560272  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.561570  4206 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:40.619369  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.564096  4087 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.567718  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.637349  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.494642  4238 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.639150  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 204) took 1658 ms. Trace:
I20260709 14:51:40.641376  2684 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:40.571153  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 466) took 2535 ms. Trace:
I20260709 14:51:40.579795  4257 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.558604  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.585246  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:40.579595  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:40.575558  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.453922  4049 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.665562  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:40.595248  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 125548us
I20260709 14:51:40.667411  2755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:40.604537  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:40.610570  4158 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.530575  4204 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.617558  2680 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:51:40.628340  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.632227  2686 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:40.634577  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.703452  2685 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:40.643075  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.516171  4210 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:40.713104  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.650192  2679 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:51:40.515275  2607 rpcz_store.cc:269] 0709 14:51:38.781316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.782241 (+   925us) service_pool.cc:224] Handling call
0709 14:51:40.354995 (+1572754us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:40.403044 (+ 48049us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10366208}
I20260709 14:51:40.655457  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.668684  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.594460  3859 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:40.639827  2758 rpcz_store.cc:276] 0709 14:51:38.980397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.981188 (+   791us) service_pool.cc:224] Handling call
0709 14:51:38.981643 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:39.058064 (+ 76421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:39.058115 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:39.188730 (+130615us) log.cc:844] Serialized 26 byte log entry
0709 14:51:39.377074 (+188344us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:39.377138 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.615253 (+1238115us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b5400570154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:40.615308 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:40.615355 (+    47us) raft_consensus.cc:1685] finished
0709 14:51:40.615369 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:40.639039 (+ 23670us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2532,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":140020,"spinlock_wait_cycles":174476672,"thread_start_us":117194,"threads_started":1,"wal-append.queue_time_us":194210,"wal-append.run_cpu_time_us":1871,"wal-append.run_wall_time_us":1004553}
I20260709 14:51:40.616127  4150 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.616288  4233 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.694111  4049 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:40.695803  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.798883  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:40.799286  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74656.2us
I20260709 14:51:40.700547  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:40.706748  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:40.616883  4198 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:40.618673  4206 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:40.716601  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.780644  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.594899  4211 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.643103  3540 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
W20260709 14:51:40.786208  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117671us
I20260709 14:51:40.698489  3860 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.657409  4087 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:40.795543  2609 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:40.795992  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.796996  2758 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:40.797871  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:40.698994  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:40.805913  4257 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:40.805403  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.808746  2684 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:40.778772  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.812951  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:40.782744  4228 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:40.814399  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.790808  4204 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:40.824093  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:40.794067  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141218us
I20260709 14:51:40.828445  2678 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:40.794947  3859 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.650986  3537 rpcz_store.cc:276] 0709 14:51:38.035456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:39.681877 (+1646421us) service_pool.cc:224] Handling call
0709 14:51:39.682263 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:39.996423 (+314160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.570649 (+574226us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54002b9dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:40.571047 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721001344}
I20260709 14:51:40.839131  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:40.840963  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:40.844858  4150 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:40.846254  4211 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.846130  4212 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:40.812855  4199 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 14:51:40.844894  4292 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:40.847627  4199 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:40.848131  4199 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:40.847908  4292 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.848788  4202 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:40.849255  4202 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:40.849080  4199 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.813442  4198 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.825451  4087 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:40.850201  4202 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.015854  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.016601  4207 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:41.017083  4207 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:40.916924  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.984591  4290 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:41.018000  4207 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.887270  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:40.888046  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48983.4us
W20260709 14:51:40.903760  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.022006  2606 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.021806  4284 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 14:51:41.023066  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.003536  4236 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:41.024380  4236 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:40.883701  4270 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:41.025039  4270 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:40.840071  4228 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.869524  4204 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.027514  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.030220  2606 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:41.023670  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:41.031595  2681 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:41.031528  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.032991  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:41.033982  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:40.826483  3546 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:41.035918  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:41.016255  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 175380us
I20260709 14:51:40.828739  4213 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:40.940629  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.054350  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:41.054545  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 220022us
I20260709 14:51:41.055639  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:40.832520  4275 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:41.046790  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.056366  2610 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:41.057536  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:40.913085  4286 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 14:51:41.009558  4278 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 14:51:41.067646  4287 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:40.990909  4238 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.058470  4150 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.068902  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.068825  4150 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.070283  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.070950  3542 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:41.072600  4216 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.000s
I20260709 14:51:40.933815  2603 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:40.844892  4233 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.074153  2678 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:41.049154  4288 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.075146  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:41.067698  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:40.839517  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.048066  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.061257  2680 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:41.062007  2686 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:40.850502  4292 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:40.906276  3860 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:41.064703  2608 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:40.950348  4201 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.944857  3859 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.932432  4205 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:41.064859  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:41.056720  4026 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.003s
I20260709 14:51:41.065378  4284 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:41.065836  4274 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.274s	user 0.004s	sys 0.000s
I20260709 14:51:41.065985  2684 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:41.041414  2611 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:40.850561  4199 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:41.101070  4026 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.103492  4087 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.104200  4287 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:41.104513  4026 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:40.912137  4182 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:40.909916  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:40.949512  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.098071  4261 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.003896  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:41.018384  4290 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.075135  4288 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.055039  4242 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.087554  4240 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.093634  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.146136  2754 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:41.095147  3536 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:40.837769  4266 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.073083  4216 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:40.898792  4211 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:41.081434  4202 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.115453  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.118453  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:41.165568  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47246.6us
I20260709 14:51:41.167129  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:41.080097  4210 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.085813  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:41.168924  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83258.5us
I20260709 14:51:41.129781  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:41.088210  3860 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:41.174816  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:41.100977  3859 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.137629  3787 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:41.143399  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:41.148391  2757 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:41.150799  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.107246  4236 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.190239  2608 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:41.182556  3787 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.171727  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:41.174157  2760 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:40.884024  4206 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.101265  4205 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.117745  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:41.156031  4240 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.150669  4292 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.131004  4112 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 14:51:41.152959  4026 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.210549  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 767) took 1029 ms. Trace:
I20260709 14:51:41.159197  4267 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:41.200172  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.211015  2612 rpcz_store.cc:276] 0709 14:51:40.181408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.205557 (+ 24149us) service_pool.cc:224] Handling call
0709 14:51:40.205993 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:40.238617 (+ 32624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:40.238664 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:40.239345 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 14:51:40.474068 (+234723us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:40.783605 (+309537us) log.cc:844] Serialized 14 byte log entry
0709 14:51:40.822634 (+ 39029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.822721 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:41.008018 (+185297us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:41.210227 (+202209us) raft_consensus.cc:1685] finished
0709 14:51:41.210332 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:41.210480 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114563,"prepare.queue_time_us":351940,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2491,"raft.queue_time_us":77658,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":19806,"spinlock_wait_cycles":11939072,"thread_start_us":453657,"threads_started":2,"wal-append.queue_time_us":337819}
I20260709 14:51:41.208855  3787 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.131243  4237 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 14:51:41.082676  4270 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.237587  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.180986  4254 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:41.181308  3860 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.181061  4233 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:41.233290  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.180841  4266 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:41.181412  4049 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 14:51:41.211207  4080 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.003s
I20260709 14:51:41.240783  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 599) took 1133 ms. Trace:
I20260709 14:51:41.289031  3545 rpcz_store.cc:276] 0709 14:51:40.107221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.176758 (+ 69537us) service_pool.cc:224] Handling call
0709 14:51:40.681033 (+504275us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:40.681496 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:40.816702 (+135206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:40.816754 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:40.816819 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:40.816865 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:41.240482 (+423617us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002eb8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:41.240548 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:41.240707 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457902720}
I20260709 14:51:41.242203  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.285292  4266 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:41.294759  4266 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.156889  4258 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 14:51:41.296046  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.187129  4243 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.163637  4238 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 14:51:41.297472  4243 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.297490  4258 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:41.298444  4258 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:41.286698  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:41.298817  4258 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:41.174791  4211 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.299878  4243 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:41.307610  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70167.9us
I20260709 14:51:41.308851  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.318890  2751 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:41.319658  2755 raft_consensus.cc:606] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.320046  2755 raft_consensus.cc:614] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:41.187623  4242 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 14:51:41.228883  4292 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:41.337555  4279 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.714s	user 0.000s	sys 0.003s
I20260709 14:51:41.229593  4026 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.308590  4210 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:41.310062  4213 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.324506  4254 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:41.333670  4257 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.003s
W20260709 14:51:41.338733  4274 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 14:51:41.335652  4296 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:41.359189  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 490) took 1183 ms. Trace:
I20260709 14:51:41.359663  3547 rpcz_store.cc:276] 0709 14:51:40.175803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:41.072129 (+896326us) service_pool.cc:224] Handling call
0709 14:51:41.072457 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:41.072903 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:41.072931 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:41.073427 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 14:51:41.137091 (+ 63664us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:41.137133 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:41.137200 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:41.358919 (+221719us) raft_consensus.cc:1685] finished
0709 14:51:41.358942 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:41.359121 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1137,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":63313,"threads_started":1,"wal-append.queue_time_us":156638}
I20260709 14:51:41.361284  4150 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.362514  4290 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.362118  4258 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.363442  4182 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.363673  4216 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.364567  4258 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.235145  4267 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.369065  4211 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.369614  4198 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.356660  4288 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.370970  4205 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:41.372684  4236 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.357127  4228 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:41.375231  4292 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.375653  4267 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.377959  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.378175  4240 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:41.378959  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 420) took 1120 ms. Trace:
I20260709 14:51:41.379395  3541 rpcz_store.cc:276] 0709 14:51:40.258900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.392139 (+133239us) service_pool.cc:224] Handling call
0709 14:51:41.377769 (+985630us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:41.378877 (+  1108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13949312}
I20260709 14:51:41.366247  3860 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.334687  4301 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.387480  4301 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.388415  4270 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:41.388669  3860 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.391719  4207 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:41.392999  4080 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:41.393419  4080 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:41.394284  4080 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.390539  4301 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:41.400203  4300 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
I20260709 14:51:41.409471  4080 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:41.411484  4094 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.868s	user 0.012s	sys 0.012s
I20260709 14:51:41.412186  4094 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.418908  4296 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.419477  4266 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.423372  4094 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.422526  4266 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:41.475951  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 779) took 1213 ms. Trace:
W20260709 14:51:41.479179  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.485127  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.485127  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.487121  2686 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:41.489305  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:41.490154  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.491843  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:41.491432  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:41.492415  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.492954  2752 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:41.492143  2681 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:41.494032  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.495661  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.500047  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:41.501127  2681 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 5
I20260709 14:51:41.501542  4094 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.502162  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 491) took 1325 ms. Trace:
I20260709 14:51:41.502549  4094 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.502660  3542 rpcz_store.cc:276] 0709 14:51:40.176192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:41.072573 (+896381us) service_pool.cc:224] Handling call
0709 14:51:41.072898 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:41.187495 (+114597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:41.187531 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:41.188158 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 14:51:41.501189 (+313031us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:41.501291 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:41.501458 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:41.501572 (+   114us) raft_consensus.cc:1685] finished
0709 14:51:41.501588 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:41.501827 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1008,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":276538,"threads_started":2,"wal-append.queue_time_us":180643}
I20260709 14:51:41.506531  4094 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.507609  4080 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.508790  4080 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.331686  3537 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:41.513521  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:41.523155  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.528537  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 213) took 1027 ms. Trace:
I20260709 14:51:41.529129  2606 rpcz_store.cc:276] 0709 14:51:40.500728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.558314 (+ 57586us) service_pool.cc:224] Handling call
0709 14:51:41.528050 (+969736us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:41.528293 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836669696}
I20260709 14:51:41.529590  4080 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.475950  2607 raft_consensus.cc:2368] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:41.522377  4026 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:41.594062  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.596146  4233 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:41.597874  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.599022  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.477303  2604 rpcz_store.cc:276] 0709 14:51:40.262153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.262453 (+   300us) service_pool.cc:224] Handling call
0709 14:51:40.263166 (+   713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:40.281937 (+ 18771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:40.281990 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:40.327081 (+ 45091us) log.cc:844] Serialized 26 byte log entry
0709 14:51:40.487830 (+160749us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:40.487877 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:40.487949 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:41.234539 (+746590us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:41.474800 (+240261us) raft_consensus.cc:1685] finished
0709 14:51:41.474919 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:41.475575 (+   656us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27304,"raft.queue_time_us":50563,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":9669,"spinlock_wait_cycles":32013056,"thread_start_us":123347,"threads_started":1,"wal-append.queue_time_us":174778}
I20260709 14:51:41.600101  2758 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
W20260709 14:51:41.596139  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 728) took 1372 ms (client timeout 1740 ms). Trace:
W20260709 14:51:41.619356  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70584.4us
I20260709 14:51:41.620647  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.628113  2753 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:41.629982  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:41.632103  4121 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:41.632983  4121 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.633764  4121 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.638180  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:41.632441  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:41.643581  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.646739  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:41.598531  4233 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.655192  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:41.618301  2751 rpcz_store.cc:269] 0709 14:51:40.223959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.232948 (+  8989us) service_pool.cc:224] Handling call
0709 14:51:41.595644 (+1362696us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:41.595913 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2469353216}
I20260709 14:51:41.664860  4270 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.665748  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 787) took 1161 ms. Trace:
W20260709 14:51:41.671626  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 528164us
I20260709 14:51:41.622511  4094 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:41.673066  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.674481  4148 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:41.674481  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 365588us
I20260709 14:51:41.676614  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 731) took 1257 ms. Trace:
I20260709 14:51:41.677145  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:41.679278  4026 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:41.680302  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:41.688832  4094 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.691831  4208 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:41.689983  4094 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.698218  4210 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:41.666280  2607 rpcz_store.cc:276] 0709 14:51:40.504931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.805561 (+300630us) service_pool.cc:224] Handling call
0709 14:51:41.665426 (+859865us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:41.665657 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889010432}
I20260709 14:51:41.701632  4210 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:41.677340  2757 rpcz_store.cc:276] 0709 14:51:40.419100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:40.683187 (+264087us) service_pool.cc:224] Handling call
0709 14:51:40.703125 (+ 19938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:41.185340 (+482215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:41.675743 (+490403us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:41.676300 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023365376}
I20260709 14:51:41.702708  4206 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:41.703693  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:41.695740  4215 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:41.707368  4275 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:41.707989  4275 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:41.708463  4275 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.708720  4215 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:41.709342  4215 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:41.710294  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:41.699409  2681 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
W20260709 14:51:41.715194  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68668.8us
W20260709 14:51:41.716753  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 508315us
I20260709 14:51:41.721350  4310 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.721634  4320 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.722404  4310 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:41.726110  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49336.8us
I20260709 14:51:41.722404  4320 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:41.727630  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 102813us
I20260709 14:51:41.728883  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.729041  4310 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.729759  4320 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:41.731537  2753 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:41.715212  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.736406  4202 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.739236  4215 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.739329  4270 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.742297  4270 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.745764  4270 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.741048  4199 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:41.740991  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:41.759459  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:41.762456  4199 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:41.771080  2752 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:41.776949  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.777864  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:41.778092  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:41.778208  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.739310  4094 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.768198  4199 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.783203  4320 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:41.783599  2756 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:51:41.789397  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 82615.1us
W20260709 14:51:41.804448  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45323.3us
I20260709 14:51:41.810339  4094 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:41.798075  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:41.809237  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.807098  4202 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:41.812922  4026 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:41.815338  4320 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:41.828920  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 156145us
I20260709 14:51:41.837848  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.839437  3546 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:41.845178  4215 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:41.847254  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.847874  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.848328  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.848460  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:41.849980  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.850247  4276 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.851176  4276 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.851697  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.852527  4230 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:41.853127  4230 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:41.854069  4276 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:41.855707  4313 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 14:51:41.856515  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:41.857904  4315 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.858361  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.859261  4308 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 14:51:41.860785  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51873.6us
I20260709 14:51:41.861917  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.863312  4150 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:41.850685  4310 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.864660  4320 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.864779  4310 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:41.824760  4026 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:41.867841  4315 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:41.872816  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 700999us
I20260709 14:51:41.875468  4150 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:41.876564  4150 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.874418  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:41.877986  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45764.1us
I20260709 14:51:41.881855  2678 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:41.882917  4148 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:41.883471  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.884549  2679 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:41.884886  4150 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:41.884984  4148 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:41.885527  4148 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:41.885645  4087 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:41.886951  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.886920  4148 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:41.888033  2681 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:41.888680  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.888873  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.889810  4296 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:41.890944  4250 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.891638  4284 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:41.892490  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:41.892293  4284 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.878780  4026 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:41.899471  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.908615  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:41.894021  4148 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:41.886984  4087 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:41.891695  4250 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.854257  4230 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:41.916079  4284 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:41.906810  4296 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.940436  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.941632  3543 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:51:41.932143  4229 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:41.916875  4087 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:41.957472  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:41.957850  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65596.3us
I20260709 14:51:41.966956  4310 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:41.902994  4210 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:41.975291  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.974331  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:41.983678  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:41.985042  4094 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:41.986526  4296 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:41.986792  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.000053  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.001370  2680 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
W20260709 14:51:42.002328  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53002.7us
I20260709 14:51:42.006429  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:42.015082  4207 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.008849  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:42.021288  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.018970  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:41.977984  4121 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:42.035917  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 161745us
I20260709 14:51:42.037014  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:41.996229  4215 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 14:51:42.043947  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.031222  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.046397  4207 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.047308  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.044281  4215 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:51:42.048885  4215 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:42.049983  4207 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.050117  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.015858  4270 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:42.032559  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.056747  4293 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.027868  4308 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.028723  4315 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:42.063875  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49379.9us
I20260709 14:51:42.061126  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.067919  2679 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:42.076665  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77326us
W20260709 14:51:42.079864  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.059158  4308 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.068459  4150 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:42.094897  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:42.105347  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 95303.4us
I20260709 14:51:42.092412  4308 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.050042  4215 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.118242  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:42.115345  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:42.127733  4315 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:42.128602  4315 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.129174  4315 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.121698  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:42.132992  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.132961  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:42.151100  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.134932  4310 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:42.158167  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.162686  4245 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:42.163120  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:42.163738  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77757.3us
I20260709 14:51:42.155297  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:42.142050  4148 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:42.185307  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:42.175202  4210 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:42.174966  4215 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:42.181634  4202 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:42.203905  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.209198  4314 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:42.209939  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98638us
I20260709 14:51:42.197845  4148 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.172991  4245 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.196983  4250 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:42.203872  4202 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.088299  4087 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:42.215626  4324 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
I20260709 14:51:42.215579  4268 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.220526  4202 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:42.260617  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 101247us
I20260709 14:51:42.260823  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:42.246641  4275 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.005s
I20260709 14:51:42.273880  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.268042  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:42.281502  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.268041  4320 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:42.290725  4266 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.291651  4266 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.293897  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:42.294234  4266 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:42.294734  2757 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:42.295295  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.296309  2603 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:42.297289  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.298189  2608 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:42.267656  4268 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.241071  4229 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 14:51:42.305958  4268 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:42.307181  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:42.283094  4230 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.308950  4230 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.310217  4199 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.275481  4275 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.312501  4199 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:42.312996  4277 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.313576  4277 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.313664  4026 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:42.314947  4230 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.315223  3542 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:42.315845  4277 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.317440  3547 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:42.321616  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.314030  4245 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:42.308282  4148 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.279702  4121 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:42.343122  4332 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:51:42.351313  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46930.3us
I20260709 14:51:42.353067  4316 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.346359  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.370024  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:42.382683  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100190us
I20260709 14:51:42.389621  3547 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:42.393411  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.400063  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:42.402550  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 263308us
I20260709 14:51:42.402307  2650 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:42.404120  2650 leader_election.cc:403] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:42.406726  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.397763  2603 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:42.410352  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.403925  4277 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.411170  2680 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:42.411906  4277 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.413944  4026 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:42.405540  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 139947us
I20260709 14:51:42.404337  4245 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.404379  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.367931  4315 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.419739  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.420173  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.429930  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:42.422922  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.425457  2684 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:42.416749  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:42.435617  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90794.2us
I20260709 14:51:42.394471  2754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:42.449146  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.449471  2754 raft_consensus.cc:606] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.449908  2754 raft_consensus.cc:614] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:42.450066  4239 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.421443  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.450749  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:42.449173  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.453709  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.414819  4277 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:42.458488  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.427757  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.436733  4121 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.426537  4315 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:42.464582  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.454217  4208 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:42.487485  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.467799  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:42.488361  3541 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.423467  4245 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.489656  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:42.490746  4245 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:42.491179  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:42.471566  4275 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.451442  4026 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.492585  2685 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:42.492507  4026 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.493880  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.495416  4250 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.495888  4335 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.496335  3537 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:42.477030  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.480471  4296 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:42.494952  4331 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.497382  2682 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:42.481479  4031 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:42.498402  4031 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.484838  4320 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.437525  4202 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.499166  4320 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:42.499696  4320 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.446714  4230 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:42.496210  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:42.496786  4335 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:42.501497  4230 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.504714  4031 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.438704  4308 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.473999  4121 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.506817  4031 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:42.524744  4087 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.505703  4202 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.529441  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:42.527072  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111140us
I20260709 14:51:42.515383  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:42.517258  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.540516  2754 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:42.520313  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:42.509425  4121 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.556176  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:42.557230  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.523548  4230 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.559765  2680 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:42.512969  4026 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.562976  4268 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 14:51:42.564613  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44421.4us
I20260709 14:51:42.565263  2607 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:42.566123  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.583526  4026 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.584223  4026 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.566584  2753 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:42.583658  2607 raft_consensus.cc:606] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:42.566174  4292 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.568142  4210 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.547279  4335 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:42.570465  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55227us
I20260709 14:51:42.557904  4339 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.571079  4198 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.571624  2756 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:42.519711  4245 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 14:51:42.519889  4315 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 14:51:42.508095  4308 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.596028  4026 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:42.566849  4316 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.603813  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:42.578554  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.601788  4206 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.612578  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.614228  4349 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:42.618548  4349 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.618983  4349 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:42.619032  4277 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:42.622113  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:42.623266  4268 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:42.624877  4026 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:42.625846  4026 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:42.626703  4094 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:42.627357  4094 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.630326  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:42.634958  4229 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:42.636291  4229 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:42.636677  4229 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.637567  4229 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.606966  4202 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.639591  4316 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.639421  4202 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.641520  3541 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:42.642786   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 14:51:42.603237  4292 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.646096  4292 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.634650  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.610908  4332 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 14:51:42.647804  4332 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:42.648277  4332 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.649243  4206 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.649468  4332 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.612864  4210 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.651772  4206 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.610607  4250 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:42.653005  2652 leader_election.cc:400] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:42.653381  2652 leader_election.cc:403] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:42.653494  4210 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:42.654844  4148 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:42.656128  4148 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:42.656675  4320 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.657951  2606 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 4. Ops: []
I20260709 14:51:42.659162  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:42.659570  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.659689  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.611969  4198 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.660359  2609 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:42.660641  3543 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:42.661414  4239 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:42.661722  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.662261  4198 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:42.612211  4339 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.658522  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:42.664709  4339 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:42.664801  4094 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.667187  2612 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:42.659896  2605 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.658355  4314 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:42.670217  4314 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.643407  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.670415  2686 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:42.671156  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.671388  4314 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.672086  3538 raft_consensus.cc:2368] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:42.671972  4315 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:42.672761  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.672883  4315 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.673491  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.674842  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.674997  4202 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.675662  3536 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:42.675818  3542 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:42.677443  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.678385  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.676525  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.678902   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:42.679345  4314 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:42.679627  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120286us
I20260709 14:51:42.677652  2435 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 5, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:42.683900  4229 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:42.688846  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.689313  2604 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:42.631366  2756 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:42.691497  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.692324  2680 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:42.696908  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.697674  2683 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.695004  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:42.700049  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.700736  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.700901  2684 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:42.701637  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.702016  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:42.702291  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:42.702669  2680 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:42.703935  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105571us
I20260709 14:51:42.704864  2608 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:42.703548  2576 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.561s (SENT) [suppressed 9 similar messages]
I20260709 14:51:42.714915  4345 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.715533  4345 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.715914  4345 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.720288  3536 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:42.721385  2435 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 2, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:42.723601  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.724434  2685 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:42.725193  4239 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.725567  4239 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.725870  4239 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.742642  2683 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:42.674322  4315 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:42.744350  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 131866us
I20260709 14:51:42.750311  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.705310  4320 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:42.759747  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.768210  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.769861  4343 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:42.770323  4343 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:42.771435  4150 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.772063  4150 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:42.772626  2608 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:42.775806  4332 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:42.779755  3536 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 3. Ops: []
I20260709 14:51:42.780916  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.782507  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.783519  3545 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:42.784075  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.784437  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.763201  2685 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:42.785465  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.783231  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.786509  3540 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:42.782646  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.787679  3547 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:42.760603  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:42.764545  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.742058  4339 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:42.804370  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:42.809460  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49890.2us
I20260709 14:51:42.817946  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.819134  3536 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:42.760100  2435 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:51:42.811573  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:42.824976  2756 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:42.825297  4342 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.826160  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.827530  4342 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:42.827757  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77601.7us
I20260709 14:51:42.828085  4342 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.828627  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.828889  4307 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:42.829401  2605 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:42.829674  4307 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:42.829866  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:42.830189  4307 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.830627  4307 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.811827  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:42.835508  3547 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.836092  2753 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:42.811319  2433 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 95) took 9577 ms. Trace:
I20260709 14:51:42.809403  4339 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.806967  4150 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:42.811751  4358 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 14:51:42.874482  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.863987  4348 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:42.864751  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.866341  4349 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:42.813634  4335 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.868629  4345 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:42.812237  4148 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:42.840657  4121 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.875718  3543 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:42.864260  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:42.884296  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.874053  2435 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:42.857084  4339 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:42.892445  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.889633  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.885294  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:42.872311  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.889748  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.888279  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:42.891553  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52710.7us
I20260709 14:51:42.896773  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.894917  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:42.902873  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:42.822063  4353 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 14:51:42.879240  4348 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.895375  4150 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.911768  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.916918  4239 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.839746  2433 rpcz_store.cc:276] 0709 14:51:33.234058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:33.234323 (+   265us) service_pool.cc:224] Handling call
0709 14:51:33.535459 (+301136us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001130d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5b0090 00007fa16e64ef3a 00007fa1744051cc 00007fa17450337e 00007fa1729c42a4 00007fa1729c4232 00007fa1729c4161 00007fa1729c40dc 00007fa1729c3392 00007fa16c98148c 00007fa16c9808c6
0709 14:51:42.800709 (+9265250us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:40.900003 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:41.082150 (+182147us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:41.088982 (+  6832us) write_op.cc:435] Acquired schema lock
0709 14:51:41.089013 (+    31us) tablet.cc:662] Decoding operations
0709 14:51:41.096989 (+  7976us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:41.101444 (+  4455us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:41.108129 (+  6685us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:51:41.136578 (+ 28449us) tablet.cc:701] Row locks acquired
0709 14:51:41.136611 (+    33us) write_op.cc:260] PREPARE: finished
0709 14:51:41.156035 (+ 19424us) write_op.cc:270] Start()
0709 14:51:41.156193 (+   158us) write_op.cc:276] Timestamp: P: 1783608701155972 usec, L: 0
0709 14:51:41.156214 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 14:51:41.240538 (+ 84324us) log.cc:844] Serialized 540 byte log entry
0709 14:51:42.300957 (+1060419us) op_driver.cc:464] REPLICATION: finished
0709 14:51:42.351717 (+ 50760us) write_op.cc:301] APPLY: starting
0709 14:51:42.351873 (+   156us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:42.352042 (+   169us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:42.352231 (+   189us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:42.352588 (+   357us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:42.352862 (+   274us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 14:51:42.352943 (+    81us) write_op.cc:312] APPLY: finished
0709 14:51:42.353446 (+   503us) log.cc:844] Serialized 22 byte log entry
0709 14:51:42.353682 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:42.353819 (+   137us) write_op.cc:454] Released schema lock
0709 14:51:42.353887 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:42.354423 (+   536us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:42.354466 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29327,"reported_tablets":10,"spinlock_wait_cycles":425786112,"child_traces":[["op",{"apply.queue_time_us":50312,"apply.run_cpu_time_us":2693,"apply.run_wall_time_us":2693,"num_ops":1,"prepare.queue_time_us":437840,"prepare.run_cpu_time_us":11507,"prepare.run_wall_time_us":798294,"raft.queue_time_us":476018,"raft.run_cpu_time_us":6190,"raft.run_wall_time_us":82976,"replication_time_us":1119338,"thread_start_us":843790,"threads_started":4,"wal-append.queue_time_us":398240}]]}
I20260709 14:51:42.919786  4026 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:42.920374  4026 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:42.920818  4026 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:42.920904  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.904466  4210 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:42.908221  4094 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.924706  4210 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:42.926041  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:42.925815  4210 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:42.924710  4094 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.927533  4094 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.929131  4121 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:42.890165  4148 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.931859  2682 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:42.892660  4202 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:42.937486  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:42.873641  4343 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:42.900208  4320 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:42.912393  4150 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:42.931524  4348 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.984824  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:42.971740  4334 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 14:51:42.933755  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:42.994189  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:43.001590  2754 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:42.935446  4148 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:42.908924  4206 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 14:51:42.904719  4335 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.007403  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:42.943495  4202 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.014099  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:42.990561  4150 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.019404  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.005664  4315 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:43.025753  2759 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:43.026959  2757 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:42.998821  4320 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:43.024374  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 98429.1us
I20260709 14:51:43.031494  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:43.013062  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.019780  4351 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.015098  4354 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.045917  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:43.055733  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.048415  4364 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:43.047283  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.089529  3542 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 14:51:43.061275  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.024899  4202 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:43.107910  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67466.3us
I20260709 14:51:43.043834  4342 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.038604  4292 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:43.151994  4370 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:43.155149  4239 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.139592  4292 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:43.156292  3542 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:43.111478  4351 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.162513  2680 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
W20260709 14:51:43.172176  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 134189us
I20260709 14:51:43.176426  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:43.159381  3541 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:43.190862  3547 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:43.187158  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.192835  2606 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.164721  4292 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:43.107760  4364 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:43.204795  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 106759us
I20260709 14:51:43.206673  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:43.127861  4354 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.196033  4349 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.194955  4308 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.230449  4363 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:43.231091  4363 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:43.231994  4363 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.233229  4363 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:43.235383  4376 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.096s	user 0.002s	sys 0.007s
I20260709 14:51:43.235168  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.238492  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.239529  3541 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:51:43.246225  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:43.248936  2603 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.585s	user 0.014s	sys 0.000s
I20260709 14:51:43.249449  2603 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:51:43.252250  4307 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.253629  4094 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.252847  4150 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.256122  3543 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:43.256342  4364 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.263420  4348 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.265799  4348 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.268579  4354 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.212787  4345 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.272629  4284 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:43.273183  4284 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:43.274112  4284 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.277585  4342 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.280377  4094 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.281342  4308 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.286531  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.287694  2678 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.292809  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:43.300540  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:43.312276  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:43.314002  2757 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:43.317034  2606 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:43.212395  4353 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:43.261703  4148 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.319447  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.322849  2680 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.324820  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.327064  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.336798  2680 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:43.327854  2603 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:43.315892  4150 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:43.359969  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:43.360970  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.364825  2606 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:43.359387  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:43.371356  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.381453  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.383018  4336 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:43.221194  4320 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.384882  2684 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:43.385363  4335 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.386986  4351 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.313778  4026 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.362406  4364 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:43.322014  4087 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:43.382081  4202 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.399492  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.415666  2575 leader_election.cc:400] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:43.416285  2575 leader_election.cc:403] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:43.398371  2685 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:43.418519  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.419307  2606 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.421545  4343 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.421808  4333 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.422493  4333 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.409219  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.382589  2678 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:43.398440  4277 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:43.441058  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.392951  4026 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.397199  4336 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:43.444440  2604 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:43.449513  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:43.445405  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 233661us
I20260709 14:51:43.446611  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.477607  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.478060  2609 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:43.452078  4277 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:43.476872  2680 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:43.479660  4277 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.424471  4353 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:43.481841  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:43.453194  4206 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:43.455312  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:43.454631  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:43.463110  2684 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:43.449724  4331 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:43.446784  4230 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:43.435904  4335 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:43.438786  4148 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.458576  4202 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.484359  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.432185  4087 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:43.398136  4364 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.510648  2606 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:43.494391  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.475409  3541 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 14:51:43.496459  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:43.517400  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:43.523504  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46027.3us
I20260709 14:51:43.390784  4339 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.507051  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.535272  3542 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:43.507932  2609 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:43.506449  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:43.492007  4026 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:43.504513  4307 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:43.532271  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:43.503813  4094 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 14:51:43.542048  4206 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:43.502405  4292 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.547935  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.527010  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.550297  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:43.530272  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.529718  4202 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:43.550442  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.514052  4335 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.564343  3537 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:43.562009  3542 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:43.568308  2757 raft_consensus.cc:2368] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:43.570767  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:43.571017  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64684.9us
I20260709 14:51:43.572700  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.574100  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:43.577374  4202 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.578029  4202 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.579156  4371 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.579816  4371 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.580386  4335 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.581252  2754 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:43.580401  4202 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:43.582747  4365 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.583388  4365 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.584398  4292 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.586184  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:43.481864  4353 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.498731  4331 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.570036  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:43.603281  2680 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:43.551544  4333 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.610105  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:43.612270  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.613035  3543 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
W20260709 14:51:43.615391  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 67601.5us
I20260709 14:51:43.617657  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:43.620170  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.586694  4198 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.622320  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.627820  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.628237  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:43.628650  2612 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:43.570721  4245 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:43.621701  4210 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:43.604646  4239 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:43.621745  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.645493  2680 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:43.609704  4148 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:43.586169  4292 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:43.623256  4250 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:43.627666  4352 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:43.628362  4351 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:43.655314  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89487.6us
I20260709 14:51:43.633715  4266 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:43.661727  3411 leader_election.cc:400] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:43.662211  3411 leader_election.cc:403] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:43.576202  4206 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.608003  4341 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.217s
I20260709 14:51:43.652025  4245 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:43.673767  4385 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 14:51:43.675419  4366 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 14:51:43.657814  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:43.636566  4198 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.659839  4351 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:43.658310  4239 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:43.694458  4198 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:43.695649  4239 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.664804  4333 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:43.697433  4379 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:43.628994  4087 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.562510  4336 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.684403   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:43.659963  4320 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.687282  4351 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.690555  2605 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:43.684893  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.660658  4266 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:43.670586  4250 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:51:43.726197  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104542us
I20260709 14:51:43.678897  4210 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:43.681149  4292 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:43.732208  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.732230  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.744882  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:43.751569  4331 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.730805  4266 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:43.738600  4292 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:43.766420  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:43.759528  4363 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.526s	user 0.015s	sys 0.000s
I20260709 14:51:43.794405  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.698118  4379 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.801865  4379 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.743641  4320 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.803575  4292 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.804885  4320 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.806238  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.774317  4397 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.807495  4331 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.775321  4395 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:43.808128  4331 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.775164  4250 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.769117  4245 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.772192  4388 raft_consensus.cc:2764] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 14:51:43.796043  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.796408  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.799271  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.804134  4230 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3.
I20260709 14:51:43.771138  4391 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:43.824796  2680 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:43.834088  4284 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.836073  4365 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:43.841442  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45123.2us
I20260709 14:51:43.844970  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:43.849254  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.850379  2686 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:43.807729  4397 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.842080  4094 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:43.855336  4087 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:43.859740  4331 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.865190  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.875481  4349 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.843866  4198 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.876756  4198 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.875859  4210 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:43.876591  4349 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.884424  4349 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:43.885507  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:43.890558  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.842947  4395 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.916373  4087 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.854707  4397 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.920429  4266 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:43.949838  2606 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:43.962954  4331 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:43.970115  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.970530  4331 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:43.963593  4391 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:43.980057  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.981110  3545 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:43.976876  3540 raft_consensus.cc:2368] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:43.983786  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:43.984633  3540 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:43.984778  4392 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.985319  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.985391  4392 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.986462  3538 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:43.959830  4395 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.987699  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:43.987950  4392 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.963339  4239 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.988617  3546 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:43.983896  4331 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:43.988714  4239 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.972291  4397 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:43.989974  4397 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.991077  4239 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.988839  4395 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:43.992188  4397 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:43.979256  4351 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:43.996083  2752 raft_consensus.cc:2368] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:43.995867  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.997056  4351 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:43.997299  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:43.998637  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.001181  2686 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:44.007120  3540 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:43.998149  4391 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:43.963228  4087 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.015087  2686 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.019513  3545 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:44.020576  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.022799  3540 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 14:51:44.024847  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.023226  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.024317  3540 pending_rounds.cc:85] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 1
I20260709 14:51:44.027454  2609 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:44.027299  3540 pending_rounds.cc:107] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:51:44.032948  3540 raft_consensus.cc:2889] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:44.022069  4198 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.036645  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.025580  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.030190  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:44.029964  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.042270  2678 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:44.043612  3538 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.044445  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.044790  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.045207  2611 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.045490  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.046207  3541 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:44.049506  2576 leader_election.cc:400] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:44.045703  2605 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:44.050151  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.050016  2576 leader_election.cc:403] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:44.053285  4087 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.055627  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.056085  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.057565  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.058516  4266 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.060289  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:44.078117  3547 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.063782  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.064801  4391 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.064908  4087 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:44.095880  4202 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:44.096432  4202 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:44.065306  4364 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:44.066677  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.067065  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.097699  2752 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:44.097505  4202 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.067755  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.102067  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.103147  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.103536  4202 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.040776  4336 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.083425  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.108530  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.109468  2751 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:44.109923  2752 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:44.110777  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.111454  4372 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.112141  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.112968  2760 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:44.114284  2755 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.123636  4333 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:44.123910  4026 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:44.124466  4368 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:44.124794  4333 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:44.125273  4333 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:44.125173  4368 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.126343  4333 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.127024  4245 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:44.127499  4333 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.127808  4245 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.130402  4245 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.132681  4239 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:44.133301  4239 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.133648  4031 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.134231  4031 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.135000  4357 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.135726  4357 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.136417  4031 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.138142  4357 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.147351  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.149230  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.149686  2684 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 14:51:44.151057  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.151273  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.151894  2682 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:44.152249  2685 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:44.153443  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.154220  2679 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:44.158164  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:44.159226  2760 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:44.160085  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.160856  2680 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:44.167613  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.168781  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:44.171085  4314 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:44.171675  4314 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:44.172650  4314 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.173344  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.174074  2612 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:51:44.176203  4308 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:44.181000  3541 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:44.182729  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.185241  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 819) took 1019 ms. Trace:
I20260709 14:51:44.185710  2604 rpcz_store.cc:276] 0709 14:51:43.165904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:43.167049 (+  1145us) service_pool.cc:224] Handling call
0709 14:51:43.167575 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:43.449391 (+281816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.185036 (+735645us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54002cdb54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:44.185187 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636128000}
I20260709 14:51:44.188342  2605 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:44.191264  4371 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.186036  2612 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:44.087445  2611 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:44.086766  3547 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:44.087359  2435 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 163) took 9353 ms. Trace:
I20260709 14:51:44.197047  4292 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.089107  2607 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:44.090186  3538 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:44.091027  4339 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:44.091516  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.200178  2610 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:44.200320  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.201138  3543 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:44.056496  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:44.098532  4351 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:44.202527  4351 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:44.099531  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49494.3us
I20260709 14:51:44.203853  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:44.204140  2577 leader_election.cc:400] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:44.204492  2577 leader_election.cc:403] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:44.205973  4384 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.208609  4351 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.094934  4401 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:44.211257  4401 raft_consensus.cc:2764] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 14:51:44.212767  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.205986  4368 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.200968  4239 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.195981  2606 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:44.215583  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.216598  4339 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:44.217123  4339 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:44.217654  4335 raft_consensus.cc:2764] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 14:51:44.218060  4339 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.099023  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.219503  3542 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:44.215757  4121 raft_consensus.cc:993] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: : Instructing follower 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:44.220993  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:44.221381  4121 raft_consensus.cc:1081] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Signalling peer 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:51:44.222536  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.223786  4352 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:44.205807  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.224713  4352 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:44.225782  4352 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.226764  4150 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.228179  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.229485  2677 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.229457  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:44.231848  4308 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:44.232187  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.232651  4308 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:44.233042  4308 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:44.233392  4308 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.235718  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.236413  2606 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.197247  2435 rpcz_store.cc:276] 0709 14:51:34.734163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:34.734945 (+   782us) service_pool.cc:224] Handling call
0709 14:51:44.087280 (+9352335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:43.168644 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:43.278915 (+110271us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:43.278974 (+    59us) write_op.cc:435] Acquired schema lock
0709 14:51:43.278995 (+    21us) tablet.cc:662] Decoding operations
0709 14:51:43.279876 (+   881us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:43.280020 (+   144us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:43.280061 (+    41us) tablet.cc:685] Acquiring locks for 4 operations
0709 14:51:43.280700 (+   639us) tablet.cc:701] Row locks acquired
0709 14:51:43.280755 (+    55us) write_op.cc:260] PREPARE: finished
0709 14:51:43.280965 (+   210us) write_op.cc:270] Start()
0709 14:51:43.281156 (+   191us) write_op.cc:276] Timestamp: P: 1783608703280885 usec, L: 0
0709 14:51:43.281185 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 14:51:43.282049 (+   864us) log.cc:844] Serialized 1705 byte log entry
0709 14:51:43.501717 (+219668us) op_driver.cc:464] REPLICATION: finished
0709 14:51:43.651310 (+149593us) write_op.cc:301] APPLY: starting
0709 14:51:43.651415 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:43.659483 (+  8068us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:43.669835 (+ 10352us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:43.684256 (+ 14421us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:43.684462 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 14:51:43.684530 (+    68us) write_op.cc:312] APPLY: finished
0709 14:51:43.687499 (+  2969us) log.cc:844] Serialized 40 byte log entry
0709 14:51:43.687731 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:43.687862 (+   131us) write_op.cc:454] Released schema lock
0709 14:51:43.687912 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:43.688338 (+   426us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:43.688366 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":126968,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":131115,"apply.run_cpu_time_us":3344,"apply.run_wall_time_us":37019,"num_ops":4,"prepare.queue_time_us":113634,"prepare.run_cpu_time_us":4830,"prepare.run_wall_time_us":191645,"replication_time_us":220332,"thread_start_us":122205,"threads_started":2}]]}
I20260709 14:51:44.241258  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:44.226080  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.242225  2609 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:44.245148  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.246443  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:44.084901  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.244606  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:44.246295  3540 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:44.273945  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47959.5us
I20260709 14:51:44.274864  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:51:44.275087  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54201.4us
I20260709 14:51:44.283681  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:44.283684  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:44.280977  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.292290  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.293646  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.294382  2686 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.298136  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.298408  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.299517  2680 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
W20260709 14:51:44.303507  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53628.3us
I20260709 14:51:44.256877  2677 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.300913  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.272542  4206 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:44.310650  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.311240  2679 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:44.308792  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.272820  4385 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:44.310241  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.241283  4372 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:44.351521  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.314585  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.286979  4245 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:44.259507  2754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:42488
W20260709 14:51:44.352352  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56341.5us
I20260709 14:51:44.257637  4230 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.283666  4275 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.313621  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:44.353916  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45250.3us
W20260709 14:51:44.355880  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51244.5us
I20260709 14:51:44.357085  2754 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:51:44.358564  2754 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:44.315326  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 230477us
I20260709 14:51:44.359995  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:44.305020  4026 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:44.296739  4087 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 14:51:44.362778  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52653.8us
I20260709 14:51:44.364051  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.364615  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.365737  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.368327  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.302310  4401 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:44.295709  4320 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.310601  4198 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.369408  2755 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:44.345455  4206 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.353052  4385 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.392189  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.304152  4031 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 14:51:44.344481  4390 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.389226  2686 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:44.410168  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.295166  4333 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 14:51:44.411116  2606 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:44.356257  4388 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:44.302057  4335 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 14:51:44.304842  4387 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 14:51:44.356863  4245 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.361506  4026 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:44.435163  4245 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:44.435334  4026 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.434511  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:51:44.437150  2430 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.398223  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:44.314993  4331 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.401703  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.372685  4094 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:44.314662  4229 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 14:51:44.371141  4395 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:44.428256  2754 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:44.443207  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 79262.4us
I20260709 14:51:44.392395  4150 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:44.442083  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.445294  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.463929  2610 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.362308  4369 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 14:51:44.412781  4206 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:44.423388  4320 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:44.458416  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:44.417505  4385 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.486151  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.488065  4379 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:44.488687  4416 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.449690  4293 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:44.489634  4293 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:44.489303  4416 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.490067  4293 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.372342  4397 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 14:51:44.434579  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.492307  4339 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:44.387959  4343 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 14:51:44.494938  4416 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:44.479507  2754 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:44.399086  4087 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:44.446395  4094 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.495591  2430 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 4, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.496666  4087 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:44.497547  2430 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 3, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.421257  4250 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:44.499480  4250 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.500597  4230 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:44.502349  4250 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.496780  4094 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.425532  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.455085  4198 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 14:51:44.524776  4308 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.551396  4308 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.526649  2752 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:44.556051  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.537930  2430 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.540786  4366 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 14:51:44.547770  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:44.560417  4379 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:44.561079  4379 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:44.563310  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.558301  4366 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:44.564572  2604 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:44.565176  4366 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.576370  4384 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4.
I20260709 14:51:44.558324  2685 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.579173  2606 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.584160  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.585358  2679 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.546000  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.590247  2680 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 2: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:44.593317  4397 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:44.567886  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.496044  4385 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.607149  4250 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:51:44.609453  4412 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
I20260709 14:51:44.609804  4331 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:44.609220  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150860us
I20260709 14:51:44.610572  2605 raft_consensus.cc:2368] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3.
W20260709 14:51:44.611047  4409 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.094s	user 0.003s	sys 0.002s
I20260709 14:51:44.613044  2606 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
W20260709 14:51:44.613330  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:44.615267  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.615862  3543 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:44.617631  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.619560  2606 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:51:44.618017  4413 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 14:51:44.618872  3540 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.620265  2757 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:44.620806  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.626375  4390 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:44.626896  4390 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.617812  4366 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.627254  4390 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.621208  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.622990  3538 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:44.623629  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.629580  2685 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.630172  2576 leader_election.cc:400] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:44.630564  2576 leader_election.cc:403] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:44.626304  4384 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 14:51:44.631619  4198 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:44.631556  2430 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.634353  2430 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 4, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:44.620321  4351 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.636087  2756 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:44.637933  4206 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.640797  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.641294  4351 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.642875  4388 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.643639  4388 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.644719  4397 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:51:44.645910  4416 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:44.646365  4416 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.522711  4245 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.646728  4388 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.562326  4379 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.646703  4416 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:44.651336  4379 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.647164  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.652998  4374 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.653631  4374 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.656183  4391 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:44.658039  4391 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 14:51:44.659044  4382 consensus_queue.cc:1175] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "56620e612f0549e6abf320aa80398e74" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 3 index: 3 } }
I20260709 14:51:44.659626  2684 raft_consensus.cc:2368] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
I20260709 14:51:44.661861  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.662590  2684 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:44.665536  4121 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:44.666067  4121 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:44.667320  4374 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:44.668172  4385 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:44.671126  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 312546us
I20260709 14:51:44.682199  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:44.683146  2757 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.688853  2608 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:44.647990  2610 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:44.699783  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.703615  2684 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:44.712284  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:44.710299  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60326us
I20260709 14:51:44.706692  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.654589  4388 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:44.721738  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.721743  2430 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 2, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:44.672986  4401 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:44.667253  4121 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.705075  4198 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:44.667409  4245 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.722754  4384 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:44.727136  4239 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:44.736292  2726 leader_election.cc:400] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3.
I20260709 14:51:44.724835  4371 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.753419  2679 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:44.753796  2726 leader_election.cc:403] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:44.752804  2683 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:44.755611  2685 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
W20260709 14:51:44.756840  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:44.761369  4418 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:44.762341  4418 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.765450  4418 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.757326  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:44.725754  4382 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:44.772579  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.732401  4374 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 14:51:44.795077  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:44.799558  2754 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:44.751420  4384 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.791330  4364 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:44.788939  4397 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:44.804870  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:44.807240  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:44.818269  2612 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:44.808640  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:44.771129  4239 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:44.820653  4239 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:44.824973  3543 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:44.825502  3543 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:44.808136  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:44.827009  2754 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 7
I20260709 14:51:44.828574  4150 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:44.829210  4150 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:44.786020  4121 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:44.781862  4206 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:44.830574  4150 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.804989  4425 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:44.741336  4380 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 14:51:44.848513  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 442056us
I20260709 14:51:44.851501  2754 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:44.852770  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:44.854176  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.815343  4384 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:44.941670  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 216158us
I20260709 14:51:44.866475  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:44.811527  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 826) took 1199 ms. Trace:
I20260709 14:51:44.816694  4395 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:44.863612  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:44.859917  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52400.4us
I20260709 14:51:44.776706  4371 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:44.786921  4382 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.944628  3547 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:44.946061  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:44.812649  4308 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 14:51:44.797825  4293 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:44.955471  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92021.3us
I20260709 14:51:44.859416  4414 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:44.961855  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:44.965457  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:44.889312  4397 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:44.971156  3547 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:44.806422  4390 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:44.977113  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:44.974566  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:44.978163  3545 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:44.941457  4364 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:44.981556  4364 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.850234  4425 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:44.983546  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:44.984467  2754 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:44.994355  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:44.946165  4385 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:45.000698  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.003473  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:45.004629  3545 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:45.008574  2685 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:44.947422  4424 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:45.013116  2606 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:44.962818  4031 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:44.962567  4333 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:45.016748  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:44.970602  4148 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:45.036566  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:45.022897  2684 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:44.975404  4229 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:45.019585  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:45.026402  4424 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.045022  3536 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:45.045872  4424 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:45.030277  4429 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 14:51:45.029934  4432 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 14:51:44.971637  4308 consensus_queue.cc:1243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:51:45.031935  4293 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:45.009121  4349 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.041350  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:45.028621  3547 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:45.058858  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:45.048817  4239 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.010546  4352 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.063484  2723 leader_election.cc:400] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
I20260709 14:51:45.088165  2723 leader_election.cc:403] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:51:45.037981  4275 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.000s
I20260709 14:51:45.064662  4031 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:45.065398  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:45.070446  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:44.958395  4395 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:44.943058  2603 rpcz_store.cc:276] 0709 14:51:43.606098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:43.622478 (+ 16380us) service_pool.cc:224] Handling call
0709 14:51:44.123242 (+500764us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:44.123487 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:44.123742 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:44.361476 (+237734us) log.cc:844] Serialized 14 byte log entry
0709 14:51:44.747155 (+385679us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:44.769360 (+ 22205us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:44.775783 (+  6423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.789621 (+ 13838us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:44.805723 (+ 16102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":332082,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":1551,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7555,"spinlock_wait_cycles":5157120,"thread_start_us":584246,"threads_started":2,"wal-append.queue_time_us":488881}
I20260709 14:51:45.049146  4416 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:45.046528  4094 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.000s
I20260709 14:51:44.940869  4401 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.026664  4385 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:45.095737  4349 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:45.105041  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46322.3us
I20260709 14:51:45.106146  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:45.054538  4372 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:45.150503  2757 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:45.273111  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:45.074960  4333 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:45.199908  4094 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:45.214478  4382 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.181346  4397 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:45.126271  4250 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.000s
I20260709 14:51:45.101995  4266 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
I20260709 14:51:45.119467  4374 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 14:51:45.108367  4275 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:45.131916  4420 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:51:45.333199  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60238.5us
W20260709 14:51:45.353083  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:45.198709  3546 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:45.309470  4351 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3.
I20260709 14:51:45.367735  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:45.368763  2679 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.343279  4245 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:45.347602  4250 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:51:45.121541  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 173426us
W20260709 14:51:45.141012  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:45.401360  2609 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:51:45.104846  2684 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.136443  2685 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:45.364708  4390 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.427181  2756 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:51:45.436074  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:45.100631  4425 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:45.446647  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:45.449638  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.450382  2757 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:45.451992  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.454741  2753 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:45.455116  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:45.455902  2751 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 6
I20260709 14:51:45.456616  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:45.487170  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.043s	sys 0.097s
W20260709 14:51:45.487743  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.043s	sys 0.098s
I20260709 14:51:45.427798  4331 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:45.491712  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.492058  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.493724  2612 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:45.486606  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:45.493759  2606 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.495434  3547 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:45.436169  4436 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.103888  4352 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.507707  4416 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:45.502205  4331 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.518868  4208 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:45.519325  4208 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:45.504415  4275 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.521137  4275 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:45.523123  4417 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.481s	user 0.003s	sys 0.001s
W20260709 14:51:45.524273  4427 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.262s	user 0.001s	sys 0.001s
I20260709 14:51:45.505300  4266 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:45.525224  4266 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:45.525660  4266 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:45.500351  4250 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 14:51:45.529335  4121 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
W20260709 14:51:45.513001  4434 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 14:51:45.513763  4441 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 14:51:45.452316  4333 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.516722  4422 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:45.517448  4349 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:45.517992  4366 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.890s	user 0.010s	sys 0.000s
W20260709 14:51:45.562852  4438 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 14:51:45.512593  4150 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:45.536464  2612 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:45.452654  4415 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:45.496934  4026 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.061s	user 0.015s	sys 0.002s
W20260709 14:51:45.543955  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51110.6us
I20260709 14:51:45.571537  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.489714  4371 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:45.550895  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64365.2us
I20260709 14:51:45.497186  4425 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:45.500066  4245 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 14:51:45.540634  4331 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:45.492884  4432 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 14:51:45.436462  4385 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:45.596110  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146725us
W20260709 14:51:45.596097  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75363.4us
I20260709 14:51:45.556491  4422 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:45.608029  2751 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:45.570447  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.530879  4436 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.612166  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:45.596607  4425 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:45.623698  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:45.625383  2757 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:45.625603  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:45.626685  2754 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:45.627381  2753 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:45.629952  4121 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:45.630468  4444 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:45.631283  4430 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 14:51:45.630883  4121 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.633522  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 714) took 1079 ms. Trace:
I20260709 14:51:45.633955  2677 rpcz_store.cc:276] 0709 14:51:44.553778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.555328 (+  1550us) service_pool.cc:224] Handling call
0709 14:51:44.555782 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.556232 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.556273 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:44.556802 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 14:51:44.944866 (+388064us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:44.947074 (+  2208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.957622 (+ 10548us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:45.633217 (+675595us) raft_consensus.cc:1685] finished
0709 14:51:45.633258 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:45.633428 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1491,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":305635,"threads_started":1,"wal-append.queue_time_us":790042}
I20260709 14:51:45.635859  4121 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.619051  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.616791  4284 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 14:51:45.639724  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 719) took 1042 ms. Trace:
I20260709 14:51:45.609156  4422 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:45.589072  4026 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.615972  4450 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.588816  4415 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.650921   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:45.628826  4425 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:45.648938  2680 rpcz_store.cc:276] 0709 14:51:44.597148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.597572 (+   424us) service_pool.cc:224] Handling call
0709 14:51:44.598045 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.598482 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.598511 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:44.601142 (+  2631us) log.cc:844] Serialized 26 byte log entry
0709 14:51:44.792323 (+191181us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:44.792381 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.792477 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:45.639464 (+846987us) raft_consensus.cc:1685] finished
0709 14:51:45.639489 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:45.639652 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":620,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":216,"thread_start_us":190758,"threads_started":1,"wal-append.queue_time_us":498031}
I20260709 14:51:45.646364  2603 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:45.672205  4379 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:45.672753  4379 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:45.673130  4379 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:45.660429  2605 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.676594  4284 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:45.680104  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:45.681048  3547 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.686987  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:45.691637  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:45.640305  4380 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 14:51:45.695670  3547 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:51:45.665967  4382 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:45.677114  4284 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:45.703378  4452 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 14:51:45.707207  4352 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.707752  4451 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:45.708356  4439 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 14:51:45.710291  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 475) took 1028 ms. Trace:
I20260709 14:51:45.710752  3540 rpcz_store.cc:276] 0709 14:51:44.682002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.682181 (+   179us) service_pool.cc:224] Handling call
0709 14:51:44.682674 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.725269 (+ 42595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.725305 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:44.725919 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 14:51:44.951551 (+225632us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:44.951602 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.951691 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:45.710047 (+758356us) raft_consensus.cc:1685] finished
0709 14:51:45.710076 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:45.710223 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54321,"raft.queue_time_us":76810,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8585,"thread_start_us":198527,"threads_started":1,"wal-append.queue_time_us":666573}
I20260709 14:51:45.719808  4026 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.722069  4436 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:45.723963  2753 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:45.691303  4372 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 14:51:45.726994  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 114925us
I20260709 14:51:45.723485  4351 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.730775  4351 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:45.734194  4351 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.745234  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 474) took 1066 ms. Trace:
I20260709 14:51:45.745000  4208 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.745770  3538 rpcz_store.cc:276] 0709 14:51:44.679028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.720167 (+ 41139us) service_pool.cc:224] Handling call
0709 14:51:44.786396 (+ 66229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.888853 (+102457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.941293 (+ 52440us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:45.016328 (+ 75035us) log.cc:844] Serialized 26 byte log entry
0709 14:51:45.615862 (+599534us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:45.615939 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:45.616061 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:45.744900 (+128839us) raft_consensus.cc:1685] finished
0709 14:51:45.744924 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:45.745125 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7860,"raft.queue_time_us":33919,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":39409,"thread_start_us":420384,"threads_started":1,"wal-append.queue_time_us":638528}
I20260709 14:51:45.712641  4382 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:45.679104  4450 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.708382  4451 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:45.753711  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:45.765499  2726 leader_election.cc:400] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:45.755116  4449 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:45.753748  4382 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.769600  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:45.774439  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.774176  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:45.775786  2753 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:45.746670  4208 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:45.782572  4284 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:45.779297  4384 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.809s	user 0.017s	sys 0.003s
I20260709 14:51:45.759892  4415 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.760994  4031 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:45.799151  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:45.761356  4397 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:45.781534  4452 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:45.781030  4150 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 14:51:45.815732  4426 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.635s	user 0.010s	sys 0.005s
I20260709 14:51:45.805032  4452 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:45.799389  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.820758  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 219) took 1291 ms. Trace:
I20260709 14:51:45.821499  4275 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:45.823647  2681 rpcz_store.cc:276] 0709 14:51:44.529622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.535155 (+  5533us) service_pool.cc:224] Handling call
0709 14:51:44.535622 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.536016 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.536046 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:44.536659 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 14:51:44.791884 (+255225us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:44.791948 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:44.792049 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:45.784444 (+992395us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:45.817222 (+ 32778us) raft_consensus.cc:1685] finished
0709 14:51:45.817244 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:45.820680 (+  3436us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7174,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":141,"spinlock_wait_cycles":23757184,"thread_start_us":254744,"threads_started":1,"wal-append.queue_time_us":471131}
I20260709 14:51:45.805745  4031 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.811249  4284 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:45.823395  4452 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:45.822558  4150 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:45.800712  4384 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:45.808424  4397 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:45.860752  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:45.819456  4266 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.867195  3536 raft_consensus.cc:2368] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:45.876883  2610 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:45.878808  2650 leader_election.cc:400] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:45.879323  2650 leader_election.cc:403] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:45.879712  4150 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.881862  4422 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:45.887135  4379 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:45.880234  4418 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:45.912290  4397 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.912642  4395 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:45.921454  2607 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:45.905961  4425 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.923451  4333 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.923825  3542 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:45.924341  4333 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.929292  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:45.933148  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:45.934078  2603 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:45.936443  4422 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:45.936705  4384 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:45.918012  4031 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:45.947302  4425 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:45.938472  4266 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:45.966006  4333 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:45.972034  4384 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:45.972775  4384 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:45.973688  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:45.975154  4384 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:45.983513  4364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.012s	sys 0.032s
W20260709 14:51:45.985126  4364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.012s	sys 0.032s
I20260709 14:51:45.992810  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 832) took 1430 ms. Trace:
I20260709 14:51:45.998082  4452 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:45.967206  4469 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 14:51:46.005126  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:46.005873  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.013391  2610 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:45.993425  2607 rpcz_store.cc:276] 0709 14:51:44.561921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.563026 (+  1105us) service_pool.cc:224] Handling call
0709 14:51:45.992562 (+1429536us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:45.992709 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2259163520}
W20260709 14:51:46.008174  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:46.019544  4418 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:46.029068  2726 leader_election.cc:400] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:46.030202  2726 leader_election.cc:403] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 14:51:46.036791  2752 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:46.025802  4336 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 14:51:46.043228  4239 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:46.046471  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:46.047922  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:46.049330  4442 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.591s	user 0.000s	sys 0.004s
I20260709 14:51:46.050144  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:46.052182  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.053088  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.054000  4452 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:46.058383  4336 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:46.063073  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:46.064652  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:46.036625  4349 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.075311  2755 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:46.081278  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:46.082008  3542 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.076706  3536 pending_rounds.cc:85] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 3
I20260709 14:51:46.083186  3536 pending_rounds.cc:107] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 14:51:46.078564  3547 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.083587  3536 raft_consensus.cc:2889] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:46.046471  4379 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:46.089294  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:46.092065  4210 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:46.100016  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 846) took 1029 ms. Trace:
I20260709 14:51:46.100701  2609 rpcz_store.cc:276] 0709 14:51:45.070347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.070503 (+   156us) service_pool.cc:224] Handling call
0709 14:51:45.070820 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:45.406912 (+336092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.099587 (+692675us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54002fe4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:46.099922 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353893248}
I20260709 14:51:46.105882  3547 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:46.107786  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:46.108109  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:46.078075  4336 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:46.112011  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:46.120260  4385 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:46.114189  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 840) took 1356 ms. Trace:
I20260709 14:51:46.116019  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 790) took 1098 ms. Trace:
I20260709 14:51:46.123066  2685 rpcz_store.cc:276] 0709 14:51:45.017571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.017688 (+   117us) service_pool.cc:224] Handling call
0709 14:51:45.018064 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:45.407691 (+389627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.103988 (+696297us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:46.115890 (+ 11902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270940544}
I20260709 14:51:46.084899  3541 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.122574  2611 rpcz_store.cc:276] 0709 14:51:44.758174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.758309 (+   135us) service_pool.cc:224] Handling call
0709 14:51:44.786604 (+ 28295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:44.789893 (+  3289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:44.789941 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:44.811494 (+ 21553us) log.cc:844] Serialized 26 byte log entry
0709 14:51:45.067870 (+256376us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:45.067926 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:45.068012 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:46.048701 (+980689us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:46.095629 (+ 46928us) raft_consensus.cc:1685] finished
0709 14:51:46.095654 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:46.114071 (+ 18417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3460,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":13385,"spinlock_wait_cycles":77543808,"thread_start_us":120033,"threads_started":1,"wal-append.queue_time_us":520971}
I20260709 14:51:46.138317  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:46.141296  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.092518  4210 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:46.142547  2611 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.142710  3542 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:46.143893  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:46.144354  2755 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:46.096032  4397 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.147847  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:46.148794  2609 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:46.114521  4121 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:46.134598  4452 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:46.154321  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.136234  2607 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:46.117786  4332 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.157864  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.158506  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:46.159485  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.159848  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.155234  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.146452  4210 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:46.230417  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:46.231099  2682 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:46.160596  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:46.162166  2611 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:46.138078  4333 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:46.140441  4458 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:46.236720  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 92903.6us
I20260709 14:51:46.146178  4382 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:46.236037  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 771) took 1169 ms. Trace:
W20260709 14:51:46.241449  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 80966.1us
I20260709 14:51:46.244133  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:46.165323  4366 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.241966  2756 rpcz_store.cc:276] 0709 14:51:45.000602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.000960 (+   358us) service_pool.cc:224] Handling call
0709 14:51:45.010793 (+  9833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:45.427799 (+417006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.170013 (+742214us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:46.170257 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956567424}
I20260709 14:51:46.249042  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:46.250487  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.251199  2610 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.148579  4384 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 14:51:46.258639  3541 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:46.161226  4031 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 14:51:46.153119  4397 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:46.244134  4453 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:46.242864  4150 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:46.276177  4476 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 14:51:46.173880  4332 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.229455  4121 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.254254  4333 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:46.246085  4366 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.274154  4422 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:46.325367  4293 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.371196  4293 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.372236  4333 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.372902  4333 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.373502  4293 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.373864  4366 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.304523  4453 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:46.376035  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.376900  2755 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:46.378643  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:46.268240  4458 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.251400  4336 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.402554  2685 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:46.406931  2681 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:46.383045  4293 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:46.384193  4432 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.427861  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.430599  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.431597  3547 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:46.409860  4349 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:46.435537  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.464093  3541 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:46.417052  4293 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:46.435513  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.467694  3537 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 14:51:46.468768  4481 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 14:51:46.475739  2610 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:46.483222  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 547) took 1615 ms. Trace:
W20260709 14:51:46.439478  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:46.426077  4401 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:46.488770  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 369127us
I20260709 14:51:46.426453  4333 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:46.474519  4293 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:46.426113  4432 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.428741  4094 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.462210  4384 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 14:51:46.500357  4387 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.436077  4453 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:46.526019  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.526264  4349 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.524051  4397 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:46.506501  4401 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.514671  4239 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.534631  4419 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.526512  4432 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.490374  3537 rpcz_store.cc:276] 0709 14:51:44.867883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.886082 (+ 18199us) service_pool.cc:224] Handling call
0709 14:51:46.417800 (+1531718us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:46.433106 (+ 15306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:46.473292 (+ 40186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.483144 (+  9852us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22796160}
I20260709 14:51:46.550978  4239 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.558429  3542 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:46.528805  4387 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:46.547036  3543 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.561887  3543 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:46.544854  4419 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.537683  4332 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:46.535166  4349 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:46.585404  4382 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:46.586963  4382 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.592341  4486 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:46.593303  4486 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:46.596421  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:46.597918  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.598920  2685 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:46.600630  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 672) took 1177 ms. Trace:
I20260709 14:51:46.600776  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 552) took 1590 ms. Trace:
I20260709 14:51:46.601194  3546 rpcz_store.cc:276] 0709 14:51:45.422893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.427727 (+  4834us) service_pool.cc:224] Handling call
0709 14:51:45.491681 (+ 63954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:45.699248 (+207567us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:45.725207 (+ 25959us) log.cc:844] Serialized 14 byte log entry
0709 14:51:45.847536 (+122329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:45.849468 (+  1932us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:45.855424 (+  5956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.600346 (+744922us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400034654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:46.600405 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:46.600558 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8581,"prepare.queue_time_us":20168,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":28684,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4396,"spinlock_wait_cycles":448084864,"thread_start_us":79219,"threads_started":1,"wal-append.queue_time_us":137240}
I20260709 14:51:46.604806  4349 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:46.606693  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.607432  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:46.461730  4379 consensus_queue.cc:1175] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:51:46.609630  4458 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.603371  3545 rpcz_store.cc:276] 0709 14:51:45.010647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.036215 (+ 25568us) service_pool.cc:224] Handling call
0709 14:51:46.600551 (+1564336us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54004b0fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:46.600685 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591813376}
I20260709 14:51:46.607674  2679 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:51:46.612214  3538 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 14:51:46.622634  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 235444us
I20260709 14:51:46.603678  4486 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:46.642285  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:46.629527  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.645753  2679 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.645977  2760 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:46.625613  2685 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:46.631456  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.647869  2751 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:46.625203  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:46.575364  4387 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:46.644786  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:46.667402  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.641805  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.683760  2686 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 14:51:46.686223  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.641556  4422 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:46.689208  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.694950  2679 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:46.697149  2751 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:46.697683  2685 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:46.678270  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 174736us
I20260709 14:51:46.649379  4401 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:46.712550  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:46.713516  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:46.714582  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:46.714457  2751 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:46.642237  4332 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 14:51:46.644291  4418 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 14:51:46.724800  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.662011  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.623827  4210 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:46.753087  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.677263  4414 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:46.709196  4424 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:46.745169  4349 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:46.742892  4379 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:46.770081  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.806787  4210 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:46.796731  4349 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:46.809916  3543 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 14:51:46.809549  4210 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:46.811568  4419 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:46.813551  3543 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
I20260709 14:51:46.814734  3543 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 14:51:46.816075  3543 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:46.812578  4349 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:46.820474  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 472) took 2198 ms. Trace:
I20260709 14:51:46.821214  3543 rpcz_store.cc:276] 0709 14:51:44.619064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.619176 (+   112us) service_pool.cc:224] Handling call
0709 14:51:44.619460 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:44.824852 (+205392us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002894a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:51:46.543793 (+1718941us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54002894a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c406f 00007fa16e63cf60 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62
0709 14:51:46.816757 (+272964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.817148 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992804864}
I20260709 14:51:46.803006  4314 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.787773  4424 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.838634  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:46.850317  2683 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:46.820194  4320 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 14:51:46.852329  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:46.788053  4414 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:46.808674  4293 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:46.892266  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:46.943499  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.944983  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:46.945907  2609 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:46.947206  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.948244  2611 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:46.949499  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:46.873752  4333 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.805438  4422 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:46.966447  4401 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.975852  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:46.979636  2610 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:46.856727  4314 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:46.984987  4293 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:46.986137  4450 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:46.987280  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:46.994581  4414 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:46.996549  4451 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:46.904008  4501 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:46.999155  4501 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.001042  2607 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:47.004746  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.023s	sys 0.065s
W20260709 14:51:47.005283  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.024s	sys 0.065s
W20260709 14:51:47.011425  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 761316us
I20260709 14:51:47.020186  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:47.038559  2607 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:47.040988  2683 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:47.041608  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:47.045291  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:47.048187  2755 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:47.067684  2611 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 14:51:47.011848  4452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.027s	sys 0.062s
W20260709 14:51:47.029487  2678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.003s	sys 0.051s
I20260709 14:51:47.015352  4496 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:46.983044  4333 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.101944  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:47.083009  4453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.021s	sys 0.039s
I20260709 14:51:47.103974  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:47.083268  4452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.028s	sys 0.063s
I20260709 14:51:47.003528  4401 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.114913  4489 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:47.103765  4454 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 14:51:47.088239  2678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.003s	sys 0.051s
W20260709 14:51:47.091898  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 71883.8us
I20260709 14:51:47.136189  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.136467  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:47.124989  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:47.142596  2755 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:47.071038  4351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.022s	sys 0.036s
I20260709 14:51:47.090199  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:47.092072  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63003.9us
I20260709 14:51:47.144137  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:47.118922  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:47.137989  2610 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
W20260709 14:51:47.138310  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 93121.9us
I20260709 14:51:47.099489  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:47.110677  4453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.021s	sys 0.039s
I20260709 14:51:47.148061  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:47.108256  4415 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:47.155476  2611 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:47.168573  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.120384  4239 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:47.131314  4452 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:51:47.171765  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.023s	sys 0.042s
I20260709 14:51:47.138271  4458 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:47.175429  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65300.7us
W20260709 14:51:47.155865  4351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.023s	sys 0.036s
W20260709 14:51:47.178370  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.023s	sys 0.042s
W20260709 14:51:47.185091  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 540334us
I20260709 14:51:47.179147  2755 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:47.148435  4418 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:47.147334  4489 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:47.122785  4349 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:47.169512  4454 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:47.180375  4387 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:47.201033  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:47.229107  3543 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:47.109305  4496 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.180655  4492 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:47.184111  4031 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:47.245227  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.188411  4418 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:47.245257  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:47.253878  4418 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:47.256565  4031 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:47.177650  4415 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.190528  4424 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:47.198962  4332 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:47.269656  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.270045  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:47.270546  4314 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:47.244524  4501 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:47.271121  4268 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:47.272169  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.190171  4510 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:47.277411  3543 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:47.303344  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.274716  4385 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:47.273422  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:47.186141  4458 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:47.230651  4484 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:47.270763  4509 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:47.297295  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:47.265631  4387 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:47.249064  4293 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:47.306172  2610 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 14:51:47.287732  4495 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.014s	sys 0.065s
W20260709 14:51:47.315196  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.006s	sys 0.023s
W20260709 14:51:47.344789  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.006s	sys 0.024s
W20260709 14:51:47.345301  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 645717us
W20260709 14:51:47.315834  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.011s	sys 0.020s
I20260709 14:51:47.322265  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:47.323664  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:47.351006  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.012s	sys 0.020s
I20260709 14:51:47.294723  2605 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:47.303239  4510 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:47.298323  4314 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:47.344707  4495 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.014s	sys 0.066s
I20260709 14:51:47.293854  4332 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:47.315608  4382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.009s
I20260709 14:51:47.343773  4506 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:47.389998  4506 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:47.274945  4492 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:47.390511  4349 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:47.326460  4501 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 14:51:47.304883  4453 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 14:51:47.305668  4268 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.393793  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:47.289927  4452 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.374346  4314 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:47.391410  4202 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:47.321429  4385 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.367097  4494 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:47.372217  4510 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:47.392448  4382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.021s	sys 0.010s
W20260709 14:51:47.400483  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:47.390543  4506 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:47.353600  4293 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.326866  4484 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.419611  2755 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 14:51:47.444336  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 407128us
W20260709 14:51:47.539456  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:47.444478  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 98203.9us
I20260709 14:51:47.541523  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:47.309139  4489 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.549186  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:47.401947  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.001s	sys 0.022s
W20260709 14:51:47.551134  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.003s	sys 0.023s
I20260709 14:51:47.426530  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:47.546479  4366 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:47.546792  2753 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:47.315609  4454 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:47.552980  2757 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:47.341807  4509 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.559485  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:47.609762  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50416.1us
I20260709 14:51:47.562171  4094 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:47.562817  4501 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:47.563653  2753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "194ae28527c14d78817238308fb284c6"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:47.571594  4450 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:47.577366  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 266) took 1650 ms (client timeout 1554 ms). Trace:
W20260709 14:51:47.581514  4487 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.988s	user 0.010s	sys 0.000s
I20260709 14:51:47.593935  2430 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 159) took 9061 ms. Trace:
I20260709 14:51:47.412421  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:47.414043  4202 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:47.444535  4384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.013s	sys 0.007s
I20260709 14:51:47.617630  2753 raft_consensus.cc:606] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:51:47.621223  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:47.619355  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:47.533895  4031 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:47.534799  4401 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:47.420758  4364 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:47.416648  4517 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:47.642266  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.660562  2606 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:47.642609  4094 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:47.643532  4374 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:51:47.662750  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43484us
I20260709 14:51:47.454324  4121 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:47.427816  4507 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:47.637310  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:47.682766  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45603us
W20260709 14:51:47.665355  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44216.5us
I20260709 14:51:47.624969  2430 rpcz_store.cc:276] 0709 14:51:38.577872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:38.615339 (+ 37467us) service_pool.cc:224] Handling call
0709 14:51:47.593827 (+8978488us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:45.216417 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:45.415386 (+198969us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:45.420640 (+  5254us) write_op.cc:435] Acquired schema lock
0709 14:51:45.425390 (+  4750us) tablet.cc:662] Decoding operations
0709 14:51:45.650523 (+225133us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:45.663107 (+ 12584us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:45.667117 (+  4010us) tablet.cc:685] Acquiring locks for 8 operations
0709 14:51:45.708468 (+ 41351us) tablet.cc:701] Row locks acquired
0709 14:51:45.708489 (+    21us) write_op.cc:260] PREPARE: finished
0709 14:51:45.708704 (+   215us) write_op.cc:270] Start()
0709 14:51:45.708879 (+   175us) write_op.cc:276] Timestamp: P: 1783608705708637 usec, L: 0
0709 14:51:45.708902 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 14:51:45.753908 (+ 45006us) log.cc:844] Serialized 3257 byte log entry
0709 14:51:46.041218 (+287310us) op_driver.cc:464] REPLICATION: finished
0709 14:51:46.245760 (+204542us) write_op.cc:301] APPLY: starting
0709 14:51:46.245858 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:46.246089 (+   231us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:46.288813 (+ 42724us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:46.482589 (+193776us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:46.482955 (+   366us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 14:51:46.483030 (+    75us) write_op.cc:312] APPLY: finished
0709 14:51:46.483391 (+   361us) log.cc:844] Serialized 64 byte log entry
0709 14:51:46.483602 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:46.483762 (+   160us) write_op.cc:454] Released schema lock
0709 14:51:46.483840 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:46.484400 (+   560us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:46.484443 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":516152,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":187913,"apply.run_cpu_time_us":5184,"apply.run_wall_time_us":238592,"num_ops":8,"prepare.queue_time_us":410915,"prepare.run_cpu_time_us":34340,"prepare.run_wall_time_us":631401,"raft.queue_time_us":47949,"raft.run_cpu_time_us":18093,"raft.run_wall_time_us":64023,"replication_time_us":332128,"thread_start_us":366035,"threads_started":4,"wal-append.queue_time_us":142553,"wal-append.run_cpu_time_us":22659,"wal-append.run_wall_time_us":1637806}]]}
W20260709 14:51:47.687165  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 389960us
I20260709 14:51:47.689637  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:47.660399  3537 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:47.647928  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:47.690701  2753 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:47.617501  4486 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 14:51:47.694778  2757 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 14:51:47.701718  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 378172us
W20260709 14:51:47.671639  4503 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.333s	user 0.002s	sys 0.000s
I20260709 14:51:47.647997  4202 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:47.647380  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:47.686618  4384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.016s	sys 0.009s
I20260709 14:51:47.428563  4387 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.619343  4418 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:47.694327  4121 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:47.664165  4517 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:47.676105  4374 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:47.694657  4507 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:47.709002  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:47.732479  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 855) took 1106 ms. Trace:
I20260709 14:51:47.666754  4415 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:47.676234  4293 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:47.743362  4485 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.152s	user 0.005s	sys 0.005s
I20260709 14:51:47.741165  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:47.769950  2683 raft_consensus.cc:2368] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5.
I20260709 14:51:47.676522  4492 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:47.629815  2752 rpcz_store.cc:269] 0709 14:51:45.927307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:45.928951 (+  1644us) service_pool.cc:224] Handling call
0709 14:51:47.577044 (+1648093us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:47.577260 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229013376}
I20260709 14:51:47.707671  4496 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:47.671608  4275 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:47.640044  4450 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:47.671890  4364 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:47.672533  4094 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.759972  2756 rpcz_store.cc:276] 0709 14:51:46.626278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.626474 (+   196us) service_pool.cc:224] Handling call
0709 14:51:46.626787 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:46.627203 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:46.627230 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:46.646364 (+ 19134us) log.cc:844] Serialized 26 byte log entry
0709 14:51:47.551842 (+905478us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:47.551903 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:47.552009 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:47.732087 (+180078us) raft_consensus.cc:1685] finished
0709 14:51:47.732123 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:47.732387 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5676,"raft.queue_time_us":18294,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":6954,"thread_start_us":855198,"threads_started":1,"wal-append.queue_time_us":1067698}
I20260709 14:51:47.768447  2753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:47.789534  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 863) took 1639 ms. Trace:
I20260709 14:51:47.793506  2753 raft_consensus.cc:606] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:47.793552  2608 rpcz_store.cc:276] 0709 14:51:46.149978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.150360 (+   382us) service_pool.cc:224] Handling call
0709 14:51:46.168939 (+ 18579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:46.226900 (+ 57961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:46.226925 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:46.233037 (+  6112us) log.cc:844] Serialized 26 byte log entry
0709 14:51:46.566550 (+333513us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:46.566595 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.566681 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:47.580380 (+1013699us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:47.776763 (+196383us) raft_consensus.cc:1685] finished
0709 14:51:47.776804 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:47.789425 (+ 12621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46669696,"thread_start_us":282109,"threads_started":1,"wal-append.queue_time_us":343178}
I20260709 14:51:47.759119  4458 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:47.755406  4517 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:47.421528  4494 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:47.694813  4520 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 14:51:47.778292  4031 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:47.825240  4314 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:47.759074  4121 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:47.791147  4293 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:47.817380  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:47.840101  4508 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 14:51:47.841701  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.047s	sys 0.051s
W20260709 14:51:47.842149  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.047s	sys 0.051s
I20260709 14:51:47.798261  4275 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:47.805449  4384 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:47.851760  4514 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:47.851970  4450 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:47.861694  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 573) took 1785 ms. Trace:
I20260709 14:51:47.826409  4031 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:47.826614  4332 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:47.849831  4293 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:48.154592  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:48.091393  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.090375  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:48.180008  2685 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:48.042718  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:48.041896  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.207466  4535 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.207845  4533 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:48.089865  4523 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.363s	user 0.000s	sys 0.002s
I20260709 14:51:48.227056  4528 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:48.144917  4293 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:48.040128  4341 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:47.892304  4423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.015s	sys 0.077s
I20260709 14:51:48.223173  2679 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:51:48.043846  4501 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.090773  4509 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:48.008185  2753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:48.261861  4494 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:48.125262  4418 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.182344  4420 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.208441  4496 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.035534  4275 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:48.290498  4531 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 14:51:48.024544  4490 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.232193  4031 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:48.262768  4121 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:48.297616  2685 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:47.872105  4384 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:47.870529  4449 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:48.318872  2760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.007s	sys 0.034s
I20260709 14:51:48.006273  4507 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.044637  4484 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:48.327816  4202 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.002s
W20260709 14:51:48.327775  2760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.008s	sys 0.034s
I20260709 14:51:48.232741  3536 rpcz_store.cc:276] 0709 14:51:46.075951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.076106 (+   155us) service_pool.cc:224] Handling call
0709 14:51:46.076483 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:46.086885 (+ 10402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:46.086937 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:46.087532 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 14:51:46.492398 (+404866us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:46.492455 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:46.492556 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:47.742383 (+1249827us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:47.848018 (+105635us) raft_consensus.cc:1685] finished
0709 14:51:47.848054 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:47.861594 (+ 13540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11615104,"thread_start_us":404436,"threads_started":1,"wal-append.queue_time_us":503051}
W20260709 14:51:48.328377  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.017s	sys 0.043s
I20260709 14:51:48.335740  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 782) took 1303 ms. Trace:
I20260709 14:51:48.336230  2751 rpcz_store.cc:276] 0709 14:51:47.032365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.032610 (+   245us) service_pool.cc:224] Handling call
0709 14:51:47.033153 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:47.091696 (+ 58543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.091729 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:47.092381 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 14:51:47.369383 (+277002us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:47.369450 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:47.369553 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:47.571208 (+201655us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:51:48.330334 (+759126us) raft_consensus.cc:1685] finished
0709 14:51:48.330373 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:48.335660 (+  5287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57967,"raft.queue_time_us":61035,"raft.run_cpu_time_us":2958,"raft.run_wall_time_us":473537,"spinlock_wait_cycles":10195840,"thread_start_us":276430,"threads_started":1,"wal-append.queue_time_us":586102}
W20260709 14:51:48.335806  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.018s	sys 0.043s
I20260709 14:51:48.286525  4501 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.301795  4496 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.335582  4202 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.264606  4535 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.331929  4449 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:48.365285  2604 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:48.284715  4341 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.292563  4420 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.395488  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:48.411350  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:48.273730  4533 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.418556  2682 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:48.421002  4437 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.421875  4437 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:48.424760  4437 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.425509  4437 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.291831  4528 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.429092  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.426105  4437 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:48.434962  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.011s	sys 0.073s
W20260709 14:51:48.441747  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.011s	sys 0.073s
W20260709 14:51:48.442139  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.19184e+06us
I20260709 14:51:48.443027  4341 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:48.434343  3411 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.013s	sys 0.037s
I20260709 14:51:48.444947  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:48.444355  3411 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.013s	sys 0.037s
W20260709 14:51:48.449249  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 966503us
I20260709 14:51:48.456282  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.459547  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.469439  2685 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 14:51:48.449613  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.455912  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.322175  4490 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:48.468477  4386 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.020s	sys 0.025s
W20260709 14:51:48.459749  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.014s	sys 0.031s
I20260709 14:51:48.465531  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:48.489063  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.015s	sys 0.031s
I20260709 14:51:48.469920  2685 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 1
I20260709 14:51:48.463881  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:48.491636  2685 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:51:48.492062  2685 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:51:48.484294  4538 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 14:51:48.455783  2605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.001s	sys 0.017s
I20260709 14:51:48.462008  4423 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:48.486560  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:48.508946  2608 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:48.496064  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.496590  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.470803  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:48.488752  4386 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.020s	sys 0.026s
I20260709 14:51:48.489899  4202 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.498028  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 783) took 1461 ms. Trace:
I20260709 14:51:48.504840  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:48.485965  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:48.507797  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:48.512570  4535 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:48.467267  4506 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.958s	user 0.014s	sys 0.001s
I20260709 14:51:48.485477  4533 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:48.464715  4489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.019s	sys 0.025s
I20260709 14:51:48.524119  2760 rpcz_store.cc:276] 0709 14:51:47.036787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.036962 (+   175us) service_pool.cc:224] Handling call
0709 14:51:47.037376 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:47.091730 (+ 54354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.091760 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:47.564932 (+473172us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b180030ec80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a
0709 14:51:47.566647 (+  1715us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b2c000cb098. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:51:47.598504 (+ 31857us) log.cc:844] Serialized 26 byte log entry
0709 14:51:48.332289 (+733785us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:48.332340 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:48.332419 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:48.497792 (+165373us) raft_consensus.cc:1685] finished
0709 14:51:48.497819 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:48.497962 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53869,"raft.queue_time_us":59939,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":7757,"spinlock_wait_cycles":13376256,"thread_start_us":733297,"threads_started":1,"wal-append.queue_time_us":897251}
I20260709 14:51:48.475730  4384 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:48.578184  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:48.505672  2605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.002s	sys 0.017s
I20260709 14:51:48.480482  4496 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:48.581339  4208 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:48.475041  4500 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:48.579348  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 93486.3us
W20260709 14:51:48.584440  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.012s	sys 0.030s
I20260709 14:51:48.574960  2752 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 5
I20260709 14:51:48.574618  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:48.592132  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.012s	sys 0.030s
I20260709 14:51:48.575838  4210 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 14:51:48.582154  4489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.020s	sys 0.025s
I20260709 14:51:48.579200  4501 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:48.577736  4449 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:48.600178  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:48.576265  4444 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 14:51:48.524580  4423 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:48.613094  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.029s	sys 0.037s
I20260709 14:51:48.580060  4293 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 14:51:48.588593  4496 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.612780  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.607565  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:48.590894  4420 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.586697  4532 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:48.614120  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.030s	sys 0.037s
W20260709 14:51:48.614472  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.007s	sys 0.035s
I20260709 14:51:48.581809  4506 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:48.584460  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.013s	sys 0.010s
W20260709 14:51:48.634657  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.013s	sys 0.010s
W20260709 14:51:48.582265  4452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.021s	sys 0.020s
W20260709 14:51:48.627177  3542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.029s	sys 0.048s
W20260709 14:51:48.646515  3542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.030s	sys 0.048s
I20260709 14:51:48.647121  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 492) took 1327 ms. Trace:
I20260709 14:51:48.647356  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 488) took 1460 ms. Trace:
I20260709 14:51:48.647784  3542 rpcz_store.cc:276] 0709 14:51:47.186578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.186769 (+   191us) service_pool.cc:224] Handling call
0709 14:51:47.187142 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.187375 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:47.419081 (+231706us) log.cc:844] Serialized 14 byte log entry
0709 14:51:48.646978 (+1227897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:48.647035 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:48.647078 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:48.647179 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:48.647297 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":447305,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":51,"thread_start_us":1447175,"threads_started":2}
W20260709 14:51:48.627269  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.008s	sys 0.035s
W20260709 14:51:48.619024  4510 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.177s	user 0.009s	sys 0.006s
I20260709 14:51:48.612730  4418 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:48.613032  2608 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:48.661006  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.660741  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:48.662249  2608 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:48.647675  3545 rpcz_store.cc:276] 0709 14:51:47.319799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.319975 (+   176us) service_pool.cc:224] Handling call
0709 14:51:47.353092 (+ 33117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.353383 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:47.735564 (+382181us) log.cc:844] Serialized 14 byte log entry
0709 14:51:48.646628 (+911064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:48.646706 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:48.646763 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:48.646881 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:48.647052 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":281593,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":1260285,"threads_started":2}
W20260709 14:51:48.626284  4422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.014s	sys 0.046s
W20260709 14:51:48.624119  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.007s	sys 0.018s
W20260709 14:51:48.626081  4094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.011s	sys 0.015s
I20260709 14:51:48.668623  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:48.652523  4452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.025s	sys 0.020s
I20260709 14:51:48.618703  4489 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.627002  4332 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:48.667660  4529 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:48.668666  4422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.014s	sys 0.046s
I20260709 14:51:48.595831  4500 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.671416  2608 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:48.669302  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:48.684311  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:48.626518  4497 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:51:48.632390  4532 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:48.670790  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.007s	sys 0.018s
W20260709 14:51:48.671095  4094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.011s	sys 0.016s
W20260709 14:51:48.702631  2723 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.705s (SENT) [suppressed 7 similar messages]
I20260709 14:51:48.686743  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:48.670200  4437 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.671875  4528 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:48.671555  4490 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.626468  4496 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.703547  2753 raft_consensus.cc:606] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:48.726341  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57148.6us
W20260709 14:51:48.731645  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43703.7us
I20260709 14:51:48.727837  2685 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:48.670598  4501 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 14:51:48.788139  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:48.729058  4537 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 14:51:48.785360  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:48.728971  4541 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:48.664739  4510 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.770648  4432 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.729674  4332 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:48.798620  3547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 697) took 1412 ms (client timeout 1545 ms). Trace:
I20260709 14:51:48.689939  4384 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.724512  4529 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.804454  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:48.688980  4484 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:48.706496  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:48.770396  4437 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.688412  4489 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.829682  2681 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
W20260709 14:51:48.808634  3547 rpcz_store.cc:269] 0709 14:51:47.386727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.400199 (+ 13472us) service_pool.cc:224] Handling call
0709 14:51:48.787909 (+1387710us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:48.798539 (+ 10630us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5333888}
I20260709 14:51:48.806746  2612 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 14:51:48.776417  4202 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:48.826965  3538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.004s	sys 0.036s
I20260709 14:51:48.812615  4332 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.797207  4397 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.863441  2679 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:48.822844  4444 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:48.866062  4454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.032s
I20260709 14:51:48.874812  4332 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:48.922201  4454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.012s	sys 0.033s
I20260709 14:51:48.956549  4500 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:48.979110  4529 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.980077  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 856) took 1633 ms. Trace:
W20260709 14:51:48.980906  4517 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.152s	user 0.007s	sys 0.007s
I20260709 14:51:48.981410  4532 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.981348  4517 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:48.982460  4510 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:48.980489  2609 rpcz_store.cc:276] 0709 14:51:47.346522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.346671 (+   149us) service_pool.cc:224] Handling call
0709 14:51:47.347115 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:47.705877 (+358762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.705930 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:47.706623 (+   693us) log.cc:844] Serialized 26 byte log entry
0709 14:51:48.625654 (+919031us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:48.625700 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:48.627410 (+  1710us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:48.979827 (+352417us) raft_consensus.cc:1685] finished
0709 14:51:48.979849 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:48.980011 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":214325,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":111,"thread_start_us":1276818,"threads_started":2,"wal-append.queue_time_us":1240328}
W20260709 14:51:48.986217  4031 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.004s
I20260709 14:51:48.839663  4437 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:48.836340  4541 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:48.990185  4541 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.999671  4489 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.000420  4489 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:49.000908  4489 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:49.004689  4516 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.587s	user 0.008s	sys 0.005s
I20260709 14:51:49.004923  4489 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.005481  4489 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.005942  4489 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.006680  4333 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:48.860638  2677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.032s	sys 0.044s
W20260709 14:51:48.865815  3538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.004s	sys 0.040s
I20260709 14:51:48.868996  4245 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 14:51:48.815572  4432 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:48.827041  4484 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:48.924398  4387 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.930059  4397 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:48.950564  4568 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:48.951524  4423 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:48.943159  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.071504  2685 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
W20260709 14:51:49.011068  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.013s
W20260709 14:51:49.072779  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.007s	sys 0.013s
I20260709 14:51:49.030337  2759 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:49.013504  2677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.033s	sys 0.044s
I20260709 14:51:48.821297  4384 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:49.074447  3540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.015s	sys 0.026s
W20260709 14:51:49.082608  3540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.015s	sys 0.026s
I20260709 14:51:49.002691  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:48.986708  4031 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.028802  4532 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.090090  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.095021  4384 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.095745  4532 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.096315  4385 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.098138  4532 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.098296  4496 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.076572  4568 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.100333  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 734) took 1061 ms. Trace:
I20260709 14:51:49.100802  4568 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.100749  2681 rpcz_store.cc:276] 0709 14:51:48.039192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.042169 (+  2977us) service_pool.cc:224] Handling call
0709 14:51:49.100078 (+1057909us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:49.100238 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495694720}
I20260709 14:51:49.102301  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.106400  4332 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.107992  4031 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.108350  2685 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:49.082386  4507 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.109033  4507 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.109437  4332 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.109997  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.115139  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.105508  4489 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.117118  2681 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:49.119805  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 740) took 1078 ms. Trace:
I20260709 14:51:49.124629  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 270) took 1700 ms. Trace:
I20260709 14:51:49.125310  2754 rpcz_store.cc:276] 0709 14:51:47.424692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.424881 (+   189us) service_pool.cc:224] Handling call
0709 14:51:47.425331 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.822187 (+396856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.822236 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:47.822293 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:47.822337 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.124216 (+1301879us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54002765d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:49.124320 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.124536 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912092544}
I20260709 14:51:49.128322  2575 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:49.038908  4387 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:49.136440  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.000s	sys 0.024s
W20260709 14:51:49.137079  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.000s	sys 0.025s
I20260709 14:51:49.143204  2685 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:49.120333  2684 rpcz_store.cc:276] 0709 14:51:48.041132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.225384 (+184252us) service_pool.cc:224] Handling call
0709 14:51:48.225977 (+   593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:48.226282 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:48.735331 (+509049us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.119198 (+383867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.119263 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:49.119314 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.119550 (+   236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.119722 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":729967,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":478,"thread_start_us":816075,"threads_started":2}
I20260709 14:51:49.119129  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 857) took 1734 ms. Trace:
I20260709 14:51:49.151262  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.156524  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:49.166076  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 463454us
I20260709 14:51:49.148330  4532 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.158875  4517 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.159528  4568 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.184985  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.289026  3547 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:49.187211  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.290351  3542 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:49.187538  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:49.195935  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 584) took 1389 ms. Trace:
W20260709 14:51:49.197224  4550 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 14:51:49.198004  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:49.201328  4384 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:49.202252  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.025s	sys 0.051s
I20260709 14:51:49.203779  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 275) took 1597 ms. Trace:
I20260709 14:51:49.308084  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:49.308794  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.025s	sys 0.052s
I20260709 14:51:49.309087  2685 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 14:51:49.207991  4560 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 14:51:49.308872  2755 rpcz_store.cc:276] 0709 14:51:47.606462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.606655 (+   193us) service_pool.cc:224] Handling call
0709 14:51:47.607046 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.821091 (+214045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.821137 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:47.821199 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:47.821246 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.203428 (+1382182us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:49.203500 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.203701 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491694976}
W20260709 14:51:49.209031  4564 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 14:51:49.215415  4397 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.217332  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.218075  4506 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.221387  4364 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.224807  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 727) took 3066 ms. Trace:
I20260709 14:51:49.226007  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 490) took 2036 ms. Trace:
I20260709 14:51:49.227499  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:49.228956  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.229861  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:49.233788  4526 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.555s	user 0.010s	sys 0.012s
I20260709 14:51:49.240727  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.241394  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.241978  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.360728  3541 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:51:49.361578  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 702) took 1589 ms (client timeout 1942 ms). Trace:
I20260709 14:51:49.261575  4268 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.263540  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 863) took 1620 ms. Trace:
I20260709 14:51:49.363214  2604 rpcz_store.cc:276] 0709 14:51:47.642626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.647669 (+  5043us) service_pool.cc:224] Handling call
0709 14:51:49.263291 (+1615622us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:49.263447 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539843968}
I20260709 14:51:49.268625  4387 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.275265  4437 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:49.276717  4484 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.284044  4432 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.163570  4341 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.365574  4341 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:49.285666  4569 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 14:51:49.159881  4489 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.162505  4496 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.294991  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.250550  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.349093  4332 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.312080  4568 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.165266  4385 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.312508  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 12) took 1281 ms. Trace:
I20260709 14:51:49.309201  3547 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:49.390255  2603 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:49.319223  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 437349us
W20260709 14:51:49.328599  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141183us
I20260709 14:51:49.425673  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:49.329842  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.334429  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.337575  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.338917  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.288348  4336 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:49.335165  4555 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 14:51:49.366490  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.372430  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:49.362331  3541 rpcz_store.cc:269] 0709 14:51:47.772588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.773519 (+   931us) service_pool.cc:224] Handling call
0709 14:51:49.241779 (+1468260us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:49.361506 (+119727us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21079424}
I20260709 14:51:49.186456  4532 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:49.440070  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67865.4us
W20260709 14:51:49.440438  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101690us
I20260709 14:51:49.440960  3412 leader_election.cc:400] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:49.441344  3412 leader_election.cc:403] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 14:51:49.441802  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.392916  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.347333  4284 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.399995  2679 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:49.297322  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.309585  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.446622  3542 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:49.404515  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.394495  4369 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.329367  4384 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.172910  4558 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.429419  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:49.413885  2753 rpcz_store.cc:276] 0709 14:51:48.007619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.007988 (+   369us) service_pool.cc:224] Handling call
0709 14:51:49.286816 (+1278828us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:49.288673 (+  1857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983329152}
I20260709 14:51:49.436477  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.442173  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.444717  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:49.448343  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 229776us
I20260709 14:51:49.437630  4333 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.458504  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.480126  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.459405  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.480829  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:49.464051  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.481732  3542 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:49.469246  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.485288  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.485999  3541 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:49.486352  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.438481  4316 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.457984  4284 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:49.409866  4496 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.412583  4332 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.410413  4489 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.488201  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.501904  2754 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:49.458365  4364 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:49.505378  4387 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:49.503118  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.562332  3542 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:49.563696  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.564374  3542 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:49.470713  4369 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:49.505707  4541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.009s	sys 0.018s
I20260709 14:51:49.548357  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:49.487366  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45265us
I20260709 14:51:49.487687  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.499781  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:49.551720  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.572744  3547 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:49.573040  2685 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:49.572858  2758 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:49.504756  2612 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:49.552242  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 234) took 2868 ms. Trace:
I20260709 14:51:49.574040  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.498912  4575 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.349460  3546 rpcz_store.cc:276] 0709 14:51:47.189374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.196999 (+  7625us) service_pool.cc:224] Handling call
0709 14:51:47.263722 (+ 66723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.308485 (+ 44763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.314821 (+  6336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:47.342504 (+ 27683us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:47.347383 (+  4879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.225650 (+1878267us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:49.225726 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.225922 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58632576}
W20260709 14:51:49.570022  4541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.010s	sys 0.018s
I20260709 14:51:49.600018  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.601061  3547 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:49.602340  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:49.434639  4385 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.484838  4266 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:49.559759  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.565598  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:49.486770  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 736) took 1446 ms. Trace:
I20260709 14:51:49.567404  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:49.497778  4316 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:49.458505  4451 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 14:51:49.506325  4333 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.573997  2680 rpcz_store.cc:276] 0709 14:51:46.683588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.683769 (+   181us) service_pool.cc:224] Handling call
0709 14:51:46.684175 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:46.684454 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:47.177695 (+493241us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.234946 (+2057251us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b1800127900. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:51:49.549976 (+315030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.551589 (+  1613us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:49.551904 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.552000 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.552164 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13023,"prepare.queue_time_us":459035,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":14304,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9737,"spinlock_wait_cycles":7346432,"thread_start_us":746402,"threads_started":2}
I20260709 14:51:49.514168  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:49.506750  4284 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.603164  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.571636  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.628786  3541 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:49.608943  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.611354  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.612180  3542 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:49.581257  4387 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:49.616329  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49065.6us
W20260709 14:51:49.633109  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 297358us
I20260709 14:51:49.633742  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.411685  4570 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.204s
I20260709 14:51:49.579412  4575 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:49.606561  4561 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 14:51:49.480895  4558 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.619525  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.648074  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.549839  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:49.155289  2605 rpcz_store.cc:276] 0709 14:51:47.381973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.387354 (+  5381us) service_pool.cc:224] Handling call
0709 14:51:47.413599 (+ 26245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.439925 (+ 26326us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:48.617175 (+1177250us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.078777 (+461602us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.083451 (+  4674us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:49.084885 (+  1434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.089195 (+  4310us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.116312 (+ 27117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3480,"prepare.queue_time_us":1163370,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":37207,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":4720,"thread_start_us":1376606,"threads_started":2,"wal-append.queue_time_us":509901}
I20260709 14:51:49.305397  3537 rpcz_store.cc:276] 0709 14:51:47.806673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.839725 (+ 33052us) service_pool.cc:224] Handling call
0709 14:51:47.840162 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:47.840609 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.840640 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:47.841624 (+   984us) log.cc:844] Serialized 26 byte log entry
0709 14:51:48.770505 (+928881us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:49.160864 (+390359us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.185437 (+ 24573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.195496 (+ 10059us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:49.195594 (+    98us) raft_consensus.cc:1685] finished
0709 14:51:49.195613 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.195835 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126792,"prepare.queue_time_us":318771,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":20609,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":139620,"thread_start_us":1198271,"threads_started":2,"wal-append.queue_time_us":1107053}
I20260709 14:51:49.618636  4316 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.630129  2685 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:49.652259  3541 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:49.652590  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 860) took 2035 ms. Trace:
W20260709 14:51:49.633965  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 192231us
I20260709 14:51:49.609853  2752 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:49.659642  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.611758  4566 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:49.639050  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.641649  4586 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.619606  4500 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:49.651360  4582 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 14:51:49.682096  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.682097  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 588) took 1640 ms. Trace:
I20260709 14:51:49.685243  3536 rpcz_store.cc:276] 0709 14:51:48.041334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.615395 (+574061us) service_pool.cc:224] Handling call
0709 14:51:48.652001 (+ 36606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:48.963435 (+311434us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:49.398011 (+434576us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.681598 (+283587us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.681679 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:49.681749 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.681856 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.682018 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":132419,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":298003,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":902573,"threads_started":3}
I20260709 14:51:49.657500  2684 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:49.691378  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 743) took 1195 ms. Trace:
I20260709 14:51:49.682801  2754 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:51:49.692152  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 824) took 1196 ms. Trace:
I20260709 14:51:49.691847  2684 rpcz_store.cc:276] 0709 14:51:48.495579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.656955 (+1161376us) service_pool.cc:224] Handling call
0709 14:51:49.657260 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:49.690965 (+ 33705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.691303 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:49.654350  4528 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:49.663134  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.663753  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.643342  4575 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:49.696504  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 125006us
I20260709 14:51:49.673534  4558 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.674041  4568 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.675546  4532 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.614672  2677 rpcz_store.cc:276] 0709 14:51:48.039877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.043323 (+  3446us) service_pool.cc:224] Handling call
0709 14:51:48.140776 (+ 97453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:48.141162 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:49.083698 (+942536us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.412829 (+329131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.416410 (+  3581us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:49.446043 (+ 29633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.466036 (+ 19993us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.481796 (+ 15760us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34753,"prepare.queue_time_us":812077,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":21438,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":12466,"thread_start_us":1143880,"threads_started":2,"wal-append.queue_time_us":420176}
I20260709 14:51:49.689193  3542 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:49.657068  4496 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.690230  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.653013  2610 rpcz_store.cc:276] 0709 14:51:47.617512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.617782 (+   270us) service_pool.cc:224] Handling call
0709 14:51:47.618130 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:47.639240 (+ 21110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.639294 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:47.639361 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:47.639407 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.652362 (+2012955us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b54002fe9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:49.652412 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.652534 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47781120}
I20260709 14:51:49.692550  2680 rpcz_store.cc:276] 0709 14:51:48.495721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.690592 (+1194871us) service_pool.cc:224] Handling call
0709 14:51:49.690949 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:49.691796 (+   847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:49.691830 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:49.691876 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:49.691911 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.691973 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.692098 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":477}
I20260709 14:51:49.651546  4341 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.704617  3542 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:49.671489  4384 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.698161  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.698195  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.698974  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.695530  2760 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:49.706128  3542 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:49.677047  4500 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:49.707944  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:49.707074  3542 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.715189  3542 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:49.695286  4528 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.702068  4210 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.700532  4454 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:49.725912  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:49.728303  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.349254  2678 rpcz_store.cc:276] 0709 14:51:46.158117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.160986 (+  2869us) service_pool.cc:224] Handling call
0709 14:51:46.234727 (+ 73741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:46.392725 (+157998us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:47.168314 (+775589us) log.cc:844] Serialized 14 byte log entry
0709 14:51:47.534935 (+366621us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:47.549871 (+ 14936us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:47.555582 (+  5711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.224347 (+1668765us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:49.224423 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.224688 (+   265us) inbound_call.cc:177] Queueing success response
0709 14:51:49.232980 (+  8292us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
Metrics: {"prepare.queue_time_us":691224,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":4999,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":340016384,"thread_start_us":989694,"threads_started":2,"wal-append.queue_time_us":507857}
I20260709 14:51:49.733855  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 583) took 2053 ms. Trace:
I20260709 14:51:49.707132  4387 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:49.736831  2607 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:49.676865  4566 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.734392  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:49.740798  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 864) took 1297 ms. Trace:
I20260709 14:51:49.709757  4589 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.705749  4568 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.733450  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.738087  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:49.741150  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:49.737574  3540 rpcz_store.cc:276] 0709 14:51:47.680902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.692919 (+ 12017us) service_pool.cc:224] Handling call
0709 14:51:47.693410 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:47.745149 (+ 51739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:47.745187 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:48.403418 (+658231us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b2c000e3d48. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:51:48.404092 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 14:51:49.085280 (+681188us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:51:49.085342 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.085456 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:49.729543 (+644087us) raft_consensus.cc:1685] finished
0709 14:51:49.729571 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.733771 (+  4200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":11775,"spinlock_wait_cycles":6365056,"thread_start_us":680681,"threads_started":1,"wal-append.queue_time_us":772514}
I20260709 14:51:49.714811  4532 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:49.741289  2607 rpcz_store.cc:276] 0709 14:51:48.443500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.443802 (+   302us) service_pool.cc:224] Handling call
0709 14:51:49.736098 (+1292296us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:49.736625 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:49.740523 (+  3898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.740735 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12750592}
I20260709 14:51:49.715931  4500 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.716115  4496 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:49.714581  4489 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.681579  4586 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.765959  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.725514  4528 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:49.723610  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.745246  4332 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.772961  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:49.788939  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 353315us
I20260709 14:51:49.756017  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:49.742882  4597 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.742027  4570 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 14:51:49.753454  4341 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.736053  4454 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.755432  4568 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.721437  4533 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 14:51:49.766594  4532 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 14:51:49.758559  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:49.756721  4580 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 14:51:49.769856  3536 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:49.769106  3542 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 14:51:49.836421  3542 pending_rounds.cc:85] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
I20260709 14:51:49.836752  3542 pending_rounds.cc:107] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:51:49.804314  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.836977  3542 raft_consensus.cc:2889] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:49.810801  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.764971  4506 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:49.827167  3409 connection.cc:256] RPC call timeout handler was delayed by -1.216152620999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
I20260709 14:51:49.766387  4293 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:49.775310  4539 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.826s	user 0.005s	sys 0.000s
I20260709 14:51:49.840710  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.798091  4577 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:49.785135  4385 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:49.842407  2758 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:49.816711  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.784961  2603 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:49.843505  4339 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:51:49.815461  4397 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:49.844017  4266 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:51:49.844365  4397 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.844888  4397 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:49.845615  4266 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.846191  4266 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.821785  4210 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:49.762413  4589 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.844954  4332 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:49.781491  4333 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.811473  4597 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.849491  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.849900  4597 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.841179  4293 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.851650  4293 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:49.852392  4586 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:49.853281  4585 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 14:51:49.854529  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 748) took 1123 ms. Trace:
I20260709 14:51:49.855010  2680 rpcz_store.cc:276] 0709 14:51:48.731494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.710219 (+978725us) service_pool.cc:224] Handling call
0709 14:51:49.710673 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:49.711079 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:49.711120 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:49.711628 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 14:51:49.819134 (+107506us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:49.819181 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.819253 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:49.854343 (+ 35090us) raft_consensus.cc:1685] finished
0709 14:51:49.854360 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.854478 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6338,"raft.queue_time_us":781,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6211,"thread_start_us":75515,"threads_started":1,"wal-append.queue_time_us":124226}
I20260709 14:51:49.856099  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.857038  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:49.849565  4589 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.814285  4341 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:49.826081  4587 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 14:51:49.838651  4506 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:49.869850  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 809) took 2825 ms (client timeout 1705 ms). Trace:
W20260709 14:51:49.870416  2683 rpcz_store.cc:269] 0709 14:51:47.044499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.089882 (+ 45383us) service_pool.cc:224] Handling call
0709 14:51:49.869534 (+2779652us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54003435d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:49.869739 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438639488}
I20260709 14:51:49.871192  4506 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.770150  4494 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:49.793864  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60581.8us
I20260709 14:51:49.872627  4341 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.842600  4599 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:49.820508  4458 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:51:49.834951  4568 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:49.883169  4528 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:49.860872  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:49.885901  4506 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:49.886512  4506 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:49.835639  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.843989  4339 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:49.809430  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53563.8us
I20260709 14:51:49.895495  3543 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:49.895557  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:49.865600  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.862557  4603 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:49.879063  2680 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:49.880513  4284 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.829133  2757 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:51:49.883224  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.891543  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 487) took 3266 ms. Trace:
I20260709 14:51:49.894474  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.905792  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.897064  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.906468  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:49.907161  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.897557  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.899248  3541 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:49.899849  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:49.853927  4566 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:49.912650  2650 connection.cc:256] RPC call timeout handler was delayed by -1.1511916559999236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 14:51:49.909574  2758 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:49.910772  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.913807  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.914431  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:49.887702  4266 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.907694  4210 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:49.915253  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.924187  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.916507  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:49.944109  2754 raft_consensus.cc:2368] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:49.926789  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.916009  4210 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.946468  4210 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:49.903805  3538 rpcz_store.cc:276] 0709 14:51:46.625015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.625195 (+   180us) service_pool.cc:224] Handling call
0709 14:51:48.324845 (+1699650us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:48.325150 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:49.028966 (+703816us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:49.487451 (+458485us) log.cc:844] Serialized 14 byte log entry
0709 14:51:49.767398 (+279947us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:49.770034 (+  2636us) log.cc:844] Serialized 26 byte log entry
0709 14:51:49.770275 (+   241us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:49.770328 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:49.770454 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:49.890296 (+119842us) raft_consensus.cc:1685] finished
0709 14:51:49.891284 (+   988us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:49.891470 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47283,"prepare.queue_time_us":79939,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":1303,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":4671,"spinlock_wait_cycles":42035072,"thread_start_us":1288141,"threads_started":3,"wal-append.queue_time_us":264126}
I20260709 14:51:49.947834  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:49.910176  4422 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.944271  4599 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:49.949364  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:49.949527  4422 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:49.950065  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:49.952054  4422 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.903839  4570 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:49.953658  4570 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:49.954071  4570 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:49.949544  4599 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:49.955783  4599 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:49.916119  4533 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:49.957162  4533 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.957572  4533 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:49.957707  4532 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:49.958199  4532 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:49.958611  4532 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:49.961689  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:49.963786  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.899741  4339 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:49.916468  4332 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.916652  4501 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:49.915838  4266 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.944653  4385 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:49.978458  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 102602us
I20260709 14:51:49.918143  4451 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.887694  4506 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:49.946844  4284 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:49.916402  4490 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.947474  4577 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:50.012722  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:49.983777  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.916795  4509 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:49.918334  4479 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:49.916410  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:49.981618  4594 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.188s	user 0.002s	sys 0.000s
I20260709 14:51:49.977685  4484 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.034955  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:49.909590  4597 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:49.977293  4574 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 14:51:50.037883  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54248.9us
I20260709 14:51:50.094362  4517 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:50.088203  4479 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.094729  4316 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.095045  4422 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:49.983403  4497 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:50.095263  3546 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:50.095772  4528 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.039235  4451 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.032733  4456 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:50.036056  4380 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.021271  4385 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.136534  4385 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:50.136312  4380 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.030503  4266 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.130769  4422 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.140661  4266 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:50.141678  4422 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.142189  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:50.108137  3540 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:50.117730  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:50.120101  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:50.127934  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93154us
I20260709 14:51:50.144274  4380 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.147841  4510 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:50.148425  4510 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:50.149479  4510 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.153079  4451 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.153262  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.154204  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:50.155421  4484 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.155625  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.042076  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.156203  4484 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.108225  4586 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:50.111477  4517 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.143045  2683 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.021098  4332 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.150691  4510 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.041178  4490 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.130424  4495 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:50.127996  4497 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.127977  4275 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 14:51:50.181459  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.162622  4501 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:50.182637  2758 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:51:50.169940  4604 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.278s	user 0.002s	sys 0.000s
I20260709 14:51:50.191087  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.184116  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.192816  2758 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:50.193778  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:50.195490  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:50.196455  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.197647  2755 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:50.125730  4509 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.203573  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.187062  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.169191  4517 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:50.180289  4597 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.179309  4497 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.162400  4384 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 14:51:50.161118  4500 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.209314  2760 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
W20260709 14:51:50.213742  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 102271us
I20260709 14:51:50.169230  4586 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.189157  4501 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:50.219434  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.179652  4397 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.236744  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.239104  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:50.234102  2683 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:50.247237  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:50.248214  4597 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:50.251788  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.253387  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.254128  2760 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:50.236125  4509 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.255285  4490 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:50.247560  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:50.256002  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.256385  4341 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.257259  4525 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:50.257586  2680 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.257903  4525 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:50.258642  4599 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.232803  4501 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:50.259039  4525 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.260124  4597 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:50.261248  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.261482  4599 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.257444  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.263023  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.263535  4339 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.264060  4479 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.231775  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.264246  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.348315  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.293570  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.283061  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.306913  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 274) took 2749 ms. Trace:
I20260709 14:51:50.273324  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.350069  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.269184  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.296458  3542 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.310681  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 882) took 1095 ms. Trace:
I20260709 14:51:50.285969  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.352563  3547 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:50.341907  2577 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:50.354463  2577 leader_election.cc:403] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 14:51:50.351958  2612 rpcz_store.cc:276] 0709 14:51:49.215060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.229530 (+ 14470us) service_pool.cc:224] Handling call
0709 14:51:50.310488 (+1080958us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:50.310616 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347393408}
I20260709 14:51:50.277459  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.361946  2686 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:50.280671  2679 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.355847  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 735) took 2316 ms. Trace:
I20260709 14:51:50.363292  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.365192  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.367043  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.235771  4484 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.363139  2682 rpcz_store.cc:276] 0709 14:51:48.039533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:48.042277 (+  2744us) service_pool.cc:224] Handling call
0709 14:51:48.090780 (+ 48503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:48.419266 (+328486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.355575 (+1936309us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:50.355783 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752573824}
I20260709 14:51:50.350209  2759 rpcz_store.cc:276] 0709 14:51:47.557652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:47.557807 (+   155us) service_pool.cc:224] Handling call
0709 14:51:47.558198 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:49.073851 (+1515653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.306586 (+1232735us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:50.306797 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3384878848}
I20260709 14:51:50.273754  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.380345  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:50.380668  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.387742  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.388640  2760 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.378187  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:50.373056  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108371us
W20260709 14:51:50.288326  4551 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.299s	user 0.004s	sys 0.000s
I20260709 14:51:50.392207  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.392526  4339 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.392815  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.237815  4293 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.272033  4589 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:50.273018  4507 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.394070  4603 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.394210  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.356276  4586 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:50.254128  4533 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.296072  4275 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:50.395058  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.308709  4369 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.397778  4490 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:51:50.398326  4484 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
W20260709 14:51:50.398710  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:50.399972  4239 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:50.400570  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 745) took 1897 ms. Trace:
I20260709 14:51:50.309654  4500 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:50.308153  4397 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.403847  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.265393  4332 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.404799  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.443439  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:50.438793  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 204634us
I20260709 14:51:50.306214  4385 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.299239  4494 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.446126  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.452055  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.453719  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.407781  4532 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.409610  4497 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.419778  2685 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
W20260709 14:51:50.298081  4575 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.003s
I20260709 14:51:50.390431  3541 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:50.315833  4364 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:50.423058  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 242) took 1184 ms. Trace:
I20260709 14:51:50.462334  2680 rpcz_store.cc:276] 0709 14:51:49.238262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.902117 (+663855us) service_pool.cc:224] Handling call
0709 14:51:49.916956 (+ 14839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:50.258090 (+341134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.422631 (+164541us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:50.422966 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782840192}
I20260709 14:51:50.304234  4316 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.357028  4341 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.431241  2753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:50.432739  2754 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:50.469928  2607 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:50.430953  4589 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.439273  2684 rpcz_store.cc:276] 0709 14:51:48.502939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.693239 (+1190300us) service_pool.cc:224] Handling call
0709 14:51:49.725087 (+ 31848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:49.725295 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:49.745747 (+ 20452us) log.cc:844] Serialized 14 byte log entry
0709 14:51:50.156614 (+410867us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b1800400000. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:51:50.399503 (+242889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:50.399687 (+   184us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:50.399743 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.400341 (+   598us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:50.400492 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17762,"prepare.queue_time_us":19397,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":18525,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":11481984,"thread_start_us":222079,"threads_started":1}
I20260709 14:51:50.318555  4570 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.447899  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.269528  4210 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:50.455452  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.445776  4454 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.451386  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.458551  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.466972  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.470283  2753 raft_consensus.cc:606] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.492115  2753 raft_consensus.cc:614] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:50.493006  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.493659  2611 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.496850  2684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f32f10706d04497089f6159669dc04a4"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:51:50.497538  2684 raft_consensus.cc:606] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.497884  2684 raft_consensus.cc:614] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:51:50.498023  4570 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.499500   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:51:50.501500   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:51:50.501950  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.503129  4293 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.503845  3540 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:50.504678  4533 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:50.472102  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 130461us
I20260709 14:51:50.477880  2608 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:50.461018  4494 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.479146  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.459739  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.465758  4364 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:50.487659  3536 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:50.488699  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:50.517918  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 170894us
I20260709 14:51:50.518704  2610 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:50.520280  2610 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:50.521814  2610 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:50.472888  4589 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:50.471761  4603 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:50.491446  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.527974  2606 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:50.528955  2611 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:50.529064  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.477694  4397 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:50.511536  3546 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:50.512492  2608 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.439030  4275 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:50.502348  2760 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:50.506407  4210 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.537437  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.517014  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.469796  4575 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.524096  2610 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
W20260709 14:51:50.531647  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 224985us
I20260709 14:51:50.535128  2606 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:50.539577  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.542923  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.544739  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.542989  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:50.548671  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.550393  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.548593  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.552549  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:50.554981  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.556007  2608 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:50.557159  3543 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.563830  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.565948  4575 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.569967  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:50.547072  4532 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.586262  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.592316  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.582294  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.594594  2606 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:50.533075  4603 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:50.564364  4586 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.596493  2606 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.540838  4397 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.593727  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.583563  4597 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:50.564136  4570 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.566681  4599 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:50.547957  4517 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.605873  2608 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:50.613528  2607 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 14:51:50.616012  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:50.616747  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:50.617571  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.604806  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.605082  4597 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.627012  2752 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:51:50.628507  4603 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:50.630002  2759 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:50.636909  2759 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:50.640650  4397 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.643559  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.645077  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.645325  2759 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.645759  2603 raft_consensus.cc:2376] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5. Re-sending same reply.
I20260709 14:51:50.646420  4506 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:50.646941  4506 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:50.645182  4517 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.650068  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.659409  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.612681  4385 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:50.652768  2607 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:50.652921  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.653870  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.654847  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.661399  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.662599  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:50.677536  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:50.680851  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.649605  4597 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:50.596414  4364 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.692551  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.709074  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.710752  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.648023  4506 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.715904  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.713099  4501 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.721846  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:50.722568  4031 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:50.723834  2751 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:50.714232  4210 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:50.715094  4620 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:50.728363  2611 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:50.691357  4444 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:51:50.615729  4275 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:50.731384  4293 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:50.732940  4458 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:50.739944  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:50.665949  4562 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 14:51:50.664420  4496 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 14:51:50.747366  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.754498  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.689802  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.749485  3547 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:50.676818  4617 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 14:51:50.750725  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:50.731514  4574 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.691689  4336 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 14:51:50.690917  4250 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 14:51:50.763116  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.752010  4458 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:50.765677  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.715021  4528 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 14:51:50.768874  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.714241  4372 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 14:51:50.780031  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.776046  2679 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:50.765793  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.748184  4444 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:50.784368  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.786975  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.788544  2682 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:50.753021  4494 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:50.792122  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.759908  4341 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.796039  2684 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:50.758515  4314 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:50.799163  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.801541  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.758903  4501 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:50.803795  2751 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.804684  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:50.776742  2752 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:50.817495  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:50.793079  4444 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:50.819657  4380 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:50.821564  4490 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:50.758518  4507 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 14:51:50.823781  4384 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:50.824543  4629 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:51:50.808342  4568 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 14:51:50.831053  4577 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:50.823802  2760 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:50.837869  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:50.823804  4628 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 14:51:50.904181  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:50.905459  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 183590us
I20260709 14:51:50.794950  4385 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:50.902057  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64271.9us
I20260709 14:51:50.910312  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:50.911932  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:50.811359  2605 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:50.804788  4617 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 14:51:50.908079  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 170718us
W20260709 14:51:50.881239  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 82155.5us
I20260709 14:51:50.924934  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.926327  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:50.906910  4570 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:50.812587  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.885850  4532 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:50.877059  4496 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:50.895150  4380 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:50.914047  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 820) took 2437 ms. Trace:
I20260709 14:51:50.937580  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 887) took 1098 ms. Trace:
I20260709 14:51:50.939451  2681 rpcz_store.cc:276] 0709 14:51:48.476849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.135616 (+658767us) service_pool.cc:224] Handling call
0709 14:51:49.135924 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:49.136204 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:51:49.157968 (+ 21764us) log.cc:844] Serialized 14 byte log entry
0709 14:51:50.718322 (+1560354us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180014ef00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:51:50.913657 (+195335us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:50.913724 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:50.913782 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.913856 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:50.913984 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1202,"prepare.queue_time_us":21468,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":75,"raft.queue_time_us":1305,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":2967,"spinlock_wait_cycles":3241728,"thread_start_us":193782,"threads_started":1}
I20260709 14:51:50.939982  2753 rpcz_store.cc:276] 0709 14:51:49.839446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.588992 (+749546us) service_pool.cc:224] Handling call
0709 14:51:50.589515 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:50.590075 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:50.590116 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:50.591506 (+  1390us) log.cc:844] Serialized 26 byte log entry
0709 14:51:50.664972 (+ 73466us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:50.665030 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.665431 (+   401us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:50.937212 (+271781us) raft_consensus.cc:1685] finished
0709 14:51:50.937249 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:50.937485 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":1374080,"thread_start_us":72892,"threads_started":1,"wal-append.queue_time_us":120321}
W20260709 14:51:50.938814  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 167008us
I20260709 14:51:50.913285  4385 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:50.913982  4456 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:50.942144  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:50.919291  4575 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.765908  2603 raft_consensus.cc:2376] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4. Re-sending same reply.
I20260709 14:51:50.944993  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:50.905117  4577 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:50.932840  4293 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:50.939467  4507 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:50.982182  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:50.966377  4385 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:50.947981  4570 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:50.997406  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:51.000422  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:51.005496  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 93626.6us
I20260709 14:51:51.006956  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:51.008008  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:51.012807  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84604us
I20260709 14:51:51.013098  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.014717  2681 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:51.023633  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:50.981834  4496 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 14:51:50.970680  4456 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.043265  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.014312  4570 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:51.018260  4510 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:51.050144  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65715.5us
I20260709 14:51:51.007525  4293 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 14:51:51.022009  4314 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:51.057253  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.031785  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.016513  4341 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 14:51:51.007254  4641 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 14:51:51.065158  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:51.077922  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.010056  4380 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.054486  4510 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.084460  2760 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:51:51.046955  4494 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 14:51:51.088897  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 75894.7us
I20260709 14:51:51.108582  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.042253  4596 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 14:51:51.117460  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:51.119503  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52373.5us
I20260709 14:51:51.122160  2753 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:51.126505  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:51.087750  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.106943  4380 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:51.116345  4652 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:51:51.137820  4631 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 14:51:51.122503  4500 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:51.127046  4031 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:51.072012  4570 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 14:51:51.127216  4510 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:51.129863  4332 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:51.140719  4642 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 14:51:51.135371  4341 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:51.160296  4341 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.160766  4341 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:51.144652  4634 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.205s	user 0.000s	sys 0.002s
I20260709 14:51:51.161729  4509 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:51.162345  4509 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:51.142675  4647 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:51.157402  4597 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.163085  4647 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:51.163620  4647 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:51.163913  4589 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.628s	user 0.005s	sys 0.008s
I20260709 14:51:51.164485  4589 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.165897  4603 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:51.166406  4603 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.131373  4653 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:51.149701  4632 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 14:51:51.150290  4635 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 14:51:51.089346  4646 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 14:51:51.168119  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:51.143455  4031 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:51.176852  4031 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:51.141861  4333 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:51.168370  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:51.177374  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 830) took 1328 ms. Trace:
I20260709 14:51:51.178556  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 286) took 1015 ms. Trace:
I20260709 14:51:51.179155  2755 rpcz_store.cc:276] 0709 14:51:50.163649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.309438 (+145789us) service_pool.cc:224] Handling call
0709 14:51:51.168154 (+858716us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.178478 (+ 10324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38996352}
I20260709 14:51:51.178612  2678 rpcz_store.cc:276] 0709 14:51:49.848659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.919649 (+ 70990us) service_pool.cc:224] Handling call
0709 14:51:50.039828 (+120179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:50.132686 (+ 92858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:50.135992 (+  3306us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:50.136646 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 14:51:50.609122 (+472476us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b18003ed640. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:51:50.820450 (+211328us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:50.820515 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.820632 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:51.176982 (+356350us) raft_consensus.cc:1685] finished
0709 14:51:51.177008 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:51.177300 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125,"raft.queue_time_us":10439,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"spinlock_wait_cycles":49631488,"thread_start_us":186082,"threads_started":1,"wal-append.queue_time_us":814019}
I20260709 14:51:51.177711  4653 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.184398  4653 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:51.186359  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:51.170220  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.187991  2756 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:51.139842  4652 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:51.194931  4652 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:51.195611  4646 consensus_queue.cc:1243] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:51:51.158941  4490 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:51.189122  4497 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:51.189843  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 871) took 1875 ms (client timeout 1546 ms). Trace:
I20260709 14:51:51.191589  4239 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:51.192070  4387 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.192536  4620 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:51.194140  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:51.149701  4644 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 14:51:51.197094  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:51.203742  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146617us
W20260709 14:51:51.197242  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65346.7us
W20260709 14:51:51.235073  2756 rpcz_store.cc:269] 0709 14:51:49.314663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.314862 (+   199us) service_pool.cc:224] Handling call
0709 14:51:50.311356 (+996494us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.169986 (+858630us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521e9c 00007fa16b521a5b 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.189747 (+ 19761us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53117440}
W20260709 14:51:51.242906  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48918.2us
I20260709 14:51:51.264572  2651 leader_election.cc:400] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:51.264791  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.265305  2651 leader_election.cc:403] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:51.242328  4620 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:51.267263  4620 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.268546  4620 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:51.244801  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47810.9us
I20260709 14:51:51.278312  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:51.197686  4593 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.406s	user 0.004s	sys 0.004s
W20260709 14:51:51.276244  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 198399us
I20260709 14:51:51.285051  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.241995  4239 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:51.271409  4352 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:51.283974  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.163496  4509 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.312501  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:51.312897  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.315793  2756 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:51.277554  4651 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:51.282742  4541 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4.
I20260709 14:51:51.281514  4596 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:51.353421  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.274922  4332 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 14:51:51.247346  4387 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.310220  4577 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.369689  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:51.374867  3543 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:51.360114  4651 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:51.317487  4333 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.379137  3543 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.383692  3547 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:51.325966  4589 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.324870  4646 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:51:51.386158  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142401us
I20260709 14:51:51.386622  3543 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.391848  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.358443  4636 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:51.392314  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:51.366403  4637 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:51.351243  4501 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 14:51:51.388962  4541 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:51.389386  4509 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.311136  4649 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 14:51:51.389734  4651 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:51.404151  4458 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.613s	user 0.015s	sys 0.000s
I20260709 14:51:51.391945  4661 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:51.401399  4637 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:51.405511  4352 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:51.393878  4333 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.410527  4663 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:51.422925  4637 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:51.426076  4661 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.431463  4646 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:51.420564  4458 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.460040  4352 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.462423  4647 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.473940  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.479638  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:51.458050  4663 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:51.484436  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.485662  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.486236  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.487947  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.488234  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.488750  4333 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:51.488830  4661 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.489259  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.491598  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:51.491312  2681 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:51.492669  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.493453  2685 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:51.496347  2608 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:51.473112  4031 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.499071  4647 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:51.501435  4031 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:51.503075  4333 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:51.491169  4653 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.504604  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.505515  2682 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:51.502302  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:51.513479  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:51.480472  4575 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:51.515520  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.528688  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:51.529297   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:51:51.521066  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 243006us
I20260709 14:51:51.537153  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:51.538385  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.535395  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.540486  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:51.541193  2608 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:51.541935  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:51.543752  2685 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:51.506605  4653 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:51.472174  4646 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.553489  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.509575  4387 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:51.540930  4444 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.630s	user 0.004s	sys 0.010s
I20260709 14:51:51.556396  2685 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:51.483582  4654 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 14:51:51.539558  4670 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:51.541123  4647 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 14:51:51.568605  4670 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.569180  4670 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.571698  2608 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:51.572598  2604 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:51.547834  4385 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.540s	user 0.014s	sys 0.000s
I20260709 14:51:51.573897  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:51.547417  4541 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:51.552815  4031 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:51.569757  4647 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:51.555478  4333 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:51.585923  4341 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.592518  4458 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.594092  4458 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:51:51.595712  4671 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:51:51.566541  4444 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.507130  4663 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.583356  4541 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:51.581120  2682 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:51.591058  4031 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:51.591851  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.625177  2681 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:51.631042  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.636075  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.637491  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.638111  2682 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:51.580468  4653 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:51.639279  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:51.640074  4369 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:51.614328  4541 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:51.640545  4369 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:51.643677  4575 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:51.644259  4575 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.643484  4653 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.586711  4490 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 14:51:51.651633  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:51.652379  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94811.8us
I20260709 14:51:51.610682  4663 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.608000  4577 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:51.582271  4385 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.644749  4575 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.602813  4444 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:51.645905  4647 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:51.652933  4661 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:51.606076  4510 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:51.662541  4380 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.005s
I20260709 14:51:51.647512  2677 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
W20260709 14:51:51.670076  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86208us
I20260709 14:51:51.674391  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.675160  4385 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.675257  2677 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:51.676574  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.677891  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.659741  4369 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.665450  4647 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.678752  4670 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.683064  4647 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.660775  4577 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.680037  4369 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:51.680573  4332 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:51.680694  4031 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.687192  4332 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.689541  4332 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.681233  4510 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.679248  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.776378  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:51.782534  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 271) took 1299 ms. Trace:
I20260709 14:51:51.784432  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 899) took 1255 ms. Trace:
W20260709 14:51:51.673646  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:51.701962  4239 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.672605  4341 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.714311  4577 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.717759  4456 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.690315  4603 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.814052  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.817058  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.736119  4641 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:51.818539  4641 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:51.819370  4316 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:51.819729  4239 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:51.820025  4316 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:51.820860  4641 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.711344  4646 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:51.729581  4638 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.693s	user 0.004s	sys 0.000s
I20260709 14:51:51.725776  4541 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.680843  4653 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.682065  4652 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.798096  2612 rpcz_store.cc:276] 0709 14:51:50.482862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.483648 (+   786us) service_pool.cc:224] Handling call
0709 14:51:50.484145 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:50.609584 (+125439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:50.609622 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:50.610560 (+   938us) log.cc:844] Serialized 26 byte log entry
0709 14:51:50.791217 (+180657us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:50.791261 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:51.781230 (+989969us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54003c8554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:51.781378 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:51.781545 (+   167us) raft_consensus.cc:1685] finished
0709 14:51:51.781592 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:51.782213 (+   621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287776000,"thread_start_us":175783,"threads_started":1,"wal-append.queue_time_us":292535}
I20260709 14:51:51.731146  4575 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.668330  4661 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.833428  4364 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.834113  2684 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:51.726929  4637 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.833549  4575 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.710369  4670 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:51.725785  4682 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 14:51:51.822221  4316 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.838032  4637 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:51.731236  4665 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.323s	user 0.002s	sys 0.003s
I20260709 14:51:51.808451  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:51.833542  4652 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:51.840605  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.840826  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:51.841964  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 251) took 1071 ms. Trace:
I20260709 14:51:51.842187  2610 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:51.842675  4239 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:51.810297  2756 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:51:51.843828  4316 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:51.805222  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:51.790789  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.845674  4541 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.846417  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:51.803125  4341 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.796409  2677 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:51.725705  4397 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:51:51.813509  4586 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:51.851074  4586 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.817365  4603 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.700851  4651 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.790740  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.817185  4687 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:51.867198  4687 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:51.867441  2679 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:51.867851  4687 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.835175  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 253) took 1063 ms. Trace:
I20260709 14:51:51.842478  2683 rpcz_store.cc:276] 0709 14:51:50.770535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.811337 (+ 40802us) service_pool.cc:224] Handling call
0709 14:51:51.840031 (+1028694us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.841871 (+  1840us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3426560}
I20260709 14:51:51.798295  2611 rpcz_store.cc:276] 0709 14:51:50.528549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.529890 (+  1341us) service_pool.cc:224] Handling call
0709 14:51:50.530208 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:50.729191 (+198983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:51.783765 (+1054574us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54002ce054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:51.784167 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455019904}
I20260709 14:51:51.805413  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.829746  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:51.856060  4586 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.859880  4641 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:51.861667  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:51.853358  4341 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:51.864311  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:51.864639  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:51.850167  4397 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:51.877915  4670 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:51.878978  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:51.877959  2681 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:51.884853  4661 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:51.885267  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 759) took 1118 ms. Trace:
I20260709 14:51:51.885082  3547 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:51.892784  2682 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:51.895339  2677 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:51.869175  2684 rpcz_store.cc:276] 0709 14:51:50.771175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.808817 (+ 37642us) service_pool.cc:224] Handling call
0709 14:51:51.816452 (+1007635us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.835002 (+ 18550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8912256}
I20260709 14:51:51.897657  4239 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.898365  4496 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:51.898933  4496 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:51.868789  4651 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:51.894812  2682 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:51.902674  4670 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:51.907775  2679 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:51.907387  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.909188  2756 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:51.909436  3545 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 1.
I20260709 14:51:51.895341  2678 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:51.911604  4341 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.900295  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:51.891767  2686 rpcz_store.cc:276] 0709 14:51:50.766791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.766957 (+   166us) service_pool.cc:224] Handling call
0709 14:51:50.796920 (+ 29963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:50.905635 (+108715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:50.905670 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:50.905720 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:50.905768 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:51.884623 (+978855us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:51.884720 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:51.885042 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249381120}
I20260709 14:51:51.920456  3545 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:51.932984  2607 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
W20260709 14:51:51.936028  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 282) took 1904 ms (client timeout 2160 ms). Trace:
W20260709 14:51:51.936704  2758 rpcz_store.cc:269] 0709 14:51:50.031854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.301917 (+270063us) service_pool.cc:224] Handling call
0709 14:51:51.878390 (+1576473us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:51.935893 (+ 57503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33622272}
I20260709 14:51:51.937640  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:51.949538  4210 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:51.950299  4210 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:51.951916  4210 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:51.953670  4210 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:51.950717  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:51.955186  4625 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.264s	user 0.006s	sys 0.000s
I20260709 14:51:51.955682  4314 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:51.956243  4314 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:51.957858  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 519) took 1507 ms. Trace:
I20260709 14:51:51.958420  3538 rpcz_store.cc:276] 0709 14:51:50.450380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.475156 (+ 24776us) service_pool.cc:224] Handling call
0709 14:51:50.475542 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:50.475984 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:50.476022 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:50.478865 (+  2843us) log.cc:844] Serialized 26 byte log entry
0709 14:51:50.559307 (+ 80442us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:50.564076 (+  4769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:50.564487 (+   411us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:51.945756 (+1381269us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:51.957121 (+ 11365us) raft_consensus.cc:1685] finished
0709 14:51:51.957160 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:51.957597 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3586,"spinlock_wait_cycles":22696832,"thread_start_us":67630,"threads_started":1,"wal-append.queue_time_us":137159}
I20260709 14:51:51.900439  4586 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:51.993243  4575 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:51.996266  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.996145  4586 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:51.943367  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:51.999384  4586 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:51.949815  3543 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:52.023294  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:51.962622  4314 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:52.037014  4577 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.038022  4577 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.039079  4496 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:52.040392  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 212094us
I20260709 14:51:52.044898  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:52.047149  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110017us
I20260709 14:51:52.047950  4653 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:52.048110  4675 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:52.048748  4675 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:52.049742  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.041499  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.052421  4658 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:52.053112  4658 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:52.053632  4658 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.047164  3538 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:52.050089  4675 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:52.035214  4239 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:52.061192  4675 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:52.040693  4577 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.069274  4670 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:52.099241  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.112960  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.113340  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.113909  2758 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:52.115139  4268 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:52.115628  2755 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:52.115808  4268 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:52.117220  4268 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:52.105973  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.134605  4250 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:52.135332  4250 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:52.136659  4250 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:52.139659  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.138567  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:52.144320  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79580.9us
I20260709 14:51:52.131600  4500 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:52.151881  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 102379us
I20260709 14:51:52.134627  4268 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.167966  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:52.169445  4670 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:52.178617  3537 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:52.179086  4586 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 14:51:52.179617  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:52.175369  4636 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.030s	sys 0.017s
I20260709 14:51:52.158231  4500 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:52.173939  4497 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 14:51:52.200629  3545 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:51:52.202414  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:52.204433  4369 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.014s
I20260709 14:51:52.204690  2606 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:51:52.230371  4636 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.031s	sys 0.017s
I20260709 14:51:52.215253  4532 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:52.215525  4575 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.216464  4528 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.219638  4659 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.220268  4293 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:51:52.221894  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.224114  3540 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
W20260709 14:51:52.225718  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.214727  4597 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.230816  4656 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:52.233014  3545 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:52.233660  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.206933  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:52.237394  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 887) took 1637 ms. Trace:
I20260709 14:51:52.223665  4314 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:52.252936  2759 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:52.255291  4589 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.255580  2758 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:52.255857  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:52.264000  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.264614  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:52.218809  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:52.266901  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:52.289105  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.289255  2683 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.224202  4500 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:52.282171  2606 rpcz_store.cc:276] 0709 14:51:50.600430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.602712 (+  2282us) service_pool.cc:224] Handling call
0709 14:51:52.203774 (+1601062us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:52.204183 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.234884 (+ 30701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:52.237312 (+  2428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8163712}
W20260709 14:51:52.280573  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.273849  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.296555  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:52.297724  2686 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:52.299124  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:52.300493  3545 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:51:52.299124  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77431us
I20260709 14:51:52.300407  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:52.230022  4658 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.311149  2681 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:52.281108  4656 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:52.313496  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.308878  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.319422  3543 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 14:51:52.319876  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:52.327360  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.329631  4266 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:52.330598  4689 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 14:51:52.330346  4266 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:52.331297  4266 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:52.332069  4507 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.333077  4507 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.336630  4653 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:52.337731  3546 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:52.314260  4532 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.340780  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.341199  4507 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.341220  4532 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.343254  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:52.347512  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 128878us
I20260709 14:51:52.348912  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:52.350368  4275 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:52.314247  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.351907  3540 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:52.353097  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.353852  3540 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:52.357414  4700 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 14:51:52.309331  4681 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.218s
I20260709 14:51:52.359190  4648 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:52.360944  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.346207  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.365257  4496 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.370510  4506 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:52.371912  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.373804  4372 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:52.374322  4372 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:52.376238  3545 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:52.374727  4372 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:52.379873  4250 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:52.381181  4641 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:51:52.381712  4641 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:52.381508  4397 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.277076  4528 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.382903  4641 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.384455  4641 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:52.384299  4451 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:52.385708  4364 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:52.386094  4364 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:52.387290  4364 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:52.387926  4699 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 14:51:52.388796  4364 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.320732  2756 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:52.379422  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:52.397521  4620 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:52.400144  4656 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:52.401611  4656 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.403427  4451 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:52.404632  4451 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:52.406354  4316 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.003s
I20260709 14:51:52.395607  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:52.408288  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.408414  4641 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.409583  4369 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.362740  3542 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:52.322057  3543 pending_rounds.cc:85] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
I20260709 14:51:52.376263  4437 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:52.415930  2755 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:52.413136  2680 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:52.417095  3543 pending_rounds.cc:107] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:51:52.418612  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:52.418145  4437 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:52.422540  3543 raft_consensus.cc:2889] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:51:52.426784  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:52.416138  4603 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.005s
I20260709 14:51:52.428443  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.430588  4702 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:52.431483  4454 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:52.431424  4702 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.432168  4454 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:52.434206  4702 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.434403  4454 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:52.426275  2680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 255) took 1648 ms (client timeout 1545 ms). Trace:
I20260709 14:51:52.412710  4586 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:52.378302  4643 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 14:51:52.418371  4369 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 14:51:52.436589  2680 rpcz_store.cc:269] 0709 14:51:50.771702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:50.805774 (+ 34072us) service_pool.cc:224] Handling call
0709 14:51:52.206491 (+1400717us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:52.420381 (+213890us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16827264}
I20260709 14:51:52.447361  2756 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:52.450845  3546 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:52.451184  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.422482  4528 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:52.418689  3537 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:52.458668  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.462571  3538 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:51:52.441102  4239 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.463625  3542 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:52.467662  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.468771  4397 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:52.469924  4643 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:52.474684  3546 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:52.445107  4586 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.449024  4369 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:52.483163  4687 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.011s
I20260709 14:51:52.441571  4500 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.488189  4437 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:52.491075  4643 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:51:52.493206  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.498549  2758 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:52.499069  4458 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:52.500054  2757 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:52.501024  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.502249  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.503443  4451 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:52.545121  2760 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:52.503435  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.456279  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:52.511215  4333 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:52.511936  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:52.513090  4703 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:52.478238  2681 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:52.551726  2753 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:52.552284  2680 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:52.511243  4702 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.517354  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.518335  4454 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:52.455461  4316 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 14:51:52.537567  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 408003us
I20260709 14:51:52.535521  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.444612  4641 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:52.548061  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 228328us
I20260709 14:51:52.514730  4266 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.567809  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 534) took 1198 ms. Trace:
I20260709 14:51:52.559988  3540 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:52.570365  2756 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:52.572865  2612 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:51:52.574251  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:52.576454  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.576918  4314 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:51:52.579994  4620 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:52.580318  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 172221us
I20260709 14:51:52.575773  4577 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.582167  4458 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:52.583644  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:52.575675  4454 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:52.547497  4333 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:52.591085  4437 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:52.591220  3538 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:52.595188  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:52.601565  2607 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:52.602016  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:52.602809  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:52.603166  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:52.605208  4695 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
W20260709 14:51:52.591265  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.578436  4655 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:52.568502  3543 rpcz_store.cc:276] 0709 14:51:51.369143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:51.374282 (+  5139us) service_pool.cc:224] Handling call
0709 14:51:51.374662 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:51.948689 (+574027us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b580060fed0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5a20aa 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4
0709 14:51:52.543641 (+594952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:52.567715 (+ 24074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42635136}
I20260709 14:51:52.613694  2759 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:52.602686  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:52.525146  4239 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:52.622951  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48771.2us
I20260709 14:51:52.581687  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:52.626401  2683 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:52.628062  4541 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:52.538261  4687 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:52.640239  2606 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:52.653437  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.633971  4454 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:52.640326  4333 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:52.635982  4500 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:52.626523  4702 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:52.740574  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:52.663092  4675 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.004s
I20260709 14:51:52.685612  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.799430  2682 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
W20260709 14:51:52.720430  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.695287  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.564873  4703 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:52.801625  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.802299  4451 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.802584  4589 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:52.803031  4589 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:52.802964  4451 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:52.750419  4663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.022s	sys 0.032s
I20260709 14:51:52.750730  4384 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.803759  4703 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:52.687944  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:52.804198  4384 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:52.804775  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 149152us
I20260709 14:51:52.805598  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:52.805713  4451 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:52.806645  4385 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:52.807145  4385 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:52.751919  4659 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.750492  2611 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:51:52.807957  4663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.023s	sys 0.032s
I20260709 14:51:52.753530  2433 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 3 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:52.754377  4648 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:52.807468  4384 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:52.810441  2433 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:52.656996  4458 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:52.815295  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162773us
I20260709 14:51:52.816919  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.817106  4458 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:52.819126  4597 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:52.819738  4597 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:52.820252  4597 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.819363  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:51:52.823199  2433 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 2 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:52.824868  4239 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.825752  2433 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:52.828564  2433 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:52.833287  2433 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 1 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:51:52.835963  2433 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:51:52.845826  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:52.748472  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.851219  2753 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:52.775022  4603 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.776285  4562 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:52.778577  2752 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4. Ops: []
W20260709 14:51:52.746222  4268 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.005s
W20260709 14:51:52.783183  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.785118  4641 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:52.786227  4506 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:52.786931  4316 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:51:52.787410  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.746340  4670 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.910004  2686 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:52.796125  3536 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:52.803484  4589 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:52.908046  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:52.753826  4620 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.807560  4385 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:52.755812  4454 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:52.812322  4484 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:52.812460  4659 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:52.813258  4275 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:51:52.813285  4384 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:52.814033  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:52.628861  4372 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:52.818573  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:52.666540  4681 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:51:52.652175  3537 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:52.754647  4701 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 14:51:52.652606  4658 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:52.873354  2604 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:52.781620  4643 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:52.866647  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123044us
I20260709 14:51:52.889776  2756 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:52.689389  4599 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 14:51:52.799434  4675 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:52.896339  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:52.672936  4577 consensus_queue.cc:1243] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:51:52.663995  4655 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:51:52.908520  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.912130  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:52.905706  4562 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:52.895768  4266 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:52.941766  4570 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.493s	user 0.072s	sys 0.164s
I20260709 14:51:52.944362  2607 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:52.948050  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:52.931802  4384 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:52.956034  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:52.931485  4275 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:51:52.963475  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:52.932922  4506 consensus_queue.cc:1243] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:51:52.968698  2759 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:52.915897  4316 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:52.968272  4636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.040s	sys 0.106s
I20260709 14:51:53.043663  2611 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:52.941133  4586 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:52.932542  4670 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.042160  3541 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
W20260709 14:51:53.115321  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66108.2us
I20260709 14:51:52.998538  4532 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:53.112905  4506 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:53.130075  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73966us
I20260709 14:51:52.936621  4620 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:52.970818  4451 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
W20260709 14:51:53.112743  4636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.041s	sys 0.106s
I20260709 14:51:53.135345  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:53.133845  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:53.138681  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 320187us
W20260709 14:51:53.175081  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 361168us
W20260709 14:51:53.057459  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:53.135090  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:52.975909  4653 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:53.140379  2607 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
W20260709 14:51:53.180992  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 604636us
I20260709 14:51:53.118064  4268 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:53.183437  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:53.043136  4603 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 14:51:53.032024  4570 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.596s	user 0.072s	sys 0.165s
I20260709 14:51:53.186903  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:53.075896  4658 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:51:53.048870  4675 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.200207  2608 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:53.190851  4484 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:53.217223  2680 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:51:53.135083  4641 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 14:51:53.050864  4703 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.157s
I20260709 14:51:53.193349  4653 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.178180  4506 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.210784  4687 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:53.190912  4268 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.194327  4384 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:53.226198  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50535.1us
I20260709 14:51:53.227335  2604 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:53.251627  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:53.260070  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:53.304193  2681 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:51:53.276113  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:53.251240  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:53.319700  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
W20260709 14:51:53.291936  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 108593us
I20260709 14:51:53.322183  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:53.226856  2682 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:53.305155  2684 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:53.259591  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:53.206804  4239 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:53.329867  2679 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:53.225203  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:53.339041  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:53.216920  4702 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:53.275624  4384 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:53.362174  4314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.029s	sys 0.034s
I20260709 14:51:53.274163  4268 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:53.261969  4620 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:53.284371  4316 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:53.380136  2606 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:53.245437  4675 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:53.389166  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 247887us
I20260709 14:51:53.393402  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:53.398772  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:53.222380  4458 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.144s
I20260709 14:51:53.234572  4681 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 14:51:53.346038  4659 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:53.235436  4385 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.257002  4336 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 14:51:53.242273  4266 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 14:51:53.399868  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:53.362929  4637 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:53.382140  4314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.029s	sys 0.035s
I20260709 14:51:53.433668  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:53.452505  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:53.385339  4702 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:53.455302  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:53.233662  4484 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.463069  4711 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:53.464313  4711 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:53.396297  4658 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:53.388119  4636 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:53.475857  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.036s	sys 0.054s
W20260709 14:51:53.477075  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.036s	sys 0.054s
I20260709 14:51:53.389844  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:53.233662  4655 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.384299  4603 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:53.454486  4620 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:53.475059  2751 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.014s	sys 0.077s
I20260709 14:51:53.470441  2683 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:53.448673  4372 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:53.437438  4687 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:53.492695  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:53.494449  2751 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.014s	sys 0.077s
W20260709 14:51:53.477233  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83987.2us
I20260709 14:51:53.453791  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:51:53.491523  4656 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.076s	user 0.011s	sys 0.002s
I20260709 14:51:53.485554  4316 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:53.538034  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:53.455021  4659 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.489557  4675 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:53.496822  4268 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:53.494277  4620 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:53.549952  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:53.554649  4620 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:53.555060  2610 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:53.555773  4599 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:53.557412  4599 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:53.559120  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 537) took 1053 ms. Trace:
I20260709 14:51:53.553643  4369 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.527464  4586 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:53.472200  4702 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:53.542848  4707 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.358s	user 0.001s	sys 0.001s
I20260709 14:51:53.487012  4658 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.559585  3537 rpcz_store.cc:276] 0709 14:51:52.505996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.513639 (+  7643us) service_pool.cc:224] Handling call
0709 14:51:52.623927 (+110288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.955215 (+331288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.558871 (+603656us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400003550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:53.559043 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244438912}
I20260709 14:51:53.525606  4687 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:53.526290  4636 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.611327  4636 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:53.611972  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:53.613304  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:53.614285  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:53.579198  2677 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:51:53.615108  2604 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:53.539932  4316 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:53.485967  4637 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.496275  4603 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:53.565202  2754 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.027s	sys 0.039s
W20260709 14:51:53.647392  2754 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.028s	sys 0.039s
I20260709 14:51:53.562263  4656 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:53.604979  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:53.561534  4268 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:53.579555  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.092s	sys 0.130s
W20260709 14:51:53.653806  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.092s	sys 0.130s
W20260709 14:51:53.654235  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 808470us
I20260709 14:51:53.564563  4670 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:53.574411  2603 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:53.616026  2606 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:53.560756  4506 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:53.656760  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:53.563764  4577 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 14:51:53.647017  4496 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.281s	user 0.015s	sys 0.002s
I20260709 14:51:53.620095  4586 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:53.661556  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:53.617544  4687 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:53.650635  4437 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.999s	user 0.012s	sys 0.003s
I20260709 14:51:53.651338  4369 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:53.654783  4656 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:53.667351  4670 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:53.680716  4656 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:53.675724  4506 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 14:51:53.685710  4562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.022s	sys 0.021s
W20260709 14:51:53.691342  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.024s	sys 0.019s
I20260709 14:51:53.713690   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:51:53.682905  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 287) took 1078 ms. Trace:
I20260709 14:51:53.694732  4586 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:53.714819  4562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.023s	sys 0.021s
W20260709 14:51:53.719805  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.024s	sys 0.019s
I20260709 14:51:53.709929  4437 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:53.717331  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.002s	sys 0.025s
I20260709 14:51:53.700322  4670 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:53.708676  4496 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:53.709165  4702 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:53.765097  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:53.766023  2755 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 14:51:53.720723  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 495614us
I20260709 14:51:53.768564  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:53.770655  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:53.769831  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:53.773296  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:53.767417  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.609s (SENT) [suppressed 13 similar messages]
W20260709 14:51:53.780941  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 411731us
I20260709 14:51:53.762032  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 319) took 1174 ms. Trace:
W20260709 14:51:53.762569  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.003s	sys 0.025s
W20260709 14:51:53.754966  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.016s
I20260709 14:51:53.720991  4484 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:53.760638  4641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.001s	sys 0.024s
I20260709 14:51:53.691468  4316 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.758915  4562 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:53.774452  4712 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:53.830566  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 733) took 1692 ms. Trace:
W20260709 14:51:53.809298  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 466929us
I20260709 14:51:53.781558  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:53.790084  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.009s	sys 0.016s
I20260709 14:51:53.718793  4268 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.774968  4681 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 14:51:53.776024  4532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.002s	sys 0.017s
W20260709 14:51:53.832166  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 378418us
I20260709 14:51:53.790378  4314 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:53.837217  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.10082e+06us
I20260709 14:51:53.782598  2751 rpcz_store.cc:276] 0709 14:51:52.587436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.773068 (+185632us) service_pool.cc:224] Handling call
0709 14:51:52.773600 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.774040 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:51:53.529230 (+755190us) log.cc:844] Serialized 14 byte log entry
0709 14:51:53.760533 (+231303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:53.761575 (+  1042us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:53.761663 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.761771 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:53.761952 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":886632,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":3829,"raft.queue_time_us":38375,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":3399,"thread_start_us":939883,"threads_started":2}
I20260709 14:51:53.831064  3536 rpcz_store.cc:276] 0709 14:51:52.138366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.180275 (+ 41909us) service_pool.cc:224] Handling call
0709 14:51:52.180773 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.923329 (+742556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.828149 (+904820us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:53.830475 (+  2326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278886144}
W20260709 14:51:53.814123  4641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.002s	sys 0.024s
I20260709 14:51:53.855640  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:51:53.862814  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.011s	sys 0.030s
W20260709 14:51:53.863328  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.030s
W20260709 14:51:53.856459  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 534366us
I20260709 14:51:53.865046  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:53.775391  4702 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:53.861480  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 323511us
I20260709 14:51:53.867677  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:53.827508  4507 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:53.828912  4658 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:53.718551  2603 rpcz_store.cc:276] 0709 14:51:52.602651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.622809 (+ 20158us) service_pool.cc:224] Handling call
0709 14:51:53.537073 (+914264us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:53.563531 (+ 26458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:53.666821 (+103290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.681387 (+ 14566us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47115648}
I20260709 14:51:53.830304  4484 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:53.778142  4496 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.780665  4437 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.858680  4637 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:53.868640  4532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.002s	sys 0.020s
I20260709 14:51:53.869076  4663 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:53.859892  4314 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:53.857161  4681 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:53.871135  4570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.016s	sys 0.030s
W20260709 14:51:53.876660  4597 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.056s	user 0.015s	sys 0.002s
I20260709 14:51:53.807083  4586 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:53.890902  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:53.898815  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:53.884248  4603 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:53.874316  4507 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:53.910698  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:53.841061  4712 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:53.901947  4268 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:53.883984  4484 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:53.872146  4702 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:53.896539  4532 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:53.910038  4681 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:53.916085  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 318) took 1365 ms. Trace:
I20260709 14:51:53.902733  4641 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:53.911415  4570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.017s	sys 0.030s
I20260709 14:51:53.899123  4659 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:53.945369  4658 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:53.947359  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54692.7us
I20260709 14:51:53.907392  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 915) took 1322 ms. Trace:
I20260709 14:51:53.911070  4314 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:53.963707  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65011.6us
I20260709 14:51:53.978404  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:53.917425  4316 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:53.944206  4603 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:53.996398  4722 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 14:51:53.997231  4333 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.142s	user 0.015s	sys 0.002s
I20260709 14:51:53.996832  4507 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.997691  4333 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:53.998335  4507 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:53.913169  4597 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.000478  4597 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:54.000600  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129674us
W20260709 14:51:54.002040  4733 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 14:51:54.002746  4735 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 14:51:54.003024  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 262654us
I20260709 14:51:54.003324  4735 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:54.004594  4735 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.005496  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:54.005851  4735 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.006662  4724 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.006958  4717 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:53.958403  4562 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.007289  4724 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.007571  4717 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.009976  4717 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.011732  4643 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.014209  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.015942  4541 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:54.016390  4541 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:54.017318  4541 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.019198  3545 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:54.019445  4532 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.020674  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 539) took 1514 ms. Trace:
I20260709 14:51:54.021037  3545 rpcz_store.cc:276] 0709 14:51:52.506686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.512404 (+  5718us) service_pool.cc:224] Handling call
0709 14:51:52.512696 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:54.019718 (+1507022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:54.020396 (+   678us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54004b5ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:54.020606 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3464100224}
I20260709 14:51:54.018721  4541 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.023455  4637 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:54.024533  4637 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:53.907294  4670 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.027015  4655 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:54.027421  4655 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:53.942696  4484 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:53.940408  4712 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:54.031950  4655 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.036859  4730 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:54.037827  4655 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.037670  4275 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.038483  4750 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:54.039935  4726 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 14:51:54.040328  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 311) took 1540 ms (client timeout 1780 ms). Trace:
I20260709 14:51:54.041118  4239 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:54.041570  4239 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:54.042950  4675 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:54.043231  4702 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.044350  4397 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:54.044534  4724 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.044811  4397 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:53.977428  2605 rpcz_store.cc:276] 0709 14:51:52.611262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.627549 (+ 16287us) service_pool.cc:224] Handling call
0709 14:51:52.741923 (+114374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.783632 (+ 41709us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:51:53.232165 (+448533us) log.cc:844] Serialized 14 byte log entry
0709 14:51:53.862048 (+629883us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:53.871974 (+  9926us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:51:53.876573 (+  4599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.888566 (+ 11993us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:53.903707 (+ 15141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1321,"prepare.queue_time_us":876526,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":61815,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":39860,"thread_start_us":1019802,"threads_started":2,"wal-append.queue_time_us":770745}
I20260709 14:51:54.045878  4397 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.049760  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:54.053399  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 286022us
W20260709 14:51:54.040809  2759 rpcz_store.cc:269] 0709 14:51:52.500016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.653060 (+153044us) service_pool.cc:224] Handling call
0709 14:51:54.040076 (+1387016us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:54.040250 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717887872}
I20260709 14:51:53.914150  4663 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.051116  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 908) took 1450 ms. Trace:
W20260709 14:51:54.059486  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 191922us
I20260709 14:51:54.060521  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:53.955178  2754 rpcz_store.cc:276] 0709 14:51:52.543692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.756323 (+212631us) service_pool.cc:224] Handling call
0709 14:51:52.756740 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.757033 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:51:53.647948 (+890915us) log.cc:844] Serialized 14 byte log entry
0709 14:51:53.897503 (+249555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:53.897577 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:51:53.905982 (+  8405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.906085 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:53.909061 (+  2976us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19153,"prepare.queue_time_us":817560,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":19475,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8531,"thread_start_us":1100974,"threads_started":2}
I20260709 14:51:53.918116  4731 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 14:51:53.987296  4734 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:51:53.996398  4570 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:54.054275  4384 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.004s
I20260709 14:51:54.033735  4723 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.037829  4730 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 14:51:54.075163  4589 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.148s	user 0.019s	sys 0.001s
I20260709 14:51:54.058485  4703 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:51:54.028700  4641 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.059768  2609 rpcz_store.cc:276] 0709 14:51:52.600822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.602064 (+  1242us) service_pool.cc:224] Handling call
0709 14:51:52.602359 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:52.602726 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:52.993139 (+390413us) log.cc:844] Serialized 14 byte log entry
0709 14:51:54.049543 (+1056404us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b18003bf4a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a
0709 14:51:54.049778 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:54.049837 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:54.049888 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:54.049970 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:54.051024 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1045288,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":3485,"spinlock_wait_cycles":16780032,"thread_start_us":1125160,"threads_started":2,"wal-append.queue_time_us":848516}
I20260709 14:51:54.126085  4719 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.140003  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:54.186697  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 208391us
I20260709 14:51:54.187167  2604 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:54.140774  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.141525  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:54.142048  4714 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.367s	user 0.000s	sys 0.002s
W20260709 14:51:54.142765  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.083146  4597 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 14:51:54.143203  4741 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:54.143837  4384 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.143873  4496 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.139719  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.042487  4239 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:54.157891  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.119405  4720 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:54.084533  4500 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.154554  4728 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.139655  4727 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.178368  4750 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.124961  4734 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.188354  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:54.124341  4599 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.137048  4275 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:54.134817  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.152459  2753 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:54.203415  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.175029  4675 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.181569  4506 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.236456  4506 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.188220  4589 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.238902  4506 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.184664  4702 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.197903  4724 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.231696  4750 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.239771  4702 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.239845  4724 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.241078  4589 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.241533  4750 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.241849  4702 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.242184  4724 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.197747  4562 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.217867  4500 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.242955  4562 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.132737  4723 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.245370  4500 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.215474  4384 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.247452  4723 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.196581  4719 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.249487  4384 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.234579  4740 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.234894  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.252138  2609 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.251827  4740 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.222074  4748 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 14:51:54.253237  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.254110  2609 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.254225  4740 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.233033  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.132719  4570 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.251979  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 916) took 1606 ms. Trace:
I20260709 14:51:54.250559  4719 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.235471  4675 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.241832  4589 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:54.210448  4496 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:54.200716  4736 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.222218  4437 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.256255  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.255285  2604 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:54.257577  2607 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:51:54.269482  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 197878us
W20260709 14:51:54.232023  4687 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.007s
I20260709 14:51:54.225989  4734 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:54.274469  2611 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:54.255264  2612 rpcz_store.cc:276] 0709 14:51:52.645960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.655175 (+  9215us) service_pool.cc:224] Handling call
0709 14:51:52.741783 (+ 86608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:52.758813 (+ 17030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:52.758890 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:52.759607 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 14:51:53.490360 (+730753us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:53.490418 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:53.491442 (+  1024us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:54.141152 (+649710us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:51:54.251638 (+110486us) raft_consensus.cc:1685] finished
0709 14:51:54.251661 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:54.251909 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7098880,"thread_start_us":730199,"threads_started":1,"wal-append.queue_time_us":935168}
I20260709 14:51:54.223302  4728 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:54.337642  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 149356us
I20260709 14:51:54.356489  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:54.359705  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78574.4us
W20260709 14:51:54.356096  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 154446us
I20260709 14:51:54.363523  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.364276  3545 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:54.372816  4641 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.373669  4641 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:54.374399  4641 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:54.374884  4743 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 14:51:54.375892  4746 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.376618  4746 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.377450  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:54.378223  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.380312  4675 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.381081  4675 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:54.381577  4675 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:54.381758  4648 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.282655  4747 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.383227  4747 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.371981  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.378448  3538 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.335460  4239 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:54.374768  4458 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 14:51:54.346915  4656 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.003s
I20260709 14:51:54.279453  4687 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.371366  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.335158  4702 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.399068  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.400015  3545 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.400969  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:54.256956  4599 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.370304  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:54.336961  4724 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.335947  4753 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:54.406286  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.407174  3540 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:54.385748  4747 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:54.402958  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 132321us
I20260709 14:51:54.360524  4740 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:54.404706  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.359628  4741 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.410496  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.410666  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.372020  2677 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:54.226342  4727 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.283738  4496 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.415313  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.386638  4720 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.386373  4717 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.409935  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.456996  3536 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:54.457015  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.394466  4500 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.392690  4458 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:54.458066  3538 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:54.455956  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.382364  4648 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:54.429100  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59061.1us
I20260709 14:51:54.458372  3545 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.452270  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.452729  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.420192  2754 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:54.455570  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.422519  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.424052  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.463352  4437 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.463869  4336 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:54.464823  4728 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.466358  4756 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.471016  4759 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:54.471405  4720 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.409670  4589 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.474530  4744 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:54.452745  4369 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.770s	user 0.014s	sys 0.000s
I20260709 14:51:54.474817  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.453543  4740 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.479736  2751 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:54.482478  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:54.387295  4723 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 14:51:54.485950  3541 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:51:54.480697  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.486941  3546 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:54.420849  4663 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.496559  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.510970  4759 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:54.506515  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.382620  4675 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.336162  4736 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.543860  4597 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.544728  4736 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.405310  4656 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.545589  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.486308  4603 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:54.546960  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.414614  4641 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.547655  3536 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
W20260709 14:51:54.547804  4507 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 14:51:54.548669  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.549296  3536 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 1.
I20260709 14:51:54.549383  4641 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.550056  4641 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:54.550432  4641 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:54.420945  4724 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.551041  4333 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.552286  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.553010  2682 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.554066  2610 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.477224  4500 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:54.522078  4620 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.967s	user 0.012s	sys 0.004s
I20260709 14:51:54.412856  4702 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.485486  4570 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.498446  4336 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:54.496846  4437 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.574247  4437 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.554397  4759 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:54.483783  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:54.478916  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.577397  4702 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.486174  4717 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.538865  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.523032  4586 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.599329  4586 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.600426  4720 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.601755  4586 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:54.541623  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59216.8us
I20260709 14:51:54.603833  4496 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.604291  3545 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:54.542443  4659 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:51:54.544477  4597 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.588120  3541 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:51:54.605751  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:54.606503  4597 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.607064  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.607892  3541 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:54.459439  4741 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.550678  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.611299  3536 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:54.519054  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.514595  4730 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:54.527314  4746 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.554795  4384 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.538617  4723 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.538545  4599 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.628911  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.550086  4385 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.563911  4740 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.568100  4687 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.530686  4744 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.568957  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.502020  4712 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.426517  4753 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.522460  4589 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:54.571483  4681 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.001s
I20260709 14:51:54.534181  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.537786  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.539505  4721 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.512830  4756 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.592741  4717 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.592970  4648 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:54.594081  4314 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.002s
I20260709 14:51:54.537827  4458 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:54.595242  4658 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 14:51:54.553612  4729 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.608660  3547 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 1.
I20260709 14:51:54.610144  2686 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:51:54.612764  3542 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.613603  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.613543  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.670228  3536 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.521867  2606 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:54.513067  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 915) took 1603 ms. Trace:
W20260709 14:51:54.616832  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.552529  4724 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.618063  4730 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.618702  4727 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.620715  4741 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.621008  2682 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
W20260709 14:51:54.625239  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.671906  2753 rpcz_store.cc:276] 0709 14:51:52.909554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.921624 (+ 12070us) service_pool.cc:224] Handling call
0709 14:51:54.512802 (+1591178us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:54.512969 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2617467776}
I20260709 14:51:54.560989  4656 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.632047  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.588368  4369 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.604961  4659 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:51:54.635897  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.635968  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.590265  4641 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:54.616302  4268 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.662s	user 0.002s	sys 0.011s
I20260709 14:51:54.723313  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.640844  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:54.637720  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.579514  4620 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.626514  4316 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.646876  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.651338  2681 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:51:54.654623  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 194598us
I20260709 14:51:54.630228  4384 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.632845  4723 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.679448  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:54.679862  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.680521  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.700135  4659 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.703414  4369 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.704475  4744 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:54.688400  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.707743  4500 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:54.708318  4484 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.003s
I20260709 14:51:54.711694  4753 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.711998  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.681349  4658 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:54.712646  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.754787  2608 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:54.716357  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.755831  3538 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.718021  2679 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:54.691210  4458 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.719341  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:54.720285  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:54.758203  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.680966  4729 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.721577  4756 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.645670  4681 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.728277  3540 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:54.669651  4719 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:54.732600  2604 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:54.685213  4735 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 14:51:54.737768  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.768878  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.769078  2680 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
W20260709 14:51:54.740171  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60839.8us
I20260709 14:51:54.750748  4637 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:54.721467  4653 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.743307  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.749619  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.723706  4636 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.777935  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.778699  3538 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.778580  4636 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.779230  4454 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:54.779765  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.781028  4636 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:54.751911  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.781910  3536 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:54.714443  4589 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.674304  4314 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.787362  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.788166  2680 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:54.709470  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.789261  2610 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:54.790232  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.790817  2610 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:54.777956  2683 raft_consensus.cc:2376] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8. Re-sending same reply.
I20260709 14:51:54.753418  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.757545  3541 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:54.792861  3536 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:54.793617  2684 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:54.794477  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:54.758797  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:54.795753  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.646297  4336 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.703418  4712 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:54.754882  3542 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
W20260709 14:51:54.752266  4541 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.002s
I20260709 14:51:54.739843  4643 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.763929  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.736660  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:54.771561  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.771517  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.657298  4721 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.797623  3540 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:54.762470  2682 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:54.815937  3536 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:54.825760  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.777835  4659 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:54.751263  4316 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.835913  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.829669  3541 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:54.836784  2682 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.755759  4734 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.831933  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.793675  4637 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:54.833485  2754 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:54.753808  4484 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:54.763926  2605 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:54.848323  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:54.836763  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 97651us
I20260709 14:51:54.822813  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:54.850859  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:51:54.840178  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:54.838291  2752 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:51:54.844269  2685 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:54.844822  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.790983  4719 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:54.853052  4741 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:54.853653  4741 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:54.854120  4741 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:54.854182  4719 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.801911  4730 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:54.847035  2751 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:54.855155  4730 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.855163  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.857162  2679 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:54.861085  4730 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.851016  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.862054  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:54.862874  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.863788  4648 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:54.862810  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.864418  4648 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:54.863996  2604 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.865062  4648 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.822957  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:54.849115  2756 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:54.869874  2679 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.871425  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:54.872681  4385 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:54.873210  4385 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:54.873649  4385 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.801290  4500 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 14:51:54.879025  2759 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:54.881984  2755 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:54.884191  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 153985us
I20260709 14:51:54.884442  4562 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:54.885423  2677 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:54.887593  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.887936  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.890671  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.844285  4336 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:54.844988  4637 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:54.875429  4741 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:54.894603  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 71787.2us
I20260709 14:51:54.876570  2678 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:54.918130  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.895884  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:54.801509  4759 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:54.901185  4776 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.904340  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:54.892189  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.848613  4656 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:54.893999  2612 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.907781  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:54.975879  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:54.977310  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:54.978080  2677 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:51:54.908253  4773 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:51:54.909298  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.820623  4643 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.912284  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:54.878976  4756 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:54.895074  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:54.989223  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 108580us
I20260709 14:51:54.921954  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:54.917083  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.928148  2604 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:54.936394  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:54.936826  4771 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 14:51:54.916477  4562 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:54.979101  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.904084  4636 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:54.981118  2610 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:54.979144  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:54.998046  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:54.992729  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79214.7us
I20260709 14:51:54.993959  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:54.999007  2607 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:54.998960  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:54.926880  4275 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:54.984673  4732 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:51:54.981814  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54198.7us
I20260709 14:51:55.005091  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.003918  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:55.005944  2682 raft_consensus.cc:2368] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:55.007697  4369 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.008334  4748 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.009065  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:54.994977  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:54.992650  4756 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.002519  4562 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:54.993752  4702 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:55.000890  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:54.991676  4656 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:55.016139  2604 raft_consensus.cc:2368] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:51:55.017776  3541 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:54.983274  4776 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.027647  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.028831  2680 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:51:55.031970  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.019201  4756 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:55.003068  4751 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:51:55.003904  4275 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:55.008862  4748 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:55.046890  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46131.2us
I20260709 14:51:54.996999  4658 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 14:51:55.007741  4681 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.010332  4636 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:55.019532  4702 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.058934  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:55.057624  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 149214us
I20260709 14:51:55.044701  4751 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.046463  4275 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.058771  4702 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:55.087126  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83321.3us
I20260709 14:51:55.075723  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.095007  2682 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:55.094798  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.060105  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:55.080521  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 158630us
I20260709 14:51:55.060333  4316 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.100977  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.099956  2609 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:55.103974  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.079679  4751 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:51:55.109133  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.084324  4562 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.102171  4780 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:55.124464  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:55.120371  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.125375  2683 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:55.112637  2682 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:55.083578  4776 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.107200  2680 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:55.103351  3537 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.119342  2686 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:55.120409  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.121321  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:55.122733  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:55.110998  4774 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.096s	user 0.006s	sys 0.000s
I20260709 14:51:55.108274  4316 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:55.139652  4769 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:55.128962  4451 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.143219  4454 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:55.147253  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:55.153039  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.154467  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:55.155048  2752 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.123311  4643 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:55.136796  4655 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.098s	user 0.017s	sys 0.000s
I20260709 14:51:55.124028  4780 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.162469  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.164077  4776 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:55.161111  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.135701  4781 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:55.168105  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.170727  2680 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:51:55.155261  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 107415us
I20260709 14:51:55.158439  4316 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:55.132035  4641 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.176087  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:55.176841  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.111914  4656 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.175309  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:55.183668  2683 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:51:55.169252  4562 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:55.173933  4721 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:55.197976  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.171103  4780 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.206269  2752 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.197052  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.207368  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:55.207590  4562 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:55.190441  4769 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:55.208272  4769 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:55.208702  4769 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.209265  4748 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:55.174921  4643 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:55.180645  4589 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:55.167541  4451 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.180915  4655 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:55.187608  4782 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:55.188707  4316 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:55.225594  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.157490  4636 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.238116  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:55.200985  4781 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:55.258133  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:55.258675  4781 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.255684  4369 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:55.256577  4641 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.208284  4562 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:55.229684  4589 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.229976  4655 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:55.227699  4643 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:55.259378  3545 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.264546  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.211000  4721 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:55.273046  4753 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.198700  4681 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 14:51:55.229614  4776 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:55.282994  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:55.285713  2682 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:55.243520  4782 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:55.296936  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 92099.9us
I20260709 14:51:55.305212  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:55.278990  4781 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.281934  4653 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:55.276228  4784 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 14:51:55.289263  4589 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.293648  4655 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:55.315830  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.283169  4369 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.336885  2757 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:55.339246  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.396325  2682 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.301638  4786 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:51:55.341197  4769 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.348869  4648 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.350421  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.401386  2683 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:55.359504  4562 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.361214  3537 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:55.359524  4372 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:55.364313  4275 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.309798  4782 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.301595  4753 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.294045  4721 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:55.302165  4776 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:55.399843  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 246933us
I20260709 14:51:55.313362  4658 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:55.407053  2680 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:55.322538  4454 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:55.418541  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:55.425060  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.425537  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.426170  3541 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:55.426328  3542 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:55.432173  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.421257  4776 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:55.421191  4721 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:55.385105  4653 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.405441  4372 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:55.406066  4786 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:55.457732  2575 leader_election.cc:400] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:55.458443  2575 leader_election.cc:403] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:55.463826  4702 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.467152  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.421432  4658 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:55.469193  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 322057us
I20260709 14:51:55.472335  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.473542  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:51:55.474381  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.435807  4757 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:55.443334  4748 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:55.419628  4756 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.452526  4454 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:55.480288  4658 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.418753  4643 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.502143  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.483827  4757 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:55.484762  4659 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 5
W20260709 14:51:55.521512  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 205805us
W20260709 14:51:55.526353  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:55.527222  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.524612  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.529816  3536 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:55.535158  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.536485  2682 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
W20260709 14:51:55.525753  2677 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.007s
W20260709 14:51:55.540684  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65944.1us
I20260709 14:51:55.541776  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.543358  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.517210  4788 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:55.547606  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.553380  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:55.549801  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 66838us
I20260709 14:51:55.531654  4789 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:55.560662  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.565639  4369 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.543145  2677 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:55.568158  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.549536  4790 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.588869  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:55.592703  4793 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 14:51:55.593705  4787 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 14:51:55.559952  4785 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 14:51:55.595520  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52297.2us
I20260709 14:51:55.606452  2759 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:51:55.609766  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.560801  4789 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:55.611281  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.570003  4795 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:55.590272  4275 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.613788  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:55.614248  4791 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 14:51:55.615228  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.615778  4769 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:55.616079  2760 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.617677  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:55.617902  4792 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 14:51:55.618395  4648 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.619421  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:55.620265  4385 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
I20260709 14:51:55.620816  4385 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.623170  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.623512  4789 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.623944  3542 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:55.624954  4800 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:55.540011  4372 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.627357  4372 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:55.627696  4794 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:51:55.614326  4702 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.617967  4589 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.590615  4454 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.632176  4316 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.583003  4748 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.632666  4385 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.638141  4748 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.638978  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.641721  2682 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:51:55.611568  3545 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:55.643137  3536 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:55.641830  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:55.653790  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:55.658221  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.660045  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.661259  2760 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:55.659703  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.665951  2682 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 1.
I20260709 14:51:55.667541  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.668529  2682 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:55.669827  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.595901  4798 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.551298  4757 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.606706  4785 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.612578  4788 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:51:55.677275  4751 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.554s	user 0.002s	sys 0.013s
I20260709 14:51:55.557901  4786 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:55.683218  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 134780us
I20260709 14:51:55.607466  4801 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:55.682062  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.683036  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.654099  4316 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.711033  3542 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:55.684648  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:55.722568  4655 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.710508  4801 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:55.731561  4801 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.732111  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:55.727584  4641 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:55.734133  4641 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:55.739399  3545 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:55.691646  4757 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:55.682013  4795 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:55.692623  4788 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:55.710942  4753 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.607450  4790 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.712877  4751 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.683470  4798 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:55.817560  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.761648  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:51:55.908367  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 171771us
I20260709 14:51:55.837004  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 779) took 1040 ms. Trace:
I20260709 14:51:55.867210  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 930) took 2235 ms. Trace:
I20260709 14:51:55.911633  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.909855  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.911741  2608 rpcz_store.cc:276] 0709 14:51:53.613325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:53.613476 (+   151us) service_pool.cc:224] Handling call
0709 14:51:53.613823 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:54.759894 (+1146071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:55.835342 (+1075448us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:55.848267 (+ 12925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2623501184}
I20260709 14:51:55.913239  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.912153  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.914108  4795 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.916852  4798 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:55.857144  4780 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.002s
I20260709 14:51:55.917956  4780 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.752054  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.914175  4788 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:55.811946  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 125679us
I20260709 14:51:55.920914  4780 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:55.919430  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 167536us
W20260709 14:51:55.871852  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:55.915818  4454 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.898681  4369 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.918243  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:55.919302  2752 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.682992  4785 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:55.924415  4454 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:55.911315  2679 rpcz_store.cc:276] 0709 14:51:54.796992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.825177 (+ 28185us) service_pool.cc:224] Handling call
0709 14:51:55.836709 (+1011532us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b5400269b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:55.836888 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10232192}
I20260709 14:51:55.926534  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:55.749763  4653 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.923249  4369 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.948426  4790 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.719514  4756 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.949707  4756 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:51:55.734915  4748 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:55.951035  2679 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:55.950302  4369 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.927158  4796 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.952913  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.952939  4796 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.955610  4796 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.927917  4454 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.927965  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:55.959373  4454 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:55.962015  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.985962  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.986109  2753 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:55.987968  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.988144  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.989007  2753 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:55.990371  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.991967  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:55.993325  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:55.994058  2753 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:55.995131  4369 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:55.995996  4369 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:55.995230  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.000247  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:55.932077  2758 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:55.900445  4681 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.915194  4753 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:55.945783  2760 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:55.905336  2603 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:55.810788  4384 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.936410  4562 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:55.947868  4314 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.948009  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:56.028586  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 80658.8us
I20260709 14:51:55.950230  4756 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:55.909802  2682 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.900741  4675 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:55.962738  4397 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:55.963024  4451 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.964890  4785 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:55.965554  4751 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.967537  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:56.040786  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73344us
I20260709 14:51:55.943540  4655 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:55.970155  4790 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:51:56.045279  4369 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.045424  4562 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:55.971168  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 860) took 1461 ms. Trace:
I20260709 14:51:55.980245  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.046803  4562 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.048333  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.048632  4785 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:55.982705  4734 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:55.929944  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:55.984265  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 871) took 1237 ms. Trace:
I20260709 14:51:56.049414  2686 rpcz_store.cc:276] 0709 14:51:54.746692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.756041 (+  9349us) service_pool.cc:224] Handling call
0709 14:51:55.983974 (+1227933us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:55.984168 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508449024}
W20260709 14:51:55.853820  4637 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.926s	user 0.015s	sys 0.000s
I20260709 14:51:56.039919  4675 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.053893  4675 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:56.054080  4658 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.001s
I20260709 14:51:56.025394  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.010532  4753 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:51:55.951443  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 928) took 2378 ms (client timeout 1943 ms). Trace:
I20260709 14:51:56.014087  4500 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 14:51:56.058076  2611 rpcz_store.cc:269] 0709 14:51:53.572888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:53.573084 (+   196us) service_pool.cc:224] Handling call
0709 14:51:55.951200 (+2378116us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:55.951343 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097471744}
I20260709 14:51:56.030289  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.032956  2683 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:56.037310  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.035597  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:55.937427  4804 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:56.042663  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:56.066972  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 137115us
I20260709 14:51:56.043717  4731 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:56.040026  4797 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.075770  4797 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.076272  4797 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:56.076336  4775 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.087s	user 0.000s	sys 0.002s
W20260709 14:51:56.077180  4776 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.004s
I20260709 14:51:56.077622  4776 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.078282  4735 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:56.079289  2755 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:56.005832  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:56.080549  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 74851.9us
I20260709 14:51:56.051364  4637 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.054957  2758 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:56.083249  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:56.065322  4753 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:56.050683  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.054574  4658 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.085119  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.086640  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.070701  4785 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.065761  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.067845  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.089198  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.068099  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.090848  2760 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:56.091528  2683 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:56.066991  2682 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:51:56.067310  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151524us
I20260709 14:51:56.069737  4653 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:56.066941  4804 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.065456  4641 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.014609  4681 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.072746  4620 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:56.073212  4721 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.003s
W20260709 14:51:56.073479  3547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 753) took 1675 ms (client timeout 1698 ms). Trace:
W20260709 14:51:56.079344  4781 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.002s
I20260709 14:51:56.102603  4266 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:56.111461  4484 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.111325  4781 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.112140  4815 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:56.112881  4681 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.112977  4741 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.054862  2751 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:56.113616  4781 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.113812  2677 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:56.115164  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.082453  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.116508  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.046285  2684 rpcz_store.cc:276] 0709 14:51:54.510442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.713968 (+203526us) service_pool.cc:224] Handling call
0709 14:51:54.714300 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:54.794067 (+ 79767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:55.970682 (+1176615us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:55.971065 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181930240}
W20260709 14:51:56.106417  3547 rpcz_store.cc:269] 0709 14:51:54.398247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.409533 (+ 11286us) service_pool.cc:224] Handling call
0709 14:51:56.073249 (+1663716us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.073379 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343302912}
I20260709 14:51:56.118937  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.088317  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.121873  2758 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:51:56.089673  4314 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:56.085635  4369 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:56.103147  4804 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:56.095643  4734 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:51:56.095920  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.096992  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.084026  4675 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.082664  4756 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:56.098268  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:56.101225  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.083844  4731 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:56.097211  4653 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:56.105252  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.108428  4641 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:56.108530  4721 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:56.110350  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.111605  4266 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:56.149036  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.149878  2684 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.149619  4266 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.010541  4577 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:56.131811  4681 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:51:56.129164  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.129737  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.089766  4785 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.186324  4785 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.144259  4653 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:56.193758  4801 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.140192  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.195899  4637 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.128134  4798 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:56.151695  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.102381  4795 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:56.151203  4675 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.131850  4781 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.197991  4795 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:56.199681  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 755) took 1800 ms. Trace:
I20260709 14:51:56.199826  4795 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:56.200194  3543 rpcz_store.cc:276] 0709 14:51:54.398960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.399088 (+   128us) service_pool.cc:224] Handling call
0709 14:51:54.478952 (+ 79864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:54.577398 (+ 98446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:54.577432 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:54.577994 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 14:51:54.662743 (+ 84749us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:54.662783 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.199164 (+1536381us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:56.199231 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:56.199311 (+    80us) raft_consensus.cc:1685] finished
0709 14:51:56.199326 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:56.199609 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1685,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":25585,"spinlock_wait_cycles":224699776,"thread_start_us":84233,"threads_started":1,"wal-append.queue_time_us":181918,"wal-append.run_cpu_time_us":3601,"wal-append.run_wall_time_us":1333280}
I20260709 14:51:56.112828  4815 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.195901  4275 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:56.203850  4275 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.195932  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.204932  2683 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:56.205354  4815 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.197005  4798 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.205838  4675 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:56.206403  4675 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.207473  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 839) took 1321 ms. Trace:
I20260709 14:51:56.207841  4798 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.207983  2756 rpcz_store.cc:276] 0709 14:51:54.886365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.902193 (+ 15828us) service_pool.cc:224] Handling call
0709 14:51:56.207252 (+1305059us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.207399 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393654656}
I20260709 14:51:56.208761  4776 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:56.136960  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.209621  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.210453  2756 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:56.198230  4781 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:51:56.212401  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 916) took 2126 ms (client timeout 1698 ms). Trace:
W20260709 14:51:56.026196  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 373290us
W20260709 14:51:56.212939  2603 rpcz_store.cc:269] 0709 14:51:54.086182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.121771 (+ 35589us) service_pool.cc:224] Handling call
0709 14:51:56.212162 (+2090391us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.212313 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3938468224}
I20260709 14:51:56.198905  4721 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.195385  4731 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.165932  2751 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:56.215950  2606 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:51:56.199146  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.217249  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.217881  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 921) took 1721 ms. Trace:
I20260709 14:51:56.212062  4781 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.194167  4643 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:56.218348  2606 rpcz_store.cc:276] 0709 14:51:54.496405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.682266 (+185861us) service_pool.cc:224] Handling call
0709 14:51:56.217616 (+1535350us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540016dfd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.217787 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3429567744}
W20260709 14:51:56.219170  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.219072  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 840) took 1332 ms. Trace:
I20260709 14:51:56.204875  4275 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.218688  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.220829  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 307) took 1078 ms. Trace:
I20260709 14:51:56.241926  2683 rpcz_store.cc:276] 0709 14:51:55.143997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.195568 (+1051571us) service_pool.cc:224] Handling call
0709 14:51:56.220758 (+ 25190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.219648  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.145057  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.241966  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:56.154054  4809 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:51:56.243422  2758 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:56.223018  2682 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:56.154080  4782 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.731s	user 0.013s	sys 0.000s
I20260709 14:51:56.261011  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.240178  2757 rpcz_store.cc:276] 0709 14:51:54.886715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.890368 (+  3653us) service_pool.cc:224] Handling call
0709 14:51:56.206686 (+1316318us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540040e2d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.218985 (+ 12299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006732544}
I20260709 14:51:56.268251  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:56.225059  4659 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.010s
I20260709 14:51:56.205255  4729 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.272132  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:56.271214  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:56.244371  4789 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.013s
I20260709 14:51:56.271057  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.277070  2751 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:51:56.277110  2682 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:56.277765  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.198448  4723 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.275771  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.223569  4711 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:56.273736  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 172591us
I20260709 14:51:56.244621  4397 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:56.215723  4776 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 14:51:56.244185  4798 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:56.291079  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.283398  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:56.292096  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:56.288892  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.267112  4620 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:56.273599  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.291086  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:56.291831  2726 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.609s (SENT)
I20260709 14:51:56.269138  4734 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:56.303467  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.275550  4659 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5.
I20260709 14:51:56.293140  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.275307  4275 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:56.309715  4637 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:56.310297  4637 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:56.310730  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.311554  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.311350  4637 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.312281  2677 raft_consensus.cc:2376] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5. Re-sending same reply.
I20260709 14:51:56.316208  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.318315  4570 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:56.307262  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 208882us
I20260709 14:51:56.319178  4719 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.319684  4719 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.320103  4719 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.321132  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:56.322129  4810 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.472s	user 0.002s	sys 0.000s
I20260709 14:51:56.322296  4815 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.323097  4815 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:56.275207  4741 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:56.328694  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.321460  4802 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:56.246073  4577 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.342597  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.324294  4815 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.326388  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.335820  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.313405  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.324321  4735 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:56.325295  4729 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:56.268664  4782 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.285321  4789 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.327075  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.356002  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 844) took 1467 ms. Trace:
W20260709 14:51:56.340268  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 938) took 1601 ms (client timeout 1609 ms). Trace:
W20260709 14:51:56.364908  2612 rpcz_store.cc:269] 0709 14:51:54.738784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.750932 (+ 12148us) service_pool.cc:224] Handling call
0709 14:51:56.339891 (+1588959us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b54002fe4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.340164 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207461632}
I20260709 14:51:56.335443  4723 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.348953  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.329248  4620 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.392653  4782 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:56.393707  4620 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.397215  4723 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.397887  4723 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.359081  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.398726  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:56.399847  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.400537  2679 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:51:56.345115  4798 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:51:56.367841  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:56.360234  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49616us
I20260709 14:51:56.375814  4653 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.378159  4681 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.346488  4802 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.379849  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.363027  4314 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.411274  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.386430  4801 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.387838  4797 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.388981  4804 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.390328  4822 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:56.407433  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80480us
I20260709 14:51:56.369850  4577 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.303200  4397 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.364794  2752 rpcz_store.cc:276] 0709 14:51:54.888155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:55.207055 (+318900us) service_pool.cc:224] Handling call
0709 14:51:56.355758 (+1148703us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54000e6d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.355922 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635272448}
I20260709 14:51:56.294636  4711 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:56.411867  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.331928  4734 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.445391  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 851) took 2075 ms. Trace:
I20260709 14:51:56.361121  4729 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.455858  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:56.385886  4757 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.000s
I20260709 14:51:56.429085  2751 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:51:56.460361  2679 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:51:56.461251  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.462162  2751 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:56.464565  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:56.466598  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.364970  4369 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.361819  4735 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.343292  4570 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.411777  4681 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.446637  2684 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:56.412009  4756 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.482528  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.459431  4729 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:56.425657  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.487180  2760 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:56.456760  2752 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.445811  2685 rpcz_store.cc:276] 0709 14:51:54.370279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.370437 (+   158us) service_pool.cc:224] Handling call
0709 14:51:54.370873 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:54.852140 (+481267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.445074 (+1592934us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540022d154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:56.445319 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969253632}
I20260709 14:51:56.468123  2682 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:56.469465  2751 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:56.410107  4753 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.438508  4798 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:56.469450  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 101510us
I20260709 14:51:56.470465  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.484072  2754 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:56.455008  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.480911  4735 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:56.489058  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 931) took 1592 ms. Trace:
I20260709 14:51:56.494227  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:56.495119  2679 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:56.497359  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.497650  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 791) took 1342 ms. Trace:
I20260709 14:51:56.498030  2679 rpcz_store.cc:276] 0709 14:51:55.155571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.469968 (+1314397us) service_pool.cc:224] Handling call
0709 14:51:56.497583 (+ 27615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.498955  2751 rpcz_store.cc:276] 0709 14:51:54.896318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.468536 (+1572218us) service_pool.cc:224] Handling call
0709 14:51:56.469027 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.488755 (+ 19728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.488998 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.498119  2684 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:56.438491  4785 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.508227  2679 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:51:56.508526  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:56.509202  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 780) took 1712 ms. Trace:
I20260709 14:51:56.509632  2679 rpcz_store.cc:276] 0709 14:51:54.797279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.507501 (+1710222us) service_pool.cc:224] Handling call
0709 14:51:56.507966 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.508879 (+   913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.509141 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.511456  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.482127  4577 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.483871  4681 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.514264  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:56.444979  4822 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.485096  4801 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:51:56.527211  4756 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:56.527509  4828 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:56.527746  4801 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.528172  4756 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.528223  4828 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:56.528894  4828 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:56.529239  4756 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.530102  4655 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:56.480116  4675 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.468648  4719 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.515445  4757 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:56.532944  4675 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.534222  2760 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:56.535135  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 340) took 1030 ms. Trace:
I20260709 14:51:56.536127  2756 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:56.536693  2685 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:51:56.537640  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 783) took 1712 ms. Trace:
W20260709 14:51:56.537987  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103980us
I20260709 14:51:56.520864  4802 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.521113  2686 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:51:56.522403  3543 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:51:56.515306  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:56.523260  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:56.545548  3545 raft_consensus.cc:2368] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:56.529024  4801 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.483559  4796 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.539885  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:56.555279  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 857) took 2134 ms (client timeout 1841 ms). Trace:
I20260709 14:51:56.537979  2685 rpcz_store.cc:276] 0709 14:51:54.825565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.513009 (+1687444us) service_pool.cc:224] Handling call
0709 14:51:56.536418 (+ 23409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.537298 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.537577 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.535503  2760 rpcz_store.cc:276] 0709 14:51:55.504920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.522446 (+1017526us) service_pool.cc:224] Handling call
0709 14:51:56.534028 (+ 11582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.534792 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.535066 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.544428  2757 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:56.498703  4798 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:56.558266  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:56.556043  2681 rpcz_store.cc:269] 0709 14:51:54.411121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:54.411265 (+   144us) service_pool.cc:224] Handling call
0709 14:51:56.545572 (+2134307us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400445a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.545783 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387794304}
I20260709 14:51:56.520910  4753 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.521934  4577 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:56.571676  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.579521  2680 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.582026  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:56.540968  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.585376  2684 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:56.592914  2680 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:56.540689  4795 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.596618  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.566282  4802 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.567399  4798 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:56.561975  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:56.565834  4663 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.598881  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.600468  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 287) took 1631 ms. Trace:
I20260709 14:51:56.598881  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.610975  2760 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:56.607834  2681 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:56.587531  4577 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.571081  4796 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:56.604842  4637 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:56.601126  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:56.604516  4802 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:51:56.598541  4314 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:56.609977  4663 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.596846  4562 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 14:51:56.611953  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.664700  2680 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:56.655217  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.613215  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 292) took 1530 ms. Trace:
I20260709 14:51:56.665786  2760 raft_consensus.cc:2376] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4. Re-sending same reply.
I20260709 14:51:56.665747  2681 rpcz_store.cc:276] 0709 14:51:55.082466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.596519 (+1514053us) service_pool.cc:224] Handling call
0709 14:51:56.607409 (+ 10890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.612822 (+  5413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.613137 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.613397  2684 rpcz_store.cc:276] 0709 14:51:54.956089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.582933 (+1626844us) service_pool.cc:224] Handling call
0709 14:51:56.583336 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.587041 (+  3705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.587367 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.672736  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:56.673385  2680 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:56.662122  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 796) took 1184 ms. Trace:
I20260709 14:51:56.675146  2677 rpcz_store.cc:276] 0709 14:51:55.474875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.498398 (+1023523us) service_pool.cc:224] Handling call
0709 14:51:56.658945 (+160547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.675823  4795 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:56.617188  4789 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.659581  4802 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:56.685393  2681 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:56.686244  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 296) took 1546 ms. Trace:
I20260709 14:51:56.660161  4753 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:56.663290  4637 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:56.683003  2677 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:56.719627  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 310) took 1569 ms. Trace:
I20260709 14:51:56.720095  2677 rpcz_store.cc:276] 0709 14:51:55.150002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.676042 (+1526040us) service_pool.cc:224] Handling call
0709 14:51:56.676408 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.719246 (+ 42838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.719531 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.684008  4314 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.724118  4681 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.686685  2681 rpcz_store.cc:276] 0709 14:51:55.139422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.668183 (+1528761us) service_pool.cc:224] Handling call
0709 14:51:56.685173 (+ 16990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.685968 (+   795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.686185 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.690430  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.726658  4681 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.727342  4681 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:56.727564  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 877) took 1188 ms. Trace:
I20260709 14:51:56.727763  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.728067  2681 rpcz_store.cc:276] 0709 14:51:55.539177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.726547 (+1187370us) service_pool.cc:224] Handling call
0709 14:51:56.726969 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.727189 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:56.727216 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:56.727260 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:51:56.727297 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.727355 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:56.727476 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.729497  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:56.681902  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:56.731422  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.732077  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:56.719661  4637 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.733538  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:56.734447  4620 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.735101  4620 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.736199  4620 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:56.737622  4757 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:51:56.668277  4675 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:56.730626  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 878) took 1099 ms. Trace:
I20260709 14:51:56.751984  2681 rpcz_store.cc:276] 0709 14:51:55.631293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.729496 (+1098203us) service_pool.cc:224] Handling call
0709 14:51:56.729910 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.730173 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:56.730216 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:56.730280 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:51:56.730327 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.730405 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:56.730546 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.692759  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:56.755249  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62561.3us
I20260709 14:51:56.756435  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:56.693434  4589 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:56.694517  4570 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.696962  4785 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.675863  4822 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.615556  3537 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:51:56.702157  4723 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:56.703444  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:56.704281  2680 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:56.664237  4663 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.794340  2609 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:56.794976  2604 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:56.705184  2682 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:56.796139  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:56.611029  4719 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.664541  4711 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:56.661929  4653 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:56.716642  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 746773us
I20260709 14:51:56.838382  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.844201  2608 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:56.844691  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.845366  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.845749  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:56.677273  4397 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:56.692083  4826 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.168s	user 0.001s	sys 0.001s
I20260709 14:51:56.739584  2759 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:56.740340  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:56.740964  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:56.745904  2752 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:56.884352  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:56.784706  4675 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.888876  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:56.887821  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156982us
I20260709 14:51:56.891902  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:56.761544  4681 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:56.762089  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 283) took 4322 ms. Trace:
W20260709 14:51:56.762539  4829 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 14:51:56.763105  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 323) took 1449 ms (client timeout 1902 ms). Trace:
I20260709 14:51:56.740808  4802 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.768200  4500 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:56.769677  4828 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.739163  4753 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:56.779222  4824 raft_consensus.cc:2764] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:51:56.780341  4757 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:51:56.678318  4797 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.784135  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:56.935812  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.667143  4562 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:51:56.797719  2609 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:56.840401  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.841450  2604 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:56.685611  4734 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:56.785154  4789 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:56.781970  4369 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.848181  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.917572  4711 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:56.904109  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.885532  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:56.965189  2681 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:56.887789  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.970443  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 940) took 1183 ms. Trace:
I20260709 14:51:56.971024  2612 rpcz_store.cc:276] 0709 14:51:56.788925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.801756 (+ 12831us) service_pool.cc:224] Handling call
0709 14:51:56.970331 (+168575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.972499  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.974313  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:56.906252  2755 rpcz_store.cc:269] 0709 14:51:55.313838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:55.985531 (+671693us) service_pool.cc:224] Handling call
0709 14:51:56.762902 (+777371us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.763042 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211870080}
I20260709 14:51:56.912693  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.982000  2612 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:56.922010  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 181739us
I20260709 14:51:56.931913  4397 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.796753  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 875) took 1485 ms. Trace:
I20260709 14:51:56.936282  4757 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:56.910614  4785 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.948745  2606 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:56.863806  2607 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:56.918053  4831 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:56.951126  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.910454  4759 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:56.909955  4500 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:56.947914  4789 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.751448  4796 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:56.965189  2752 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:56.980684  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 884) took 1016 ms. Trace:
I20260709 14:51:56.984313  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:56.935690  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 151692us
I20260709 14:51:57.034399  4397 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:57.033856  2686 rpcz_store.cc:276] 0709 14:51:55.963867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.095601 (+131734us) service_pool.cc:224] Handling call
0709 14:51:56.980341 (+884740us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400376754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:56.980572 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894459776}
I20260709 14:51:57.035178  4397 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:56.982409  2760 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:56.990427  2609 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:57.049635  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.050534  2609 raft_consensus.cc:2368] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:56.997713  4785 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:56.924639  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:57.055513  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 131277us
I20260709 14:51:56.944830  4653 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:56.991680  2682 rpcz_store.cc:276] 0709 14:51:55.311811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.704395 (+1392584us) service_pool.cc:224] Handling call
0709 14:51:56.704760 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.796383 (+ 91623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.796675 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:56.906196  4781 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 14:51:56.994707  2685 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:56.942035  4266 raft_consensus.cc:2764] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:51:56.910162  2678 rpcz_store.cc:276] 0709 14:51:52.439543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:52.449480 (+  9937us) service_pool.cc:224] Handling call
0709 14:51:52.546462 (+ 96982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:54.916798 (+2370336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.761637 (+1844839us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b540024efd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:56.761978 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5178086656}
I20260709 14:51:56.938789  4562 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:51:56.908565  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:56.999173  2604 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:57.015561  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 337) took 1536 ms. Trace:
I20260709 14:51:57.129173  2753 rpcz_store.cc:276] 0709 14:51:55.479528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.088422 (+608894us) service_pool.cc:224] Handling call
0709 14:51:57.015210 (+926788us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b5400570154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:57.015438 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075055488}
I20260709 14:51:57.129810  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:57.013540  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.009025  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:57.034256  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50028.6us
I20260709 14:51:57.038560  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.132009  2606 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:57.040329  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:57.132118  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:57.043246  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:56.999790  4757 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:57.036185  4397 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.051441  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.160905  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.162242  2609 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:56.935124  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:56.997658  4711 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:57.033748  4789 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:57.119820  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:57.125530  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:57.049209  4824 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 14:51:57.140815  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:57.152053  2757 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:57.133754  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:57.152361  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.059161  4653 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:57.208428  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.209403  3545 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:57.171412  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.172856  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:57.212152  2758 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.213380  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.213868  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.214237  3537 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.215164  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.215946  3543 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:57.217144  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.217952  3542 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:57.219106  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.219172  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.220134  3536 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 14:51:57.213263  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:57.231642  2607 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:57.060312  4785 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:57.183729  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.238039  2606 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.168044  4723 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:57.189826  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56206.7us
I20260709 14:51:57.248632  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:57.190806  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:57.250922  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60255us
I20260709 14:51:57.063009  4655 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:57.291874  4655 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:57.292399  4655 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:57.199749  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.048225  4570 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 14:51:57.211055  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.173218  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:57.213263  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.163194  4637 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:57.236063  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.187338  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.268574  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.244622  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.275205  4562 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:57.282781  4802 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:57.180569  4804 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:57.283141  4734 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:57.222173  4620 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:57.138273  4500 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:57.222426  4799 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 14:51:57.287243  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.291115  4824 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:51:57.286262  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 298) took 2146 ms. Trace:
I20260709 14:51:57.294205  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:57.468365  2611 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.378381  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.468159  2684 rpcz_store.cc:276] 0709 14:51:55.140025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.668653 (+1528628us) service_pool.cc:224] Handling call
0709 14:51:56.687874 (+ 19221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.688179 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:56.688197 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:56.736046 (+ 47849us) log.cc:844] Serialized 26 byte log entry
0709 14:51:56.993036 (+256990us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:56.993082 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:56.993159 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:57.285976 (+292817us) raft_consensus.cc:1685] finished
0709 14:51:57.286010 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:57.286188 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":256451,"threads_started":1,"wal-append.queue_time_us":509885}
I20260709 14:51:57.440546  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:57.341034  2758 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:51:57.471129  2681 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.471310  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.472121  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 932) took 1496 ms. Trace:
I20260709 14:51:57.224076  4653 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:57.354405  3536 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:51:57.442531  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.308950  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.333454  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.488487  3541 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:57.487375  3547 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 5
W20260709 14:51:57.466208  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 335203us
I20260709 14:51:57.470001  3542 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:57.190297  4801 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:57.468824  2760 raft_consensus.cc:2368] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:51:57.483289  2606 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.294967  2603 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.475044  2758 rpcz_store.cc:276] 0709 14:51:55.975945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.471187 (+1495242us) service_pool.cc:224] Handling call
0709 14:51:57.471544 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.471755 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:57.471782 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:57.471835 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:57.471871 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.471928 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:57.472050 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.469398  2604 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.483566  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.507418  3536 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:57.485646  2757 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:57.498870  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:57.517181  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81851.8us
I20260709 14:51:57.515506  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.515175  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.517707  2609 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.399199  4723 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:57.518105  3543 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.509945  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.510398  2753 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:57.518172  2611 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.519616  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:57.475878  4734 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:57.487035  4641 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:57.534724  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 847) took 1557 ms. Trace:
I20260709 14:51:57.534404  2760 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:51:57.535264  2753 rpcz_store.cc:276] 0709 14:51:55.977024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.477331 (+1500307us) service_pool.cc:224] Handling call
0709 14:51:57.505359 (+ 28028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.530605 (+ 25246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.534656 (+  4051us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.525454  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.477480  4620 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:51:57.499955  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:57.472121  4824 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:57.536244  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:57.536185  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 849) took 1558 ms. Trace:
I20260709 14:51:57.540138  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.537559  2758 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:57.546422  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 851) took 1568 ms. Trace:
I20260709 14:51:57.441780  4753 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:57.545449  2606 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.537595  3541 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:57.544181  2760 rpcz_store.cc:276] 0709 14:51:55.977697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.505923 (+1528226us) service_pool.cc:224] Handling call
0709 14:51:57.528130 (+ 22207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.535661 (+  7531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.536089 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.553778  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:57.548239  2758 rpcz_store.cc:276] 0709 14:51:55.978317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.506666 (+1528349us) service_pool.cc:224] Handling call
0709 14:51:57.537279 (+ 30613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.546111 (+  8832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.546352 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.550470  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.509493  4637 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:57.555768  2604 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:57.557941  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.559295  2611 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.500960  4801 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:57.533633  4734 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:57.571643  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:57.573807  2684 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:57.465557  4711 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:57.579397  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 935) took 1463 ms. Trace:
I20260709 14:51:57.583972  2760 rpcz_store.cc:276] 0709 14:51:56.115675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.554781 (+1439106us) service_pool.cc:224] Handling call
0709 14:51:57.578689 (+ 23908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.578952 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:57.578985 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:57.579039 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:57.579072 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.579128 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:57.579295 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.580826  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 942) took 1440 ms. Trace:
I20260709 14:51:57.542654  4500 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:57.587201  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.548586  4620 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.577841  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.585284  2758 rpcz_store.cc:276] 0709 14:51:56.142105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.556885 (+1414780us) service_pool.cc:224] Handling call
0709 14:51:57.580192 (+ 23307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.580443 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:57.580480 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:57.580531 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:57.580559 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.580623 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:57.580753 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.590675  3412 leader_election.cc:400] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:57.592398  3412 leader_election.cc:403] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:57.557639  4637 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.597239  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 300) took 2456 ms. Trace:
I20260709 14:51:57.597920  2680 rpcz_store.cc:276] 0709 14:51:55.140664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.673877 (+1533213us) service_pool.cc:224] Handling call
0709 14:51:56.703931 (+ 30054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:56.789574 (+ 85643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.596895 (+807321us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b54001d10d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:57.597145 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68498688}
I20260709 14:51:57.587953  3546 raft_consensus.cc:2376] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:51:57.602290  3543 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:57.638981  4636 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:57.591830  4802 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:57.670351  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 347) took 1384 ms (client timeout 1531 ms). Trace:
W20260709 14:51:57.702641  2756 rpcz_store.cc:269] 0709 14:51:56.286431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.328324 (+ 41893us) service_pool.cc:224] Handling call
0709 14:51:57.670132 (+1341808us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:57.670282 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461258112}
I20260709 14:51:57.617461  2680 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:57.704454  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 880) took 1973 ms. Trace:
I20260709 14:51:57.704890  2680 rpcz_store.cc:276] 0709 14:51:55.731440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.615939 (+1884499us) service_pool.cc:224] Handling call
0709 14:51:57.617222 (+  1283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.704171 (+ 86949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.704397 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.706466  2680 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 14:51:57.684226  2726 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.001s	sys 0.010s
W20260709 14:51:57.707237  2726 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.002s	sys 0.011s
W20260709 14:51:57.707628  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 543507us
I20260709 14:51:57.708590  3542 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:57.572222  4801 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:57.665472  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.707531  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 883) took 1775 ms. Trace:
I20260709 14:51:57.712527  2680 rpcz_store.cc:276] 0709 14:51:55.931837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.705717 (+1773880us) service_pool.cc:224] Handling call
0709 14:51:57.706087 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.707096 (+  1009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.707414 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.713210  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.714006  2680 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:57.714246  3541 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 14:51:57.540426  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.715684  2609 raft_consensus.cc:2368] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:51:57.715942  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.717032  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.717891  2609 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:57.610587  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:57.625537  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:57.608522  4663 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.771s	user 0.014s	sys 0.000s
I20260709 14:51:57.599437  4756 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:57.734627  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.592677  4855 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:57.529841  4723 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:57.672236  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:57.700912  2612 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.005s
I20260709 14:51:57.764320  2612 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:51:57.590209  4532 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:57.570945  3537 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:57.616936  2758 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:57.779819  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 348) took 1218 ms. Trace:
I20260709 14:51:57.780418  2758 rpcz_store.cc:276] 0709 14:51:56.561902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.592084 (+1030182us) service_pool.cc:224] Handling call
0709 14:51:57.616694 (+ 24610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.779360 (+162666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.779726 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.629879  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.548269  4641 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:57.605654  4577 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.955s	user 0.013s	sys 0.009s
I20260709 14:51:57.636317  2684 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:51:57.669965  2755 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:57.809072  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 354) took 1234 ms. Trace:
I20260709 14:51:57.809512  2755 rpcz_store.cc:276] 0709 14:51:56.574565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.669148 (+1094583us) service_pool.cc:224] Handling call
0709 14:51:57.669556 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.808785 (+139229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.809013 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23424}
I20260709 14:51:57.723476  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:57.735356  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 887) took 1753 ms. Trace:
I20260709 14:51:57.811215  2680 rpcz_store.cc:276] 0709 14:51:55.982183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.713422 (+1731239us) service_pool.cc:224] Handling call
0709 14:51:57.713795 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.714524 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.735231 (+ 20707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.813081  2680 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:57.813941  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 803) took 1454 ms. Trace:
I20260709 14:51:57.811632  3543 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:57.814321  2680 rpcz_store.cc:276] 0709 14:51:56.359520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.812365 (+1452845us) service_pool.cc:224] Handling call
0709 14:51:57.812827 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.813603 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.813864 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.818035  2680 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:51:57.669428  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:57.749764  4756 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:57.767584  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:51:57.767818  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 157349us
I20260709 14:51:57.836562  3542 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:57.836882  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:57.770959  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:57.796020  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:57.839416  2758 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:57.799374  4532 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:51:57.802479  4729 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.299s	user 0.015s	sys 0.004s
I20260709 14:51:57.802785  2609 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:57.726439  4852 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:51:57.849028  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 290379us
I20260709 14:51:57.851817  2684 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:57.852982  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 314) took 1494 ms. Trace:
I20260709 14:51:57.853454  2684 rpcz_store.cc:276] 0709 14:51:56.358411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.809237 (+1450826us) service_pool.cc:224] Handling call
0709 14:51:57.851511 (+ 42274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.852563 (+  1052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.852892 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.714574  4802 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.857825  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.738631  4663 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.825198  2607 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:57.783658  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.019s	sys 0.012s
I20260709 14:51:57.772795  4781 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 14:51:57.840986  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 315) took 1400 ms. Trace:
I20260709 14:51:57.876789  2680 rpcz_store.cc:276] 0709 14:51:56.440256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.817112 (+1376856us) service_pool.cc:224] Handling call
0709 14:51:57.817546 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.832353 (+ 14807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.840860 (+  8507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.859933  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.859804  3541 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:57.747597  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:57.863080  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:57.863723  3546 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:57.727950  4636 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:57.868602  2604 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:51:57.840683  3412 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
W20260709 14:51:57.848407  4842 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 14:51:57.789814  4853 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5.
I20260709 14:51:57.762140  4855 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:51:57.873029  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.020s	sys 0.012s
I20260709 14:51:57.859885  4729 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:57.949970  4843 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
I20260709 14:51:57.906579  2684 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:57.955598  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 320) took 1172 ms. Trace:
I20260709 14:51:57.955936  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.956175  2684 rpcz_store.cc:276] 0709 14:51:56.782986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.884629 (+1101643us) service_pool.cc:224] Handling call
0709 14:51:57.906135 (+ 21506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.955183 (+ 49048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.955518 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:57.907063  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:51:57.855293  4397 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 14:51:57.909794  4643 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:57.913093  2678 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.001s
W20260709 14:51:57.913646  4845 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.353s	user 0.000s	sys 0.002s
I20260709 14:51:57.915596  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.917933  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:57.923234  3536 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:57.923441  3546 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:57.975059  3546 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:57.976169  3546 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:57.976987  4856 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:57.733820  4824 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.865427  4795 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:57.835234  4577 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:57.938089  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:57.915652  4847 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 14:51:57.866916  4850 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 14:51:57.962610  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 215158us
I20260709 14:51:57.977406  3546 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:57.860853  4852 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.928114  3412 leader_election.cc:403] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:51:57.980806  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 240836us
I20260709 14:51:57.994681  2679 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:57.973551  2605 raft_consensus.cc:2376] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6. Re-sending same reply.
I20260709 14:51:57.868321  4862 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 14:51:57.987178  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 312) took 1138 ms. Trace:
I20260709 14:51:57.930274  4860 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:57.988152  3547 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
W20260709 14:51:57.988680  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 123958us
W20260709 14:51:57.990082  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 325) took 2588 ms (client timeout 1730 ms). Trace:
I20260709 14:51:57.980964  2683 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: []
I20260709 14:51:57.995913  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 946) took 1204 ms. Trace:
I20260709 14:51:58.007670  2605 rpcz_store.cc:276] 0709 14:51:56.791137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.837745 (+ 46608us) service_pool.cc:224] Handling call
0709 14:51:57.915370 (+1077625us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:57.995838 (+ 80468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12537216}
W20260709 14:51:58.008940  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 168402us
I20260709 14:51:58.009366  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.010182  2605 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:58.010921  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:58.006559  2759 rpcz_store.cc:269] 0709 14:51:55.396611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:55.659701 (+263090us) service_pool.cc:224] Handling call
0709 14:51:57.984233 (+2324532us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:57.984376 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2480608640}
I20260709 14:51:58.012132  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.013615  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:57.997360  2684 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:57.999001  2611 rpcz_store.cc:276] 0709 14:51:56.848311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.917563 (+1069252us) service_pool.cc:224] Handling call
0709 14:51:57.987102 (+ 69539us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.014524  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 891) took 1147 ms. Trace:
I20260709 14:51:58.014875  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.015406  2611 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:51:57.971838  4786 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:58.016094  4786 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:58.016178  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.017318  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.017191  4786 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.002954  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:58.018401  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.019129  4857 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:58.019536  4857 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:58.019645  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:58.019944  4857 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:58.014921  2684 rpcz_store.cc:276] 0709 14:51:56.867307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.980034 (+1112727us) service_pool.cc:224] Handling call
0709 14:51:57.996789 (+ 16755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.014256 (+ 17467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.014468 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.022591  2684 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:58.023303  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 892) took 1155 ms. Trace:
I20260709 14:51:58.023598  2684 rpcz_store.cc:276] 0709 14:51:56.868277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.021941 (+1153664us) service_pool.cc:224] Handling call
0709 14:51:58.022344 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:58.022968 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.023242 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.018472  4786 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:58.026674  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.028105  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:57.996101  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 946) took 1726 ms. Trace:
I20260709 14:51:58.013715  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.031145  2605 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:51:58.033676  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.032438  2684 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 14:51:58.039477  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 893) took 1166 ms. Trace:
I20260709 14:51:58.039985  2684 rpcz_store.cc:276] 0709 14:51:56.873401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.024308 (+1150907us) service_pool.cc:224] Handling call
0709 14:51:58.032070 (+  7762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.039070 (+  7000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.039394 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.041988  2684 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:51:58.030112  2760 rpcz_store.cc:276] 0709 14:51:56.261083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.584951 (+1323868us) service_pool.cc:224] Handling call
0709 14:51:57.603374 (+ 18423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.603675 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:51:57.833680 (+230005us) log.cc:844] Serialized 14 byte log entry
0709 14:51:57.986047 (+152367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:57.986131 (+    84us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:57.986209 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.986315 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:57.986787 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2231,"prepare.queue_time_us":396112,"raft.queue_time_us":2392,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":4955,"thread_start_us":343773,"threads_started":2}
I20260709 14:51:58.029245  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.047457  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 316) took 1197 ms. Trace:
I20260709 14:51:58.048041  2611 rpcz_store.cc:276] 0709 14:51:56.849620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.028954 (+1179334us) service_pool.cc:224] Handling call
0709 14:51:58.047371 (+ 18417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.034026  4857 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:57.972685  2678 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:57.994161  4397 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:57.980432  4643 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.084110  2610 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:58.224676  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:58.080058  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60485.3us
I20260709 14:51:58.225436  2611 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:58.226409  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.208315  3546 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 14:51:58.228201  3546 pending_rounds.cc:85] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 1
I20260709 14:51:58.218216  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.228681  3546 pending_rounds.cc:107] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:51:58.228797  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.229012  3546 raft_consensus.cc:2889] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:58.209389  2681 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 14:51:58.161999  3412 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.510s (SENT)
I20260709 14:51:57.989971  3537 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:58.230141  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.048416  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:58.230898  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68966.3us
I20260709 14:51:58.232468  2684 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:58.045766  4785 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.759s	user 0.015s	sys 0.000s
W20260709 14:51:58.171818  4734 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
I20260709 14:51:58.209896  2603 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 14:51:58.236882  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:57.990213  4846 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:58.239359  3412 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.516s (SENT)
I20260709 14:51:58.241133  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.242599  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:51:58.048828  4757 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.847s	user 0.013s	sys 0.000s
I20260709 14:51:58.045997  4825 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:51:58.258657  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 245132us
I20260709 14:51:58.245121  2611 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:58.004765  4860 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:58.172650  4865 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 14:51:58.253836  2681 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:58.168336  4641 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.045560  4859 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:58.253430  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:58.235139  2684 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:58.262780  3547 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.173669  4532 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.286134  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:58.255759  4655 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.963s	user 0.015s	sys 0.000s
I20260709 14:51:58.223775  4397 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.209043  4795 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 14:51:58.287153  3547 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:58.302170  3547 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:51:58.303766  3547 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:58.170138  4729 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.265100  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.307850  2681 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:51:58.308314  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.271327  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 335) took 1069 ms. Trace:
I20260709 14:51:58.308952  2605 rpcz_store.cc:276] 0709 14:51:57.202172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.242144 (+1039972us) service_pool.cc:224] Handling call
0709 14:51:58.271164 (+ 29020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.261379  4856 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:51:58.248474  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 318) took 1462 ms (client timeout 1811 ms). Trace:
I20260709 14:51:58.248333  4723 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.255573  4850 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.314088  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:58.284109  4863 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 14:51:58.258095  4798 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.612s	user 0.010s	sys 0.005s
I20260709 14:51:58.083606  4858 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.244s
I20260709 14:51:58.259656  4663 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.257267  4880 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:58.288506  4859 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:58.264465  4874 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:58.282663  4731 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:58.350260  2760 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:58.369587  3547 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:51:58.282781  4860 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:58.250155  4785 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.382266  3411 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 3. Current term is 4.
I20260709 14:51:58.346472  2605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:51:58.271384  4846 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.254156  4734 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.400729  4878 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:51:58.402796  4869 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 14:51:58.404872  4786 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.405582  4802 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.406206  4756 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.382416  4729 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:58.407603  4723 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.313448  4872 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.208s
I20260709 14:51:58.411794  4658 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.412434  4577 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.303501  4532 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.528957  3545 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:58.304617  4641 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:58.264966  4757 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.260435  4864 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 14:51:58.473340  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 232319us
I20260709 14:51:58.557354  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:51:58.475870  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 115679us
W20260709 14:51:58.349223  4735 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.826s	user 0.015s	sys 0.002s
I20260709 14:51:58.342690  4856 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:58.566375  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.567173  2752 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:51:58.567677  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 353) took 2000 ms (client timeout 1940 ms). Trace:
W20260709 14:51:58.568058  2752 rpcz_store.cc:269] 0709 14:51:56.567603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.033595 (+465992us) service_pool.cc:224] Handling call
0709 14:51:58.566170 (+1532575us) spinlock_profiling.cc:243] Waited 914 us on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:58.567545 (+  1375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103040}
I20260709 14:51:58.564600  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:58.500250  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 271525us
W20260709 14:51:58.339475  2678 rpcz_store.cc:269] 0709 14:51:56.782470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.187074 (+404604us) service_pool.cc:224] Handling call
0709 14:51:58.244806 (+1057732us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:51:58.507325  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 498348us
I20260709 14:51:58.577695  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.579167  4658 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:58.579874  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:58.581012  4846 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.581247  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.570809  4856 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:58.583304  4643 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:58.583959  4643 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:51:58.584664  4643 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:51:58.584735  2726 leader_election.cc:336] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.943s (SENT)
I20260709 14:51:58.585172  4643 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:58.585381  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:58.586752  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.588294  4785 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:58.589074  4757 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:58.589181  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:58.509761  2754 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:58.590039  3538 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:58.591058  2755 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:58.559003  4859 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.591723  4734 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.520870  4786 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.592597  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:58.512317  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 273002us
I20260709 14:51:58.593991  4786 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:58.594326  3412 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.684s (SENT)
I20260709 14:51:58.594858  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.566509  4735 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.595101  4729 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:58.595644  4729 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:58.309512  4655 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.555951  4532 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.597023  4735 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.597769  4735 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:58.597924  4655 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.598691  2756 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.562631  4864 consensus_queue.cc:1243] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:51:58.521992  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.599928  2759 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:51:58.497761  4637 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.869s	user 0.007s	sys 0.005s
I20260709 14:51:58.523841  2609 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:58.409309  4397 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.533396  4723 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:58.602622  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.504405  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 876) took 3184 ms. Trace:
I20260709 14:51:58.604270  2677 rpcz_store.cc:276] 0709 14:51:55.312076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.721055 (+1408979us) service_pool.cc:224] Handling call
0709 14:51:56.721484 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:56.721938 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:56.721974 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:56.722550 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 14:51:57.959100 (+1236550us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:57.972452 (+ 13352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:57.984190 (+ 11738us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:51:58.470716 (+486526us) raft_consensus.cc:1685] finished
0709 14:51:58.470758 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:58.496240 (+ 25482us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4180,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"thread_start_us":1196026,"threads_started":1,"wal-append.queue_time_us":1145343}
I20260709 14:51:58.605392  4889 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.606096  4889 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.608217  2611 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:58.609027  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 315) took 1759 ms. Trace:
I20260709 14:51:58.609395  2760 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:51:58.609365  2611 rpcz_store.cc:276] 0709 14:51:56.849307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.607640 (+1758333us) service_pool.cc:224] Handling call
0709 14:51:58.608007 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.608755 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.608967 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.609927  2760 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:58.509305  4731 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:58.612191  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 787) took 3760 ms. Trace:
I20260709 14:51:58.611660  4731 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.612615  2685 rpcz_store.cc:276] 0709 14:51:54.851395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.566063 (+1714668us) service_pool.cc:224] Handling call
0709 14:51:56.591794 (+ 25731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.061530 (+469736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.611922 (+1550392us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.612127 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926055168}
I20260709 14:51:58.614058  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:58.615875  4620 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.008s	user 0.012s	sys 0.002s
I20260709 14:51:58.381361  4880 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:58.341841  4871 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.616379  4620 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.616976  4880 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:58.617158  4871 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.619751  4871 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.467298  4850 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.620795  2758 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:58.622197  4850 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.541119  4835 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:58.528759  4802 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.623179  2755 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:58.529239  2603 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.589449  4785 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:51:58.567071  4875 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:58.596056  4729 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:58.600535  4637 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.302940  4870 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.592612  4857 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:58.602686  4653 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.098s	user 0.006s	sys 0.015s
W20260709 14:51:58.603129  4789 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.354s	user 0.011s	sys 0.005s
I20260709 14:51:58.605167  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.558454  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:58.610613  4500 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.018s	user 0.006s	sys 0.009s
I20260709 14:51:58.626166  3541 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.549374  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 782) took 3728 ms. Trace:
I20260709 14:51:58.664157  2679 rpcz_store.cc:276] 0709 14:51:54.821275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.510741 (+1689466us) service_pool.cc:224] Handling call
0709 14:51:56.514499 (+  3758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.995231 (+1480732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.548999 (+553768us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.549286 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3403506560}
I20260709 14:51:58.629048  2758 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:51:58.629523  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 108293us
I20260709 14:51:58.665041  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:58.509023  2607 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.002s
I20260709 14:51:58.666014  2607 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:58.479065  4798 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.630509  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.485585  4663 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:51:58.632288  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:58.576198  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:58.636269  4883 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.671104  4797 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:58.671435  4883 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.611639  2760 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.672713  2760 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:58.673009  4860 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.673640  2605 raft_consensus.cc:606] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.674088  4883 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.635430  4802 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:58.642027  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.623144  4835 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:58.634788  4815 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.656518  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 952) took 1863 ms. Trace:
I20260709 14:51:58.640424  4875 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.683256  3536 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.683871  4815 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.683272  2603 rpcz_store.cc:276] 0709 14:51:56.793081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.346635 (+1553554us) service_pool.cc:224] Handling call
0709 14:51:58.503851 (+157216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.640344 (+136493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.656392 (+ 16048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.684099  4875 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.685530  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.685667  4875 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:58.624714  4577 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.686261  4815 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.675207  4867 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.686622  4577 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.687254  4867 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.687724  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.688470  4890 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:51:58.689000  4577 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.659310  4870 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.660799  4781 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:58.689724  4781 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:58.690761  4781 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.691370  4870 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:58.635059  4801 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.895s	user 0.009s	sys 0.003s
I20260709 14:51:58.691834  4835 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.692446  4801 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.693032  4835 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:58.586215  4643 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.695696  4859 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:58.573706  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:58.667706  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62620.5us
I20260709 14:51:58.634291  4532 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.668807  4636 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:58.665815  4789 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.706732  4801 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.706936  4789 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.691932  4781 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:51:58.711359  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.712261  2758 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:58.712527  4852 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.713330  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.713994  2758 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:51:58.715152  2612 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:58.715294  3540 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:58.716349  4798 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:58.717310  2612 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:58.718011  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 318) took 1867 ms. Trace:
I20260709 14:51:58.669538  4856 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.719698  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 859) took 1664 ms. Trace:
I20260709 14:51:58.720139  2756 rpcz_store.cc:276] 0709 14:51:57.054881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.703852 (+648971us) service_pool.cc:224] Handling call
0709 14:51:57.741947 (+ 38095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.599154 (+857207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.719416 (+120262us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.719627 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969823104}
I20260709 14:51:58.720587  4856 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:51:58.722196  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 572) took 1511 ms (client timeout 1879 ms). Trace:
W20260709 14:51:58.722680  3538 rpcz_store.cc:269] 0709 14:51:57.212631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.212883 (+   252us) service_pool.cc:224] Handling call
0709 14:51:58.721963 (+1509080us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54004b3cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:58.722132 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579657600}
I20260709 14:51:58.563387  4641 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.681715  4860 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.725400  4641 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.727142  3542 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
I20260709 14:51:58.729415  4729 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.729984  4620 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.662312  4653 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.670910  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 573) took 1441 ms. Trace:
W20260709 14:51:58.740082  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.671918  4797 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.632939  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:58.675416  3546 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:58.675848  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:51:58.742234  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 109424us
I20260709 14:51:58.742434  2605 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 2: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:58.676414  3547 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.736865  4532 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 14:51:58.743110  2650 leader_election.cc:400] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:58.743456  2650 leader_election.cc:403] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 14:51:58.677114  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.744318  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:58.744414  2751 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:58.745077  4883 raft_consensus.cc:2764] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:51:58.745213  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.677839  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:58.745798  4883 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:58.678109  4868 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:58.679904  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 953) took 1016 ms. Trace:
I20260709 14:51:58.746624  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 953) took 1881 ms. Trace:
I20260709 14:51:58.746783  2754 rpcz_store.cc:276] 0709 14:51:57.663252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.669110 (+  5858us) service_pool.cc:224] Handling call
0709 14:51:58.679658 (+1010548us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:58.679827 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1558014720}
I20260709 14:51:58.747133  2606 rpcz_store.cc:276] 0709 14:51:56.865605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.745698 (+1880093us) service_pool.cc:224] Handling call
0709 14:51:58.746093 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.746289 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:58.746317 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:58.746363 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:51:58.746398 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.746452 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:58.746555 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.748137  4653 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:58.680150  4802 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:51:58.682255  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 106225us
I20260709 14:51:58.749488  4889 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.750180  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:58.640980  4881 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.751168  4881 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.752308  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 667) took 1299 ms. Trace:
I20260709 14:51:58.752733  3543 rpcz_store.cc:276] 0709 14:51:57.452979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.899303 (+446324us) service_pool.cc:224] Handling call
0709 14:51:58.751502 (+852199us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:58.751742 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.751954 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:58.751984 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:58.752033 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:51:58.752068 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.752124 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:58.752244 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3404672}
I20260709 14:51:58.753607  4881 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.685878  4858 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.688721  4734 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:51:58.754793  4883 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:58.754505  4858 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.755489  4675 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.757393  4858 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.758327  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:58.759665  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:58.761260  4801 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:58.761751  4801 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.763090  4789 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.763147  4875 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.763649  4789 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.764088  4875 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.764672  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 661) took 1313 ms. Trace:
I20260709 14:51:58.764743  4789 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.765239  3537 rpcz_store.cc:276] 0709 14:51:57.451227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.893303 (+442076us) service_pool.cc:224] Handling call
0709 14:51:57.962120 (+ 68817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.243760 (+281640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.764434 (+520674us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b54004b2dd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:58.764604 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59550080}
I20260709 14:51:58.765044  4875 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.762655  4801 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:58.746654  4868 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:58.767859  4855 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:58.767975  4868 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.769120  4868 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:58.770957  4867 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:51:58.668522  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 110158us
I20260709 14:51:58.774895  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.776185  3411 leader_election.cc:400] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:51:58.776479  3411 leader_election.cc:403] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:58.699733  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.700546  2755 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:51:58.700716  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 127045us
I20260709 14:51:58.777791  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.703608  4577 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:58.704038  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:58.704278  2760 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:58.711530  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.784274  4862 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:51:58.784925  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.784744  4880 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:58.669858  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 792) took 3509 ms. Trace:
I20260709 14:51:58.726622  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 342) took 3181 ms. Trace:
I20260709 14:51:58.737464  2752 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:58.718310  2612 rpcz_store.cc:276] 0709 14:51:56.850285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.716701 (+1866416us) service_pool.cc:224] Handling call
0709 14:51:58.717068 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:58.717742 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.717954 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.784329  2608 raft_consensus.cc:2368] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:51:58.786242  2683 rpcz_store.cc:276] 0709 14:51:55.160854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.695154 (+1534300us) service_pool.cc:224] Handling call
0709 14:51:56.695531 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.007160 (+1311629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.669597 (+662437us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000c9ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.669793 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2955696512}
I20260709 14:51:58.788095  2612 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 14:51:58.788707  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 958) took 1658 ms. Trace:
I20260709 14:51:58.789024  2612 rpcz_store.cc:276] 0709 14:51:57.129882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.787474 (+1657592us) service_pool.cc:224] Handling call
0709 14:51:58.787777 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.788470 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.788649 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.741312  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:58.790282  2612 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:58.786407  2757 rpcz_store.cc:276] 0709 14:51:55.544845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.234735 (+1689890us) service_pool.cc:224] Handling call
0709 14:51:57.235150 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:57.509090 (+273940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.726327 (+1217237us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400060254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.726546 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555408128}
I20260709 14:51:58.792595  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 337) took 1388 ms. Trace:
I20260709 14:51:58.793077  2612 rpcz_store.cc:276] 0709 14:51:57.402265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.789824 (+1387559us) service_pool.cc:224] Handling call
0709 14:51:58.790094 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.790598 (+   504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.790799 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.739457  2758 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:51:58.740187  3545 rpcz_store.cc:276] 0709 14:51:57.230077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.259665 (+ 29588us) service_pool.cc:224] Handling call
0709 14:51:57.467042 (+207377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.529533 (+1062491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.670640 (+141107us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:58.670819 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441587456}
I20260709 14:51:58.677026  4857 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:51:58.771163  4846 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:51:58.771875  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.773248  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:58.699496  3412 leader_election.cc:400] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:51:58.797143  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.774003  2605 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:58.778548  2759 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:51:58.778580  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:58.779592  2611 raft_consensus.cc:2368] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8.
I20260709 14:51:58.779781  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:58.796305  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:51:58.802044  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102690us
I20260709 14:51:58.824647  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:51:58.838979  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59744.1us
I20260709 14:51:58.839334  2751 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:58.839632  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.840581  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:58.841825  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59940.7us
I20260709 14:51:58.841992  3545 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:51:58.843858  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 67833.3us
I20260709 14:51:58.844692  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:58.843233  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.999s (SENT) [suppressed 45 similar messages]
I20260709 14:51:58.786198  4675 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:58.799793  2608 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 326) took 1912 ms (client timeout 1510 ms). Trace:
I20260709 14:51:58.850934  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:58.850513  2608 rpcz_store.cc:269] 0709 14:51:56.887051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.889752 (+  2701us) service_pool.cc:224] Handling call
0709 14:51:58.703843 (+1814091us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:58.799702 (+ 95859us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4527488}
I20260709 14:51:58.848980  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.851864  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.852348  2611 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:58.849232  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:58.795778  4719 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 14:51:58.849954  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 957) took 1806 ms. Trace:
I20260709 14:51:58.838995  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:51:58.802878  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61603.2us
I20260709 14:51:58.785874  4776 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
W20260709 14:51:58.872267  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:51:58.902416  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77881.6us
I20260709 14:51:58.786947  4815 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.901655  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:58.904779  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57707.1us
I20260709 14:51:58.904232  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.786772  4802 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:58.904024  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52261.6us
I20260709 14:51:58.861792  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.882759  2751 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:58.915771  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76875.1us
W20260709 14:51:58.906244  4796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.268s	user 0.016s	sys 0.012s
I20260709 14:51:58.916776  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.842152  4855 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:51:58.909904  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 147576us
W20260709 14:51:58.910248  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:58.840682  4577 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.912835  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.801693  4636 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 14:51:58.915947  2611 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:58.917666  2608 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:51:58.907692  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:58.919299  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.841964  4756 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 14:51:58.903530  4891 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:51:58.848919  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:58.916740  4796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.016s	sys 0.013s
I20260709 14:51:58.919181  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:51:58.860932  4835 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:58.901741  4675 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:58.809975  4653 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 14:51:58.838606  4797 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:58.798686  4846 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:58.922171  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.906926  2605 rpcz_store.cc:276] 0709 14:51:57.043296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.773225 (+1729929us) service_pool.cc:224] Handling call
0709 14:51:58.773607 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.849634 (+ 76027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:58.849876 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:58.997025  2753 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
W20260709 14:51:58.963898  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44810.2us
I20260709 14:51:58.921991  4860 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:58.804203  4500 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 14:51:58.843914  4643 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 14:51:58.928074  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:58.847833  4867 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 14:51:58.861148  4858 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:58.849746  4881 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 14:51:58.984499  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:58.962039  3542 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:58.848150  4883 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
W20260709 14:51:58.964116  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56531.2us
I20260709 14:51:58.984128  4795 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:51:58.902247  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:51:59.056903  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 128911us
W20260709 14:51:59.057420  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 155257us
I20260709 14:51:59.058143  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:59.059320  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:58.928519  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:58.921155  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:58.988387  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.061546  2608 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:59.062449  4828 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.062732  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.063243  4828 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:59.063659  4828 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:58.988605  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.064723  3543 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:51:59.065383  4898 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 14:51:58.860417  4862 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 14:51:59.064749  4878 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:59.066743  4828 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.067613  4878 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.068069  4878 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:59.068400  4828 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:51:58.994653  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 216185us
I20260709 14:51:58.799722  4729 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.050491  4880 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:51:58.999634  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 77651.8us
I20260709 14:51:58.984802  4899 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:51:58.905184  4890 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 14:51:58.850311  4857 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:58.921315  4855 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:58.912984  4852 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 14:51:58.920740  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.088165  2682 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:58.920416  4893 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 14:51:59.004032  4663 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 14:51:59.052280  4881 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:59.061434  4500 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:51:59.053000  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:59.053689  4883 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:59.034217  4901 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:59.042718  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:59.073355  4562 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:58.924100  4798 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.131s
I20260709 14:51:59.050606  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:59.055692  4858 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:58.913138  4658 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:59.067175  4723 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:58.995908  4815 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:59.070779  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:59.073405  2605 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:59.074891  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:58.913331  4859 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:51:59.081071  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 160867us
I20260709 14:51:59.082978  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.084978  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:58.999028  4846 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.086015  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.086966  2681 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:51:59.078665  2758 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:51:59.098786  3546 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:59.052587  4675 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:59.034132  4636 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:58.923182  4734 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.166s
I20260709 14:51:59.097332  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:58.985209  4641 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:51:59.072029  4637 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:59.071799  4397 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:59.132323  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.004911  4731 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.232s
W20260709 14:51:59.133728  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 74488.1us
I20260709 14:51:59.114845  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:58.916887  4894 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.302s
I20260709 14:51:59.093505  4855 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.178020  3543 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:51:59.228731  4864 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:51:59.134261  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:59.300796  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 166675us
I20260709 14:51:59.301354  4577 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:59.141580  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55749.3us
I20260709 14:51:59.093091  4835 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:51:59.086741  4891 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 14:51:59.207253  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 156817us
I20260709 14:51:59.212940  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 931) took 1504 ms. Trace:
I20260709 14:51:59.316433  4903 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:59.316653  2605 rpcz_store.cc:276] 0709 14:51:57.708842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.005283 (+1296441us) service_pool.cc:224] Handling call
0709 14:51:59.073217 (+ 67934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.212641 (+139424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.212877 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:51:59.316962  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:59.213166  2759 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:59.213483  2679 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:51:59.318813  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.319512  2679 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:51:59.214491  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 143825us
I20260709 14:51:59.215870  2606 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 14:51:59.321843  2606 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 4
I20260709 14:51:59.238659  4904 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:51:59.099303  4881 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.115571  4899 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:59.229557  2751 raft_consensus.cc:2368] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
W20260709 14:51:59.231618  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 331) took 2284 ms (client timeout 2564 ms). Trace:
I20260709 14:51:59.099462  4266 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:51:59.218670  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.139143  4500 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.088505  4729 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 14:51:59.268319  2608 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:59.138024  4883 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:59.093865  4880 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.116s
I20260709 14:51:59.141392  4795 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:59.099462  3537 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
W20260709 14:51:59.271677  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59287.2us
I20260709 14:51:59.280529  2757 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:59.178725  4658 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.132889  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.354100  3412 leader_election.cc:400] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8.
I20260709 14:51:59.088280  4620 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:51:59.282723  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.289914  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.142333  3542 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:51:59.219399  4859 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:59.034332  4797 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.208261  4786 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 14:51:59.299412  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.366887  4663 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 14:51:59.368476  4663 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:51:59.271330  4641 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.369005  4663 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:59.369412  4641 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:51:59.343565  4776 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 14:51:59.303992  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:59.337153  4266 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:51:59.373133  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 69292.7us
I20260709 14:51:59.374411  4798 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:59.375105  4798 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:59.376148  4798 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.267644  4636 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:59.268731  2677 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:59.380107  4636 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.380786  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.368872  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.382404  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.384110  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:59.307442  4902 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 14:51:59.306406  4577 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.322129  2606 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 14:51:59.220184  2611 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:59.322129  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:59.323911  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 938) took 1520 ms (client timeout 1696 ms). Trace:
I20260709 14:51:59.206038  4901 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:51:59.327061  2604 rpcz_store.cc:269] 0709 14:51:56.947671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.556606 (+608935us) service_pool.cc:224] Handling call
0709 14:51:59.215174 (+1658568us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54001ead54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.231545 (+ 16371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635918720}
I20260709 14:51:59.333844  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 2.
I20260709 14:51:59.220479  4884 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 14:51:59.306236  4876 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:51:59.317315  4903 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.384900  2677 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:59.388286  2606 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:51:59.343464  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.397722  2683 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:51:59.398778  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.400260  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.388816  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 951) took 1173 ms. Trace:
W20260709 14:51:59.401130  3540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 571) took 2190 ms (client timeout 1568 ms). Trace:
I20260709 14:51:59.401620  2611 rpcz_store.cc:276] 0709 14:51:58.215593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.132301 (+916708us) service_pool.cc:224] Handling call
0709 14:51:59.219831 (+ 87530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.388439 (+168608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.388740 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480}
I20260709 14:51:59.403577  2611 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:51:59.404552  4795 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.327741  4881 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.354665  3412 leader_election.cc:403] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:51:59.356194  2605 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 14:51:59.328948  4894 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.362877  4859 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:51:59.444047  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 122020us
I20260709 14:51:59.446189  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:51:59.405010  4723 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:51:59.448771  4723 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:59.405086  2611 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:51:59.339175  4637 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:51:59.451426  4266 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:51:59.406042  4891 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:59.295830  4905 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.452701  4905 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:59.358155  4658 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.358597  4862 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 14:51:59.337478  4500 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:59.384851  4636 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.413883  4796 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:59.466557  4796 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:51:59.467605  4796 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:59.469159  4796 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:59.416160  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:59.418321  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:59.472540  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.473213  2678 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.418926  4641 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:59.299301  4857 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:59.340543  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:59.401862  3540 rpcz_store.cc:269] 0709 14:51:57.210560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.210701 (+   141us) service_pool.cc:224] Handling call
0709 14:51:59.400835 (+2190134us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003df9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.400993 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2865546112}
I20260709 14:51:59.483099  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:51:59.390471  2604 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 1. Current term is 2. Ops: []
W20260709 14:51:59.389252  2609 rpcz_store.cc:269] 0709 14:51:57.803556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.859648 (+ 56092us) service_pool.cc:224] Handling call
0709 14:51:59.323707 (+1464059us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54002feed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.323847 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484609792}
I20260709 14:51:59.485399  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:59.434928  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:51:59.487823  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.488536  2679 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.358062  2681 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:51:59.352830  4868 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.008s
I20260709 14:51:59.475858  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:51:59.395963  4852 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:59.385524  2785 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:39575 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:39575 timed out after 15.000s (SENT)
I20260709 14:51:59.352612  4815 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.473229  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:51:59.358935  2685 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
W20260709 14:51:59.475062  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 121154us
I20260709 14:51:59.392210  4876 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.391208  4901 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:59.397919  4883 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.507753  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.521781  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.483830  4880 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.396076  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.499220  4904 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:51:59.392333  4903 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:51:59.431581  4825 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.450217  4723 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.499365  4835 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:51:59.527554  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 187154us
I20260709 14:51:59.528198  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:51:59.518352  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45198.8us
I20260709 14:51:59.551965  2685 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:51:59.531643  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.505951  4266 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:59.560868  3546 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:51:59.528625  4637 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:51:59.370462  4663 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.535584  4904 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.599795  4904 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.600154  4663 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.602181  4719 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:59.553660  2681 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:51:59.558211  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.560150  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.606244  3543 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:51:59.537744  4828 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:51:59.506639  4909 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:51:59.510634  4876 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:59.563552  3540 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:59.610227  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.502274  4675 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 14:51:59.567821  3547 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:51:59.613299  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.613976  3545 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:59.597396  4641 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.582028  4637 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:51:59.615929  4802 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.001s
I20260709 14:51:59.616271  4637 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.616503  4802 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:51:59.619544  4862 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:51:59.619639  4859 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:59.621341  4862 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:51:59.622321  4859 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:51:59.614224  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.624517  3547 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:51:59.517396  4852 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.570626  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.571074  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 324) took 2684 ms. Trace:
I20260709 14:51:59.635201  2606 rpcz_store.cc:276] 0709 14:51:56.886474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.748289 (+1861815us) service_pool.cc:224] Handling call
0709 14:51:59.215427 (+467138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.397043 (+181616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.570819 (+173776us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54001ead50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.571006 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073445632}
I20260709 14:51:59.572069  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.636930  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.637292  3536 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:59.573777  3538 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:51:59.574438  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.579263  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 932) took 1870 ms. Trace:
W20260709 14:51:59.582111  4890 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.027s	sys 0.024s
I20260709 14:51:59.642931  3541 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:59.547991  4620 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 14:51:59.550472  2677 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:51:59.603034  4891 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:51:59.657927  4658 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:59.658149  4891 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.660652  4891 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.661545  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.662315  3547 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 2.
I20260709 14:51:59.506678  4757 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.603631  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.689193  2685 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:59.528918  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:51:59.699126  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 327) took 2811 ms (client timeout 1516 ms). Trace:
I20260709 14:51:59.504294  4889 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 14:51:59.484372  4894 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.567675  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 344) took 3464 ms. Trace:
I20260709 14:51:59.740279  2753 rpcz_store.cc:276] 0709 14:51:56.103676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.549269 (+1445593us) service_pool.cc:224] Handling call
0709 14:51:57.549813 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.001387 (+1451574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.567288 (+565901us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54000d0f54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:59.567557 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3327729920}
I20260709 14:51:59.742452  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.744228  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.569461  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:51:59.628813  2679 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:51:59.746963  2679 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:51:59.638929  2608 rpcz_store.cc:276] 0709 14:51:57.709187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.063957 (+1354770us) service_pool.cc:224] Handling call
0709 14:51:59.064311 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.337502 (+273191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.566667 (+229165us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002cdb54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:51:59.579156 (+ 12489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468708736}
I20260709 14:51:59.612924  4883 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:51:59.584095  4905 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.608764  4909 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:51:59.654064  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.655377  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.544153  4880 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.657358  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.523319  4907 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 14:51:59.664258  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.668955  2678 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:51:59.767738  2679 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:59.671060  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.672462  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.770617  2759 raft_consensus.cc:2376] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5. Re-sending same reply.
I20260709 14:51:59.730211  4757 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.692063  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.779347  3541 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:51:59.780803  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.782298  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.693147  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:51:59.542074  4899 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:51:59.529111  4901 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:51:59.560667  4825 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:51:59.698500  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.698457  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:51:59.699589  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:51:59.807945  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108450us
I20260709 14:51:59.808283  4875 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.809157  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:51:59.699610  2610 rpcz_store.cc:269] 0709 14:51:56.887315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:56.891854 (+  4539us) service_pool.cc:224] Handling call
0709 14:51:57.947274 (+1055420us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:51:59.698902 (+1751628us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.699029 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315321984}
I20260709 14:51:59.808323  4894 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:51:59.811304  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.812603  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.813325  2610 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:51:59.812637  4903 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:51:59.735613  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.814791  2683 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:51:59.737129  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.737267  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:51:59.816025  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78914.9us
I20260709 14:51:59.816147  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.816473  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.816813  2758 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:59.640864  4641 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:51:59.738729  3538 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: []
I20260709 14:51:59.832768  2685 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:51:59.748243  2681 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:51:59.642516  4858 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:51:59.637174  4828 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:51:59.650184  4589 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:51:59.751222  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.850626  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:51:59.755124  2755 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:51:59.851539  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.758352  3546 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:51:59.758898  3545 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:59.763016  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.762897  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.854979  2608 raft_consensus.cc:2368] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:51:59.691138  2752 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:51:59.650769  4776 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 14:51:59.735621  4620 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 14:51:59.753366  4801 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:59.786954  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 340867us
I20260709 14:51:59.632150  4862 consensus_queue.cc:1243] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:51:59.736123  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:51:59.814538  2610 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:51:59.822358  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:51:59.761329  4909 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:51:59.841500  4905 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.817199  2756 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:51:59.771529  4867 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.836979  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:51:59.939872  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103046us
W20260709 14:51:59.867964  4397 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 14:51:59.845093  2681 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:51:59.847280  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:51:59.943476  2759 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:51:59.848831  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 929) took 2140 ms. Trace:
I20260709 14:51:59.945086  2612 rpcz_store.cc:276] 0709 14:51:57.708179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.794088 (+1085909us) service_pool.cc:224] Handling call
0709 14:51:59.700511 (+906423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.700779 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:59.700820 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:51:59.700889 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:51:59.700936 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:51:59.701016 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:51:59.848551 (+147535us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54001eb250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.848744 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083498240}
I20260709 14:51:59.864204  4858 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:51:59.763913  4889 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 14:51:59.862782  3542 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:51:59.850476  4589 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:51:59.792201  4911 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:51:59.868897  2753 raft_consensus.cc:2368] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:51:59.878409  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:51:59.885596  2606 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:51:59.888252  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:59.892625  3546 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:51:59.896704  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:51:59.900305  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:59.902227  2608 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
W20260709 14:51:59.921180  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:51:59.838275  4868 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:51:59.823539  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:51:59.832738  4641 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:51:59.934795  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 954) took 3018 ms (client timeout 1999 ms). Trace:
I20260709 14:51:59.857795  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:51:59.948783  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:59.949353  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:59.868245  4859 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:51:59.953766  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 579615us
I20260709 14:52:00.016657  3411 leader_election.cc:400] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:00.017089  3411 leader_election.cc:403] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 14:51:59.852336  2677 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:52:00.022883  2755 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:00.025864  3546 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 14:52:00.032841  3546 pending_rounds.cc:85] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
I20260709 14:52:00.033246  3546 pending_rounds.cc:107] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:52:00.033608  3546 raft_consensus.cc:2889] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:00.028931  3547 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 2: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:51:59.860240  4500 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:00.020376  2607 rpcz_store.cc:269] 0709 14:51:56.916737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:57.235766 (+319029us) service_pool.cc:224] Handling call
0709 14:51:59.934466 (+2698700us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:51:59.934676 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802185088}
I20260709 14:52:00.018343  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:00.031206  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81992us
I20260709 14:52:00.035434  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:00.046063  3546 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:00.069588  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120895us
I20260709 14:52:00.071234  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:00.071807  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:51:59.952314  4577 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:00.072470  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:51:59.967381  4856 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:00.013082  4911 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:51:59.936211  4801 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:51:59.957719  4789 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.952601  4776 consensus_queue.cc:1243] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:51:59.952872  4867 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:00.097388  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.019s	sys 0.028s
I20260709 14:52:00.019760  4884 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.035252  4797 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.011180  4858 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:00.030444  4798 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:00.031237  4868 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.085824  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.029974  4802 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:00.095312  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:52:00.104085  4795 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:51:59.951933  4905 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:00.122589  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50206.7us
W20260709 14:52:00.146941  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.019s	sys 0.028s
W20260709 14:52:00.169003  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83283.7us
W20260709 14:52:00.077790  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 494803us
I20260709 14:52:00.099017  4901 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.154634  3546 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
W20260709 14:52:00.144726  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.016s	sys 0.034s
I20260709 14:52:00.040958  4859 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.155126  4789 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.166563  4903 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.194509  4577 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.204034  4798 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 14:52:00.196739  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.015s	sys 0.045s
I20260709 14:52:00.204659  4500 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.204798  4798 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:00.122383  4711 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:52:00.205888  4711 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:00.031239  4828 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.207131  4711 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.207767  4757 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:00.169178  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 693398us
W20260709 14:52:00.211851  4913 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 14:52:00.212464  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:00.212994  4846 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.940s	user 0.004s	sys 0.007s
I20260709 14:52:00.194128  4795 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:00.213783  4919 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 14:52:00.213644  4846 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:00.214679  4855 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.890s	user 0.008s	sys 0.003s
I20260709 14:52:00.147886  4641 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:00.196385  4636 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.733s	user 0.001s	sys 0.011s
W20260709 14:52:00.197862  4796 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.002s
I20260709 14:52:00.193703  4802 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:00.169986  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.150518  4658 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.215212  4855 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.225847  4893 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:00.227324  4855 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:00.228344  4756 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:00.227933  4858 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:00.228505  4266 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.000s
I20260709 14:52:00.228796  4756 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:00.229214  4266 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:00.229722  4756 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.232412  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 328) took 1140 ms. Trace:
I20260709 14:52:00.158623  4797 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.233040  4577 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:00.233805  4577 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:00.234045  4905 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.234205  4577 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.234738  4577 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:00.216048  2680 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:00.232874  2679 rpcz_store.cc:276] 0709 14:51:59.092222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.753562 (+661340us) service_pool.cc:224] Handling call
0709 14:51:59.767502 (+ 13940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:59.768189 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.232200 (+464011us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b54002b2850. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:00.232347 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31257856}
I20260709 14:52:00.222978  2758 raft_consensus.cc:606] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.243330  2758 raft_consensus.cc:614] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:00.124248  4856 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:00.223757  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53866.7us
I20260709 14:52:00.250870  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:00.252437  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.260512  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:00.321398  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60978.8us
I20260709 14:52:00.234671  4797 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.122861  4675 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 14:52:00.205178  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.016s	sys 0.045s
I20260709 14:52:00.075948  4562 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:52:00.337349  4562 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.337862  4562 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:00.238134  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:00.343744  2679 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:00.124439  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 334) took 2976 ms. Trace:
I20260709 14:52:00.272185  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:00.216363  4878 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.148s	user 0.015s	sys 0.000s
I20260709 14:52:00.348590  4878 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.351620  4878 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:00.170527  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.016s	sys 0.034s
I20260709 14:52:00.215940  4641 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:00.172986  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.227314  4893 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:00.355756  4893 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.357043  4893 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:00.167786  4868 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:00.358693  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:00.361892  4915 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 14:52:00.348914  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 899) took 1274 ms. Trace:
I20260709 14:52:00.363008  2686 rpcz_store.cc:276] 0709 14:51:59.074796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.076892 (+  2096us) service_pool.cc:224] Handling call
0709 14:52:00.271940 (+1195048us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:00.348809 (+ 76869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42861824}
I20260709 14:52:00.348042  2603 rpcz_store.cc:276] 0709 14:51:57.129281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:58.785062 (+1655781us) service_pool.cc:224] Handling call
0709 14:51:58.785385 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:51:58.785612 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:51:59.175317 (+389705us) log.cc:844] Serialized 14 byte log entry
0709 14:52:00.013698 (+838381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:00.034803 (+ 21105us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:00.039680 (+  4877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.076599 (+ 36919us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:00.105887 (+ 29288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21162,"prepare.queue_time_us":553492,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":176473,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5562,"thread_start_us":1020205,"threads_started":3,"wal-append.queue_time_us":556141}
I20260709 14:52:00.362216  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:00.366052  2677 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:00.224234  4880 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.396008  4734 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.396652  4734 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:00.223024  4911 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.400069  4911 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:52:00.368296  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 195426us
I20260709 14:52:00.221132  4802 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:00.403198  4734 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.224601  4825 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.369966  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 333) took 1268 ms. Trace:
I20260709 14:52:00.404201  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 815) took 1117 ms. Trace:
I20260709 14:52:00.404011  2683 rpcz_store.cc:276] 0709 14:51:59.101141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.399940 (+298799us) service_pool.cc:224] Handling call
0709 14:52:00.369549 (+969609us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:00.369866 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951148928}
I20260709 14:52:00.404844  2680 rpcz_store.cc:276] 0709 14:51:59.289100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.368530 (+ 79430us) service_pool.cc:224] Handling call
0709 14:52:00.403926 (+1035396us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:00.404122 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912987904}
I20260709 14:52:00.405831  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.215883  4795 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.271628  4905 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:00.407037  4905 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.268208  4786 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:00.407903  4786 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:00.408941  4786 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.410360  4786 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:00.206219  4798 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.411590  4859 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.151252  4589 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.413075  4589 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:00.224246  4867 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.156982  4852 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.276152  4901 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:00.272424  4756 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:00.332454  4850 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.005s	sys 0.016s
I20260709 14:52:00.381836  4856 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.439109  2752 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:00.382578  4663 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.782s	user 0.013s	sys 0.001s
I20260709 14:52:00.153985  4884 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:00.383424  4876 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.004s
I20260709 14:52:00.383553  4906 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:00.425551  4901 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:00.384521  4835 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.005s
W20260709 14:52:00.385317  4883 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.000s
W20260709 14:52:00.389258  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.390779  4532 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.392285  4801 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:00.216101  4903 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 14:52:00.333760  4914 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:00.393829  4789 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:00.402591  4890 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:00.250722  4729 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.405915  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.251223  4500 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.415272  2683 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:00.371285  2751 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:00.407505  4905 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:00.419926  4880 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 14:52:00.420820  2679 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:00.431627  2433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 158) took 16294 ms (client timeout 15000 ms). Trace:
W20260709 14:52:00.424204  4875 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.001s
I20260709 14:52:00.423237  4675 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 14:52:00.375913  4815 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:52:00.423934  4904 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.824s	user 0.009s	sys 0.003s
W20260709 14:52:00.420796  4637 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.804s	user 0.005s	sys 0.007s
I20260709 14:52:00.441046  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.445124  3543 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:00.448057  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 474037us
I20260709 14:52:00.144791  4776 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.224s
I20260709 14:52:00.432938  4852 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:00.481454  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.000s	sys 0.044s
W20260709 14:52:00.585585  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.000s	sys 0.045s
I20260709 14:52:00.585726  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:00.483423  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.479362  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:00.467510  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:00.482445  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:00.456833  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.491019  2682 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 14:52:00.492449  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.460110  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:00.512156  2754 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:00.512883  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:00.514386  3538 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:00.511401  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:00.518471  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:00.519083  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:00.451958  4917 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:00.525847  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.544566  4855 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:00.649391  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.454470  4876 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:00.515738  4909 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.004s
I20260709 14:52:00.444128  4620 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 14:52:00.455998  4906 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:00.546475  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 509172us
I20260709 14:52:00.501780  4835 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8.
W20260709 14:52:00.712304  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61166.2us
I20260709 14:52:00.713896  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:00.519613  4881 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.036s	user 0.014s	sys 0.000s
I20260709 14:52:00.557214  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:00.518429  4914 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:00.578207  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 137294us
I20260709 14:52:00.490721  4901 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:00.651225  2755 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:00.651518  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 73225.9us
I20260709 14:52:00.588721  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:00.738283  2607 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:00.589296  2760 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:00.558490  4875 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:00.653949  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:00.586266  4846 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.745456  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.655228  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:00.748991  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:00.749858  3546 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:00.558117  4789 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 14:52:00.656817  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.656685  4917 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.652405  4852 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.510289  4532 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:00.654646  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.591231  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:00.767436  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:00.767917  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.768184  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:00.677054  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.700320  2753 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:00.658921  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:00.771651  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57839us
I20260709 14:52:00.772212  2604 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:00.701906  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 241902us
I20260709 14:52:00.773360  3412 leader_election.cc:400] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:00.706883  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:00.715485  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 158390us
I20260709 14:52:00.774796  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:00.659184  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:00.720589  2754 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:00.586895  4734 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.726496  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.731436  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.783488  2683 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:00.711807  4899 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:00.531409  4500 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.589226  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:00.532589  2433 rpcz_store.cc:269] 0709 14:51:44.120057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:44.120270 (+   213us) service_pool.cc:224] Handling call
0709 14:52:00.413905 (+16293635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:51:53.715220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:51:53.723235 (+  8015us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:51:53.723328 (+    93us) write_op.cc:435] Acquired schema lock
0709 14:51:53.723373 (+    45us) tablet.cc:662] Decoding operations
0709 14:51:53.724772 (+  1399us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:51:53.724887 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 14:51:53.724929 (+    42us) tablet.cc:685] Acquiring locks for 8 operations
0709 14:51:53.725959 (+  1030us) tablet.cc:701] Row locks acquired
0709 14:51:53.726020 (+    61us) write_op.cc:260] PREPARE: finished
0709 14:51:53.726304 (+   284us) write_op.cc:270] Start()
0709 14:51:53.726576 (+   272us) write_op.cc:276] Timestamp: P: 1783608713726196 usec, L: 0
0709 14:51:53.726607 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 14:51:53.898156 (+171549us) log.cc:844] Serialized 3257 byte log entry
0709 14:51:54.844410 (+946254us) op_driver.cc:464] REPLICATION: finished
0709 14:51:55.096310 (+251900us) write_op.cc:301] APPLY: starting
0709 14:51:55.103673 (+  7363us) tablet.cc:1367] starting BulkCheckPresence
0709 14:51:55.150975 (+ 47302us) tablet.cc:1370] finished BulkCheckPresence
0709 14:51:55.151177 (+   202us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:51:55.252371 (+101194us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:51:55.338967 (+ 86596us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 14:51:55.339019 (+    52us) write_op.cc:312] APPLY: finished
0709 14:51:55.339429 (+   410us) log.cc:844] Serialized 64 byte log entry
0709 14:51:55.339699 (+   270us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:55.339849 (+   150us) write_op.cc:454] Released schema lock
0709 14:51:55.339919 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 14:51:55.340473 (+   554us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:51:55.340516 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1053,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":195276,"apply.run_cpu_time_us":6406,"apply.run_wall_time_us":263713,"num_ops":8,"prepare.queue_time_us":793592,"prepare.run_cpu_time_us":42497,"prepare.run_wall_time_us":526881,"raft.queue_time_us":258143,"raft.run_cpu_time_us":30681,"raft.run_wall_time_us":191823,"replication_time_us":1094008,"thread_start_us":817080,"threads_started":4,"wal-append.queue_time_us":401304,"wal-append.run_cpu_time_us":89698,"wal-append.run_wall_time_us":1989265}]]}
I20260709 14:52:00.760813  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:00.761333  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:00.739933  2435 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:00.586755  4904 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.736805  4914 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:00.792594  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:00.764539  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:00.825208  2754 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 14:52:00.825663  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61228.5us
I20260709 14:52:00.763809  2755 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:00.765364  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:00.826489  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:00.827350  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62081.3us
I20260709 14:52:00.827951  2755 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:00.531196  4729 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:00.754988  4938 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 14:52:00.743294  4801 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:00.819283  2751 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:00.812844  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58468.3us
I20260709 14:52:00.757265  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.820612  3538 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:00.820502  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.821106  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:00.821179  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:00.786687  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 14) took 1078 ms. Trace:
I20260709 14:52:00.658789  4855 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:00.759050  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:00.761058  4757 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.621577  4857 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.753532  4789 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:00.809978  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:00.847808  2683 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:00.847997  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74295.7us
I20260709 14:52:00.848356  2604 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:00.854380  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:00.848789  2651 leader_election.cc:400] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5.
I20260709 14:52:00.769393  4643 raft_consensus.cc:2764] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:52:00.858893  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:00.838656  2612 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:00.860859  2759 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:00.848879  2758 rpcz_store.cc:276] 0709 14:51:59.736002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.817533 (+ 81531us) service_pool.cc:224] Handling call
0709 14:52:00.778547 (+961014us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:00.778684 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482185472}
W20260709 14:52:00.763253  4912 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.678s	user 0.004s	sys 0.000s
I20260709 14:52:00.824607  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:00.861907  2686 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:00.852173  3546 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:00.862727  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:00.844442  4620 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:00.863920  4620 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.864297  4620 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:00.782929  4907 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:00.868392  4928 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 14:52:00.857410  4658 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 14:52:00.736527  4881 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:00.871498  2678 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:00.871896  4815 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:00.786440  4663 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.872627  4815 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:00.873769  4663 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:00.875321  4923 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.437s	user 0.001s	sys 0.000s
I20260709 14:52:00.876458  4876 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:00.807729  4731 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.782990  4798 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.787487  4875 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:00.908968  4875 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:00.909739  4917 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.713946  4909 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:00.879446  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54941.9us
I20260709 14:52:00.894892  4899 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:00.912205  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 329) took 1819 ms. Trace:
I20260709 14:52:00.911962  4899 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.912655  2678 rpcz_store.cc:276] 0709 14:51:59.092442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.766981 (+674539us) service_pool.cc:224] Handling call
0709 14:51:59.794957 (+ 27976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.872042 (+1077085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.911868 (+ 39826us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:00.912130 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2475002112}
I20260709 14:52:00.914885  3538 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:00.844650  4858 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:00.591965  4637 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.882886  3546 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 14:52:00.919443  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 93010us
I20260709 14:52:00.849309  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:00.872637  4881 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:00.895354  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48632.1us
I20260709 14:52:00.934588  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.936233  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.898852  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:00.859258  4641 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.902588  4729 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.783331  4878 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 14:52:00.902935  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 908) took 1760 ms. Trace:
I20260709 14:52:00.940425  2686 rpcz_store.cc:276] 0709 14:51:59.142300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.364152 (+1221852us) service_pool.cc:224] Handling call
0709 14:52:00.364620 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.862538 (+497918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.902599 (+ 40061us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54002b3254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:00.902863 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142959360}
W20260709 14:52:00.851368  4921 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
I20260709 14:52:00.851939  4734 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:00.761368  4562 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:00.904340  4852 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:00.905923  4855 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:00.906221  4868 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 14:52:00.881211  4857 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.937536  4867 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:00.908190  4663 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:00.910096  4875 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:00.879664  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69800.6us
W20260709 14:52:00.914780  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60512.4us
I20260709 14:52:00.985291  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:00.944546  4884 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:01.003307  2604 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:52:00.845328  4801 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.837991  4906 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:00.913221  4795 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.886181  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:00.927939  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 688) took 1189 ms. Trace:
W20260709 14:52:00.918462  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 161307us
I20260709 14:52:01.012707  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.014134  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.011898  3536 rpcz_store.cc:276] 0709 14:51:59.738381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.015172 (+276791us) service_pool.cc:224] Handling call
0709 14:52:00.407591 (+392419us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:00.407845 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.408336 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:00.408377 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:00.408942 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 14:52:00.660002 (+251060us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:00.660065 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.660173 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:00.913847 (+253674us) raft_consensus.cc:1685] finished
0709 14:52:00.913871 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:00.927842 (+ 13971us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3999,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":11072896,"thread_start_us":249882,"threads_started":1,"wal-append.queue_time_us":489597}
I20260709 14:52:00.977054  4918 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.928774  3538 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:00.808822  4924 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
I20260709 14:52:00.949309  3540 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:00.950322  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:01.022090  2753 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:01.022444  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:00.950663  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:00.951176  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:01.023603  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:00.894701  4835 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.952852  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:01.027136  2678 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:00.953949  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:01.028802  2755 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:00.913497  4589 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:00.952256  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.961395  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:00.965112  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:00.961519  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:00.975220  2759 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:00.931648  4858 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:00.906780  4797 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:00.878554  4757 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.979786  4868 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.044735  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:01.045580  2678 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:01.045938  4868 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.931137  4917 raft_consensus.cc:2764] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:01.028980  4906 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:01.052081  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.052114  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:01.054010  2756 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:00.917642  4939 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.056074  4939 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.937493  4643 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.059315  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 332) took 1961 ms. Trace:
I20260709 14:52:01.059551  4643 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:01.060048  3543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 588) took 1484 ms (client timeout 1693 ms). Trace:
W20260709 14:52:01.060655  3543 rpcz_store.cc:269] 0709 14:51:59.575419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.608315 (+ 32896us) service_pool.cc:224] Handling call
0709 14:52:01.059786 (+1451471us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:01.059972 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855759104}
I20260709 14:52:01.061995  4643 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:01.062394  4723 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:52:00.937512  4894 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:01.063949  4894 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:00.877931  4532 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.986745  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:00.987232  3542 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:00.987365  4855 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 14:52:01.059783  2685 rpcz_store.cc:276] 0709 14:51:59.097954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.856721 (+758767us) service_pool.cc:224] Handling call
0709 14:51:59.857234 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:59.869422 (+ 12188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:59.869468 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:59.901273 (+ 31805us) log.cc:844] Serialized 26 byte log entry
0709 14:52:00.494483 (+593210us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:00.494548 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.494669 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:01.059028 (+564359us) raft_consensus.cc:1685] finished
0709 14:52:01.059057 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:01.059238 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11630,"raft.queue_time_us":12408,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":62321,"thread_start_us":418910,"threads_started":1,"wal-append.queue_time_us":1036977}
I20260709 14:52:01.064927  4894 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:00.945130  4641 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.069176  4939 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:01.069936  4876 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:01.070345  4641 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:00.874593  4815 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:01.070708  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 118515us
I20260709 14:52:01.070809  4924 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:01.071463  4924 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:01.071862  4924 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:01.072767  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:00.884080  2435 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 7, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:01.075116  2435 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:01.079185  4878 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:01.080461  4904 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:01.022318  4884 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 14:52:01.081343  4878 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 14:52:01.082227  4884 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:01.046613  2678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7584380efde04ee289b42acaea2e4742"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:52:01.083133  2678 raft_consensus.cc:606] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:01.083607  4909 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:01.039144  2603 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 14:52:01.084368  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 97718.9us
I20260709 14:52:01.085353  3409 leader_election.cc:400] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:52:01.085669  3409 leader_election.cc:403] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 14:52:01.085988  4786 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.006s
I20260709 14:52:01.086635  2685 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:01.087347  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 823) took 1623 ms. Trace:
I20260709 14:52:01.006582  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:00.915092  4731 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.088481  4835 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3.
I20260709 14:52:01.078620  4857 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:01.090114  4731 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:00.920241  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63085us
I20260709 14:52:01.016567  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:01.040657  4858 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:01.091149  4577 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.856s	user 0.009s	sys 0.004s
I20260709 14:52:01.091261  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:01.091491  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 75049.7us
I20260709 14:52:01.091691  4577 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.092374  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:01.083647  2678 raft_consensus.cc:614] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:00.936288  4932 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 14:52:01.095311  4719 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:01.095408  4932 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:00.951951  4890 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:52:01.007382  2677 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.002s
I20260709 14:52:01.096446  2677 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
W20260709 14:52:01.096611  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45866.7us
I20260709 14:52:01.021600  4795 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:01.097800  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:01.098057  2606 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 14:52:01.098544  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:01.008685  4850 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 14:52:01.099802  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:01.100739  3536 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:01.102113  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:01.034085  2686 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:01.102887  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:01.103302  4904 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:01.103524  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:01.104279  4890 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:01.104817  4890 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:01.104918  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:01.106102  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:01.105794  4890 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:01.009142  4636 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.107264  4855 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:01.107558  4636 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.078204  2435 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 5, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:01.034410  4867 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.109122  4867 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:01.109889  4636 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:01.111011  2435 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:52:01.111835  3538 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:52:01.112673  4757 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:01.112614  2435 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:01.115051  4917 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:01.115620  4917 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:01.116544  4917 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:01.117022  3546 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:01.006820  4663 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:01.125593  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:01.126912  3538 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:01.008941  4801 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:01.035095  4589 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:01.043210  4797 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:01.043983  4911 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.005s
I20260709 14:52:01.000785  4562 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:01.087793  4891 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:01.082775  4884 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:01.095916  2678 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:01.097611  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:01.198055  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 824) took 1733 ms. Trace:
W20260709 14:52:01.198352  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100864us
I20260709 14:52:01.099671  4903 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:52:01.203385  4849 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:01.203648  4795 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:01.094678  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:01.203985  4849 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:01.144430  4797 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:01.118815  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:01.036276  2683 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:01.120532  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:01.122009  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:01.123986  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:00.978475  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:01.128551  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:01.046252  4914 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.134987  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.136304  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:01.114696  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:01.236341  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 118117us
W20260709 14:52:01.239491  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 261084us
I20260709 14:52:01.135951  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:01.136832  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:01.135035  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:01.096131  4719 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:01.145399  2606 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:52:01.087689  2685 rpcz_store.cc:276] 0709 14:51:59.463876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.067966 (+1604090us) service_pool.cc:224] Handling call
0709 14:52:01.086107 (+ 18141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:01.087075 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.087283 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:01.146982  2757 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:01.136184  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.058112  4905 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.175537  4867 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:01.132972  4907 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 14:52:01.183404  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:01.189765  4927 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.401s	user 0.000s	sys 0.003s
I20260709 14:52:01.193928  4589 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:01.195257  4562 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:01.199831  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:01.204713  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110145us
I20260709 14:52:01.114193  2435 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:01.135592  4577 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:01.198685  2678 rpcz_store.cc:276] 0709 14:51:59.464168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.095117 (+1630949us) service_pool.cc:224] Handling call
0709 14:52:01.095529 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:01.197697 (+102168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.197980 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:01.206974  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 903) took 2117 ms (client timeout 2750 ms). Trace:
I20260709 14:52:01.211764  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.229909  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 814) took 1963 ms. Trace:
I20260709 14:52:01.230660  2755 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:01.214938  3536 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:01.234066  2760 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:01.236341  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 115897us
I20260709 14:52:01.182112  4620 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:01.857851  4857 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:01.411808  2679 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:01.350560  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:01.861693  3540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 589) took 2286 ms (client timeout 1862 ms). Trace:
I20260709 14:52:01.861953  2759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a57a97528f31400dbc0900e76292b021"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:01.863682  4903 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.478286  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 907) took 2337 ms. Trace:
W20260709 14:52:01.237530  4756 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.804s	user 0.014s	sys 0.000s
I20260709 14:52:01.206255  4795 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:01.414093  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 173267us
I20260709 14:52:01.264883  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:01.866384  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 619268us
I20260709 14:52:01.242676  4675 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:01.753711  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 340425us
I20260709 14:52:01.868167  4789 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:01.377748  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.945987  2605 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:01.946249  2611 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 14:52:01.201632  4893 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.839s	user 0.010s	sys 0.006s
I20260709 14:52:01.753911  4889 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:01.961081  2435 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:52:01.246011  4719 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:01.353355  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:01.400964  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:01.883385  3537 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:01.963366  2435 catalog_manager.cc:5697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:52:01.896741  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 799) took 1429 ms (client timeout 1627 ms). Trace:
I20260709 14:52:01.964735  2685 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 4: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:01.281153  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:01.350185  4884 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:01.904592  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 704763us
I20260709 14:52:01.905711  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:01.906713  2682 rpcz_store.cc:269] 0709 14:51:59.089505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.089649 (+   144us) service_pool.cc:224] Handling call
0709 14:51:59.569219 (+479570us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:01.206707 (+1637488us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54000f0354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:01.206903 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790019200}
I20260709 14:52:01.907142  2683 rpcz_store.cc:276] 0709 14:51:59.266562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.031030 (+1764468us) service_pool.cc:224] Handling call
0709 14:52:01.035054 (+  4024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:01.229524 (+194470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.229834 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:01.909224  2759 raft_consensus.cc:606] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:01.969041  2759 raft_consensus.cc:614] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:01.913040  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 356) took 1324 ms. Trace:
W20260709 14:52:01.929708  4723 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.852s	user 0.013s	sys 0.008s
I20260709 14:52:01.913817  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:01.940748  4858 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.849s	user 0.020s	sys 0.001s
W20260709 14:52:01.909004  3540 rpcz_store.cc:269] 0709 14:51:59.575665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.609865 (+ 34200us) service_pool.cc:224] Handling call
0709 14:52:01.861441 (+2251576us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400550ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:01.861600 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2650935680}
I20260709 14:52:01.915127  2681 rpcz_store.cc:276] 0709 14:51:59.140447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.013003 (+872556us) service_pool.cc:224] Handling call
0709 14:52:00.013561 (+   558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.143840 (+130279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:00.143876 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:00.148728 (+  4852us) log.cc:844] Serialized 26 byte log entry
0709 14:52:00.661315 (+512587us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:00.661376 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.345712 (+684336us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:01.345820 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:01.345910 (+    90us) raft_consensus.cc:1685] finished
0709 14:52:01.345931 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:01.477543 (+131612us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39457,"raft.queue_time_us":40152,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4147,"spinlock_wait_cycles":207534336,"thread_start_us":498855,"threads_started":1,"wal-append.queue_time_us":916718}
I20260709 14:52:01.909394  3543 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:01.925325  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:01.929085  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:01.912418  4857 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:01.969877  2608 rpcz_store.cc:276] 0709 14:52:00.588610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.590804 (+  2194us) service_pool.cc:224] Handling call
0709 14:52:01.211498 (+620694us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:01.912976 (+701478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16042240}
I20260709 14:52:01.989859  4857 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:01.991092  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:01.913187  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:01.878978  4911 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:01.992233  2678 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:01.992269  4911 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:01.993670  2678 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:01.994292  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 342) took 1528 ms. Trace:
W20260709 14:52:01.964938  3542 rpcz_store.cc:269] 0709 14:52:00.467336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.467465 (+   129us) service_pool.cc:224] Handling call
0709 14:52:01.896374 (+1428909us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400224cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:01.896634 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657631488}
I20260709 14:52:01.994931  2604 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:01.994645  2678 rpcz_store.cc:276] 0709 14:52:00.466318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.993010 (+1526692us) service_pool.cc:224] Handling call
0709 14:52:01.993341 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:01.994033 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.994233 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:01.928149  4930 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.273s	user 0.006s	sys 0.010s
I20260709 14:52:01.898705  4658 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:01.927788  4901 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.157s	user 0.013s	sys 0.007s
I20260709 14:52:01.904043  2607 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:01.912613  4894 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.979955  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:01.979583  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:01.913688  4917 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:01.986001  2609 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:52:01.989048  3536 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
W20260709 14:52:01.973181  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 67542.1us
I20260709 14:52:02.008769  2682 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 14:52:02.019345  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 909) took 1367 ms. Trace:
I20260709 14:52:02.019820  2682 rpcz_store.cc:276] 0709 14:52:00.652439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.007953 (+1355514us) service_pool.cc:224] Handling call
0709 14:52:02.008402 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:02.018968 (+ 10566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.019272 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:02.008940  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 961) took 1402 ms (client timeout 1652 ms). Trace:
I20260709 14:52:02.009469  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:02.006089  2435 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:52:01.926563  4899 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:01.935621  4589 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.955324  4577 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:01.937389  4903 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:01.944445  4862 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 14:52:02.079788  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 370) took 1130 ms. Trace:
I20260709 14:52:02.159818  2604 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:02.067193  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48578.4us
I20260709 14:52:02.207963  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:02.079308  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 324) took 2988 ms. Trace:
I20260709 14:52:02.068409  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:02.021247  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:02.218998  3540 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:02.035267  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:02.219547  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 604) took 1096 ms. Trace:
I20260709 14:52:02.219753  3538 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.145118  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:02.220498  3540 rpcz_store.cc:276] 0709 14:52:01.123060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.010130 (+887070us) service_pool.cc:224] Handling call
0709 14:52:02.219481 (+209351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.221294  3543 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:02.222030  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 697) took 1015 ms. Trace:
I20260709 14:52:02.218494  2684 rpcz_store.cc:276] 0709 14:51:59.091230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.091383 (+   153us) service_pool.cc:224] Handling call
0709 14:51:59.802059 (+710676us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:51:59.820082 (+ 18023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:59.820633 (+   551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:59.820664 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:59.821372 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 14:52:00.333460 (+512088us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:00.742998 (+409538us) log.cc:844] Serialized 14 byte log entry
0709 14:52:00.780939 (+ 37941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.781161 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:01.496997 (+715836us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b240021b948. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:52:02.078949 (+581952us) raft_consensus.cc:1685] finished
0709 14:52:02.078993 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.079223 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59204,"prepare.queue_time_us":584031,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":49404,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":41493,"spinlock_wait_cycles":76107008,"thread_start_us":745807,"threads_started":2,"wal-append.queue_time_us":670028}
I20260709 14:52:01.980154  4789 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:01.971967  4855 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.083461  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:02.215091  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 376) took 1259 ms. Trace:
I20260709 14:52:02.085933  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 686) took 2384 ms. Trace:
I20260709 14:52:01.981308  4795 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.241521  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:02.021448  2682 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:02.244130  2684 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 14:52:02.245270  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 912) took 1584 ms. Trace:
I20260709 14:52:02.245820  2684 rpcz_store.cc:276] 0709 14:52:00.661218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.223061 (+1561843us) service_pool.cc:224] Handling call
0709 14:52:02.243656 (+ 20595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.244842 (+  1186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.245190 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.247542  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 911) took 1594 ms. Trace:
I20260709 14:52:02.209082  2683 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:02.248606  2682 rpcz_store.cc:276] 0709 14:52:00.658304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.020920 (+1362616us) service_pool.cc:224] Handling call
0709 14:52:02.021269 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.243242 (+221973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.247460 (+  4218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.222474  3543 rpcz_store.cc:276] 0709 14:52:01.207153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.068724 (+861571us) service_pool.cc:224] Handling call
0709 14:52:02.221964 (+153240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.264295  2681 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:52:02.263901  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:02.260561  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 919) took 1020 ms. Trace:
I20260709 14:52:02.266932  2683 rpcz_store.cc:276] 0709 14:52:01.240381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.968771 (+728390us) service_pool.cc:224] Handling call
0709 14:52:02.260421 (+291650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.239230  3541 rpcz_store.cc:276] 0709 14:51:59.653108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.789021 (+135913us) service_pool.cc:224] Handling call
0709 14:51:59.789507 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:51:59.790040 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:51:59.790083 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:51:59.790723 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 14:52:00.586190 (+795467us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:00.586237 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:00.586307 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:01.188902 (+602595us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:02.010725 (+821823us) raft_consensus.cc:1685] finished
0709 14:52:02.010759 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.037415 (+ 26656us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7715,"spinlock_wait_cycles":11828480,"thread_start_us":795048,"threads_started":1,"wal-append.queue_time_us":901013}
I20260709 14:52:02.067781  4917 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:02.272161  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 608) took 1140 ms. Trace:
I20260709 14:52:02.113765  4884 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.272652  3543 rpcz_store.cc:276] 0709 14:52:01.132232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.263569 (+1131337us) service_pool.cc:224] Handling call
0709 14:52:02.272066 (+  8497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.272894  4620 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:02.273869  4917 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.274324  3543 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:02.263990  4786 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:02.275063  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 602) took 1152 ms. Trace:
I20260709 14:52:02.208838  2758 rpcz_store.cc:276] 0709 14:52:00.949179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.949344 (+   165us) service_pool.cc:224] Handling call
0709 14:52:00.975348 (+ 26004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.975796 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:00.975831 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:00.976450 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 14:52:01.302367 (+325917us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:01.302447 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.302584 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:02.079544 (+776960us) raft_consensus.cc:1685] finished
0709 14:52:02.079573 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.079716 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19928,"raft.queue_time_us":942,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4253,"thread_start_us":293901,"threads_started":1,"wal-append.queue_time_us":1095712}
I20260709 14:52:02.264721  4795 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:02.007097  4893 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.269s
I20260709 14:52:01.994660  4911 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:02.238652  2754 rpcz_store.cc:276] 0709 14:52:00.955524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.955683 (+   159us) service_pool.cc:224] Handling call
0709 14:52:00.956109 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:00.969484 (+ 13375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:00.969523 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:00.981494 (+ 11971us) log.cc:844] Serialized 26 byte log entry
0709 14:52:01.236037 (+254543us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:01.236080 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.236155 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:02.080034 (+843879us) raft_consensus.cc:1685] finished
0709 14:52:02.080061 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.215003 (+134942us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":817,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6848,"thread_start_us":209297,"threads_started":1,"wal-append.queue_time_us":1069244}
I20260709 14:52:01.979569  4562 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:02.241415  4930 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.164s	user 0.005s	sys 0.006s
I20260709 14:52:02.276504  4562 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:02.276863  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 600) took 1158 ms. Trace:
I20260709 14:52:02.179765  4914 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.279330  3541 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:02.211499  4905 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.220547  4907 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:02.277356  3546 rpcz_store.cc:276] 0709 14:52:01.118380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.122032 (+  3652us) service_pool.cc:224] Handling call
0709 14:52:01.122393 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:01.122858 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:01.122893 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:01.155605 (+ 32712us) log.cc:844] Serialized 26 byte log entry
0709 14:52:01.562191 (+406586us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:01.562288 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:01.562405 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:02.276614 (+714209us) raft_consensus.cc:1685] finished
0709 14:52:02.276641 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.276794 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":588,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7563,"thread_start_us":384729,"threads_started":1,"wal-append.queue_time_us":1097109}
I20260709 14:52:02.211239  4500 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.280112  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 601) took 1157 ms. Trace:
I20260709 14:52:02.291038  3541 rpcz_store.cc:276] 0709 14:52:01.122249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.269030 (+1146781us) service_pool.cc:224] Handling call
0709 14:52:02.269521 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.279791 (+ 10270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.280050 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.014110  4723 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.229683  3536 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:02.296805  3546 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:02.297544  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 607) took 1170 ms. Trace:
I20260709 14:52:02.297921  3546 rpcz_store.cc:276] 0709 14:52:01.127644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.296063 (+1168419us) service_pool.cc:224] Handling call
0709 14:52:02.296524 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.297293 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.297491 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.284957  2683 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 14:52:01.978998  4849 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:02.277531  4795 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.278128  4862 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:02.281757  4914 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.275482  3543 rpcz_store.cc:276] 0709 14:52:01.122538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.273608 (+1151070us) service_pool.cc:224] Handling call
0709 14:52:02.273973 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.274765 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.274995 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.036988  4894 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:02.309752  4801 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.162s	user 0.006s	sys 0.015s
I20260709 14:52:02.309412  4857 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:02.310943  4924 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.239s	user 0.014s	sys 0.011s
W20260709 14:52:02.311730  4906 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.262s	user 0.018s	sys 0.004s
W20260709 14:52:02.312527  4797 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.065s	user 0.010s	sys 0.005s
I20260709 14:52:02.145704  4719 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:02.277608  4562 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.307935  4907 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:02.323280  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:02.222612  4577 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:02.277484  4917 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 14:52:02.336218  4849 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:02.015527  4858 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:02.347434  4941 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.406s	user 0.000s	sys 0.004s
I20260709 14:52:02.225004  4589 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:02.260072  4855 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.315243  4862 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:02.308300  4723 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.024712  4901 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.328379  4914 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.332331  4801 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.224509  4899 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:02.334589  4924 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.390751  4562 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:02.388288  4719 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:02.336741  4797 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.418857  4899 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.345467  4907 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.418815  4801 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:02.434736  4857 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.430941  4903 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.453764  4980 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:02.458004  4974 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 14:52:02.213903  2435 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:02.212188  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:02.501030  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 976) took 1087 ms. Trace:
I20260709 14:52:02.501816  2603 rpcz_store.cc:276] 0709 14:52:01.412937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.146635 (+733698us) service_pool.cc:224] Handling call
0709 14:52:02.500627 (+353992us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.504403  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:02.207897  2607 rpcz_store.cc:269] 0709 14:52:00.606775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.739305 (+132530us) service_pool.cc:224] Handling call
0709 14:52:01.183092 (+443787us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:02.008861 (+825769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7546112}
I20260709 14:52:02.506699  2608 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:02.506851  2605 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:02.134240  2610 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:02.507443  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:02.508770  2607 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
W20260709 14:52:02.508945  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.007s	sys 0.000s
I20260709 14:52:02.509440  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 365) took 1308 ms. Trace:
W20260709 14:52:02.509588  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.007s	sys 0.000s
I20260709 14:52:02.509881  2607 rpcz_store.cc:276] 0709 14:52:01.200621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.506456 (+1305835us) service_pool.cc:224] Handling call
0709 14:52:02.509346 (+  2890us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:02.510195  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 543035us
I20260709 14:52:02.520469  2677 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:02.324630  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 917) took 1084 ms. Trace:
I20260709 14:52:02.523327  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 826) took 3058 ms. Trace:
W20260709 14:52:02.524102  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.012s
W20260709 14:52:02.524749  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.012s
W20260709 14:52:02.526132  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:02.527241  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.009s
W20260709 14:52:02.527889  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.003s	sys 0.009s
W20260709 14:52:02.530166  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 307224us
W20260709 14:52:02.531633  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.009s	sys 0.014s
W20260709 14:52:02.530279  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 616510us
W20260709 14:52:02.559154  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.014s
W20260709 14:52:02.560262  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:02.560842  2723 connection.cc:256] RPC call timeout handler was delayed by -1.2165587770000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 14:52:02.536535  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:02.545656  3542 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:02.547430  3547 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:02.524019  2680 rpcz_store.cc:276] 0709 14:51:59.464682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.209423 (+1744741us) service_pool.cc:224] Handling call
0709 14:52:01.210051 (+   628us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:01.210735 (+   684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:01.210784 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:01.211753 (+   969us) log.cc:844] Serialized 52 byte log entry
0709 14:52:02.278590 (+1066837us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:02.278651 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.278757 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:02.522458 (+243701us) raft_consensus.cc:1685] finished
0709 14:52:02.522523 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.522955 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10528,"thread_start_us":420395,"threads_started":1,"wal-append.queue_time_us":1121279}
I20260709 14:52:02.522866  2683 rpcz_store.cc:276] 0709 14:52:01.239884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.267940 (+1028056us) service_pool.cc:224] Handling call
0709 14:52:02.284697 (+ 16757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.323905 (+ 39208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.324547 (+   642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.551286  2607 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:52:02.605258  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 963) took 1377 ms. Trace:
I20260709 14:52:02.609599  2607 rpcz_store.cc:276] 0709 14:52:01.188461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.519026 (+1330565us) service_pool.cc:224] Handling call
0709 14:52:02.522928 (+  3902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.562875 (+ 39947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.565297 (+  2422us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:02.444156  4890 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.007s	sys 0.019s
I20260709 14:52:02.507181  2435 catalog_manager.cc:5697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:52:02.658188  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:02.665714  4924 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:02.669059  4858 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:02.669530  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.004s	sys 0.008s
W20260709 14:52:02.671166  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.005s	sys 0.009s
I20260709 14:52:02.672648  4723 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:02.673218  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 736381us
W20260709 14:52:02.676348  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58601.6us
I20260709 14:52:02.433864  4862 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.675428  4905 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 14:52:02.673640  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83310.5us
W20260709 14:52:02.684966  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.002s	sys 0.011s
I20260709 14:52:02.687472  2680 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 14:52:02.704304  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 142755us
W20260709 14:52:02.702929  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.002s	sys 0.012s
W20260709 14:52:02.686138  4890 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.008s	sys 0.020s
I20260709 14:52:02.671378  4906 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.695050  4894 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:02.708140  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.005s	sys 0.013s
I20260709 14:52:02.707964  3538 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
W20260709 14:52:02.714272  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:02.694597  4797 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:02.691115  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:02.692834  4907 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.693130  4901 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:02.718299  4906 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:02.719937  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 698) took 1511 ms (client timeout 1986 ms). Trace:
I20260709 14:52:02.685051  4983 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.723107  4889 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:02.671294  4914 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.724658  4577 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.749342  4975 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:52:02.751158  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60296.5us
I20260709 14:52:02.752636  4723 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:02.725762  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 969) took 1525 ms. Trace:
W20260709 14:52:02.725993  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 915) took 2063 ms (client timeout 1626 ms). Trace:
I20260709 14:52:02.726429  2435 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 4, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:02.728076  4862 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:02.720744  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.005s	sys 0.014s
W20260709 14:52:02.732234  3538 rpcz_store.cc:269] 0709 14:52:01.208088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.971901 (+763813us) service_pool.cc:224] Handling call
0709 14:52:02.719794 (+747893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.763393  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:02.738379  4899 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.753943  2604 rpcz_store.cc:276] 0709 14:52:01.199711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.245841 (+ 46130us) service_pool.cc:224] Handling call
0709 14:52:02.717601 (+1471760us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54002ff8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:02.725512 (+  7911us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66572672}
I20260709 14:52:02.754339  4797 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 14:52:02.766203  4899 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:02.767868  2604 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 7. Ops: []
I20260709 14:52:02.768790  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 965) took 1579 ms. Trace:
I20260709 14:52:02.769150  2604 rpcz_store.cc:276] 0709 14:52:01.189116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.766891 (+1577775us) service_pool.cc:224] Handling call
0709 14:52:02.767497 (+   606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.768423 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.768700 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.738720  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.739136  4894 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:02.740240  2678 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:02.741304  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:02.741549  4906 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.742913  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.744861  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:02.746025  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:02.746966  4983 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.748461  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 700) took 1465 ms. Trace:
I20260709 14:52:02.777683  3545 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:02.685742  2754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c051e44d64d14770b36c5012be981196"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:02.753527  4989 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:02.801967  4989 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:02.802655  4989 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:02.803432  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:02.806146  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:02.808715  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 687) took 3155 ms. Trace:
I20260709 14:52:02.809157  4983 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:02.809808  4855 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:02.769944  4675 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:02.811146  4675 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.799981  3543 rpcz_store.cc:276] 0709 14:52:01.252358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.322703 (+1070345us) service_pool.cc:224] Handling call
0709 14:52:02.325763 (+  3060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.326388 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.326446 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.327303 (+   857us) log.cc:844] Serialized 26 byte log entry
0709 14:52:02.670385 (+343082us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:02.670489 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.670649 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:02.700536 (+ 29887us) raft_consensus.cc:1685] finished
0709 14:52:02.700782 (+   246us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:02.717116 (+ 16334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128268,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"thread_start_us":330874,"threads_started":1,"wal-append.queue_time_us":359498}
I20260709 14:52:02.791733  4981 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.813084  4981 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:02.754205  2679 rpcz_store.cc:269] 0709 14:52:00.662187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.714763 (+ 52576us) service_pool.cc:224] Handling call
0709 14:52:02.725386 (+2010623us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:02.725672 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309115136}
I20260709 14:52:02.757886  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.815544  2755 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:02.809307  3537 rpcz_store.cc:276] 0709 14:51:59.653422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.875419 (+221997us) service_pool.cc:224] Handling call
0709 14:52:01.883053 (+2007634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:01.964534 (+ 81481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.808157 (+843623us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400550ad0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:02.808474 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4239666432}
I20260709 14:52:02.759780  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.761063  2682 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:02.817021  2751 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:52:02.817134  2760 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:02.781400  4972 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.818029  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:02.682526  4636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.009s	sys 0.009s
I20260709 14:52:02.818331  4894 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:02.818511  4972 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:02.800926  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:02.819789  4973 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.820492  4973 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:02.821604  4972 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:02.769639  4899 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.801245  2754 raft_consensus.cc:606] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:02.819214  4636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.010s	sys 0.010s
I20260709 14:52:02.822465  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.764742  2680 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:02.822543  2759 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:02.824136  2751 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:02.825515  2754 raft_consensus.cc:614] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:02.815726  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:02.779832  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:02.771543  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.827365  2684 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.776201  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 596) took 2125 ms. Trace:
I20260709 14:52:02.824795  2758 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:52:02.834013  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.830240  2604 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:02.829026  3547 rpcz_store.cc:276] 0709 14:52:00.650781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.848951 (+198170us) service_pool.cc:224] Handling call
0709 14:52:02.775687 (+1926736us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400034654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:02.776047 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3707392768}
I20260709 14:52:02.835407  2752 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:02.843822  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 966) took 1649 ms. Trace:
I20260709 14:52:02.844209  2604 rpcz_store.cc:276] 0709 14:52:01.189450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.769972 (+1580522us) service_pool.cc:224] Handling call
0709 14:52:02.770659 (+   687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.838143 (+ 67484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.838335 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.845494  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:02.856658  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:02.844900  4894 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:02.865170  2760 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:02.853226  4889 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.873028  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.876392  2680 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.860157  4855 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:02.878746  2755 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:02.880813  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:02.881422  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:02.881953  2606 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:02.882181  2754 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:02.883128  4889 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:02.884071  2608 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 978) took 1470 ms (client timeout 1629 ms). Trace:
I20260709 14:52:02.884634  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:02.884569  2608 rpcz_store.cc:269] 0709 14:52:01.413472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.990653 (+577181us) service_pool.cc:224] Handling call
0709 14:52:02.883797 (+893144us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:02.883974 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181444480}
I20260709 14:52:02.885617  4972 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:02.893441  2679 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:02.894004  2751 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 14:52:02.894428  2751 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:02.894815  2751 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 14:52:02.895133  2751 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:02.895432  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 825) took 3431 ms. Trace:
I20260709 14:52:02.895955  2677 rpcz_store.cc:276] 0709 14:51:59.464417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.097368 (+1632951us) service_pool.cc:224] Handling call
0709 14:52:01.097779 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.521489 (+1423710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.895089 (+373600us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:02.895363 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269445376}
I20260709 14:52:02.896478  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:02.897434  2751 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:52:02.897977  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:02.898546  4978 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.899408  4855 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:02.894217  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:03.208143  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 804) took 2112 ms. Trace:
I20260709 14:52:02.986527  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:03.155629  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 346) took 2666 ms. Trace:
I20260709 14:52:02.970750  2684 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:03.163986  2612 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 14:52:02.970029  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.229308  2612 pending_rounds.cc:85] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 3
I20260709 14:52:03.229977  2612 pending_rounds.cc:107] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 14:52:03.230655  2612 raft_consensus.cc:2889] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:03.035354  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 971) took 1835 ms. Trace:
I20260709 14:52:03.199898  2754 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:03.068584  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:02.906626  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:03.131548  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77394.9us
I20260709 14:52:02.912451  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:02.912693  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 166840us
I20260709 14:52:03.162386  3411 leader_election.cc:400] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:03.237257  3411 leader_election.cc:403] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 14:52:03.163552  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 350) took 3648 ms. Trace:
W20260709 14:52:03.216984  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 354) took 2574 ms (client timeout 1968 ms). Trace:
W20260709 14:52:03.203414  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 953) took 2613 ms (client timeout 2981 ms). Trace:
I20260709 14:52:02.899806  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:03.227448  2678 rpcz_store.cc:276] 0709 14:52:00.488832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.996047 (+1507215us) service_pool.cc:224] Handling call
0709 14:52:02.739514 (+743467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.835874 (+ 96360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.155267 (+319393us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b5400268750. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.155543 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684453120}
I20260709 14:52:03.189138  2604 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:03.225196  3545 rpcz_store.cc:276] 0709 14:52:01.096176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.101979 (+  5803us) service_pool.cc:224] Handling call
0709 14:52:02.777109 (+1675130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.831727 (+ 54618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.190466 (+358739us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400034650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.208025 (+ 17559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851209088}
W20260709 14:52:03.157537  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 412966us
W20260709 14:52:02.931490  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 151849us
I20260709 14:52:03.161693  4719 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.231703  2606 rpcz_store.cc:276] 0709 14:52:01.200228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.000199 (+1799971us) service_pool.cc:224] Handling call
0709 14:52:03.018452 (+ 18253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.034683 (+ 16231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:03.034735 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:03.034855 (+   120us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:03.034940 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.035042 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.035260 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:03.324882  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:03.282009  2610 rpcz_store.cc:269] 0709 14:52:00.590183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.594894 (+  4711us) service_pool.cc:224] Handling call
0709 14:52:03.173723 (+2578829us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.203190 (+ 29467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2733702400}
I20260709 14:52:03.325898  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 968) took 2108 ms. Trace:
I20260709 14:52:03.326555  2604 rpcz_store.cc:276] 0709 14:52:01.199319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.849449 (+1650130us) service_pool.cc:224] Handling call
0709 14:52:03.000074 (+150625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.304178 (+304104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.307317 (+  3139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:02.902293  4899 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:02.899900  4855 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:02.903650  4980 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:03.324693  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:03.066758  4562 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.332759  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.107151  4894 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:03.233794  4889 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.339371  2608 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 14:52:03.327325  4979 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.340643  4917 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:03.162271  4500 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.341931  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.343268  2606 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:03.191491  4960 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:03.344601  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.306425  2609 rpcz_store.cc:276] 0709 14:51:59.515053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:59.879030 (+363977us) service_pool.cc:224] Handling call
0709 14:51:59.879360 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.017595 (+2138235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.162922 (+1145327us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:03.163208 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4843596160}
I20260709 14:52:03.153903  4974 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.234026  4981 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:03.304924  2611 rpcz_store.cc:269] 0709 14:52:00.587897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:00.590688 (+  2791us) service_pool.cc:224] Handling call
0709 14:52:03.162294 (+2571606us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000ac154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.162661 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687620480}
I20260709 14:52:03.020212  4636 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:03.162269  4884 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.889s	user 0.011s	sys 0.013s
W20260709 14:52:03.351711  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 445285us
I20260709 14:52:03.341998  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:03.335875  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 980) took 1865 ms (client timeout 1656 ms). Trace:
I20260709 14:52:02.914629  4994 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:02.911084  4986 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:03.340091  2753 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:03.361318  2608 pending_rounds.cc:85] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 1
I20260709 14:52:03.154687  4637 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.398212  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:03.386891  2605 rpcz_store.cc:269] 0709 14:52:01.470464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.007948 (+537484us) service_pool.cc:224] Handling call
0709 14:52:03.322046 (+1314098us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54002ce054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.335744 (+ 13698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614316928}
I20260709 14:52:03.393222  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:03.346385  4899 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.365298  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 975) took 1952 ms. Trace:
I20260709 14:52:03.346721  4855 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:03.344166  4973 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:03.383064  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:03.297504  4675 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:02.899427  4978 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.333674  4719 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.394942  2608 pending_rounds.cc:107] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:52:03.424803  2608 raft_consensus.cc:2889] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:03.350473  4980 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:03.406050  2606 rpcz_store.cc:276] 0709 14:52:01.412705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.331766 (+1919061us) service_pool.cc:224] Handling call
0709 14:52:03.342639 (+ 10873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.364829 (+ 22190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.365198 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:03.406985  2760 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 14:52:03.406502  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:03.371210  4562 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.403563  4500 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.417052  4995 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.419243  4675 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.420374  4994 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.421638  4889 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:03.404865  4979 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:03.422945  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.009s	sys 0.019s
I20260709 14:52:03.377720  4972 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 14:52:03.389292  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 476987us
I20260709 14:52:03.427306  2760 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:03.687288  5006 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 14:52:03.687435  4901 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:03.434336  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:03.688948  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 922) took 1829 ms. Trace:
I20260709 14:52:03.444945  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 699) took 2030 ms. Trace:
W20260709 14:52:03.449806  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.009s	sys 0.020s
I20260709 14:52:03.431245  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:03.451591  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 289831us
I20260709 14:52:03.451442  2686 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:03.446434  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:03.454456  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50424.3us
I20260709 14:52:03.456691  2753 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:03.404268  4960 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:03.458317  2610 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:03.408176  4894 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.695384  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:03.696487  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:03.697263  4894 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.458626  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:03.697835  2755 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:03.698203  2606 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 2.
I20260709 14:52:03.428220  4723 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:03.701658  4723 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.682153  4994 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.702127  4723 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:03.702345  4994 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:03.702720  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.703056  4994 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.703532  2753 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:03.704564  2753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5f95289d39644bead88ff8640742871"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:03.705149  2753 raft_consensus.cc:606] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:52:03.463189  2604 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:03.707494  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 977) took 2294 ms. Trace:
I20260709 14:52:03.707945  2604 rpcz_store.cc:276] 0709 14:52:01.413189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.347694 (+1934505us) service_pool.cc:224] Handling call
0709 14:52:03.427687 (+ 79993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.707096 (+279409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.707415 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.465708  2758 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:03.471654  2609 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 14:52:03.481674  4849 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.072s	user 0.012s	sys 0.009s
I20260709 14:52:03.409957  4974 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.482462  4977 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 14:52:03.471288  4589 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:03.470642  4985 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
W20260709 14:52:03.465013  4786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.030s
I20260709 14:52:03.415062  4636 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.509240  2759 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
W20260709 14:52:03.541188  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 199307us
I20260709 14:52:03.464805  5000 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:03.450938  4995 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:03.551782  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 179728us
I20260709 14:52:03.458199  4914 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.712900  4914 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:03.475107  4903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.014s	sys 0.021s
W20260709 14:52:03.715092  4903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.014s	sys 0.026s
W20260709 14:52:03.715165  4786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.006s	sys 0.031s
I20260709 14:52:03.715655  4797 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:03.716531  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 979) took 2302 ms. Trace:
I20260709 14:52:03.486598  4893 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.716975  2609 rpcz_store.cc:276] 0709 14:52:01.413786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.464375 (+2050589us) service_pool.cc:224] Handling call
0709 14:52:03.464933 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.716093 (+251160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.716387 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.717648  4893 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:03.540194  4658 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.518s	user 0.015s	sys 0.010s
I20260709 14:52:03.511454  4719 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.475021  4905 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:03.556646  4884 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.511602  4562 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.597770  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 970) took 2397 ms. Trace:
I20260709 14:52:03.722949  2608 rpcz_store.cc:276] 0709 14:52:01.199974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.885621 (+1685647us) service_pool.cc:224] Handling call
0709 14:52:03.332128 (+446507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.474772 (+142644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.597353 (+122581us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002ce050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.597618 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261162752}
I20260709 14:52:03.605221  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 913) took 2943 ms. Trace:
I20260709 14:52:03.606509  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 340) took 3151 ms. Trace:
I20260709 14:52:03.534911  4675 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:03.727280  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.540732  4983 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:03.620275  5009 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.080s	user 0.001s	sys 0.001s
I20260709 14:52:03.642485  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:03.638305  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.008s	sys 0.028s
W20260709 14:52:03.733028  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.009s	sys 0.029s
W20260709 14:52:03.655319  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 203605us
W20260709 14:52:03.656450  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 350816us
W20260709 14:52:03.650508  4989 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.847s	user 0.012s	sys 0.006s
I20260709 14:52:03.656498  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 962) took 2468 ms. Trace:
I20260709 14:52:03.537896  4980 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:03.670264  5005 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 14:52:03.673321  5016 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.675619  4979 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:03.667505  4978 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.687978  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:03.407852  4899 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.689078  2605 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:03.445973  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:03.690826  3546 rpcz_store.cc:276] 0709 14:52:01.241668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.299021 (+1057353us) service_pool.cc:224] Handling call
0709 14:52:02.299419 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.695353 (+395934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.695410 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.696331 (+   921us) log.cc:844] Serialized 26 byte log entry
0709 14:52:02.837476 (+141145us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:02.837507 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:02.845532 (+  8025us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.056205 (+210673us) raft_consensus.cc:1685] finished
0709 14:52:03.056237 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.271964 (+215727us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147405,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":486810,"threads_started":2,"wal-append.queue_time_us":356917}
I20260709 14:52:03.699450  2754 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:03.701035  4960 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:03.700918  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:03.701943  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:03.489930  3536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.000s	sys 0.034s
I20260709 14:52:03.711387  2755 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:03.759065  3536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.001s	sys 0.052s
I20260709 14:52:03.760356  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 609) took 2625 ms. Trace:
I20260709 14:52:03.760880  3536 rpcz_store.cc:276] 0709 14:52:01.135346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.297402 (+1162056us) service_pool.cc:224] Handling call
0709 14:52:02.297701 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.718156 (+420455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.718235 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.719340 (+  1105us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.759675 (+1040335us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:03.759739 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.759921 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.759991 (+    70us) raft_consensus.cc:1685] finished
0709 14:52:03.760021 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.760270 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156168,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":58,"thread_start_us":1403978,"threads_started":2,"wal-append.queue_time_us":854216}
I20260709 14:52:03.763279  4795 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:52:03.764387  4789 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.505s	user 0.015s	sys 0.011s
I20260709 14:52:03.768245  2603 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:03.689743  2683 rpcz_store.cc:276] 0709 14:52:01.859517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.784590 (+925073us) service_pool.cc:224] Handling call
0709 14:52:02.785454 (+   864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.786012 (+   558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.786041 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.824508 (+ 38467us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.324995 (+500487us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:03.325043 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.325125 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.688673 (+363548us) raft_consensus.cc:1685] finished
0709 14:52:03.688700 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.688882 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":447303,"threads_started":1,"wal-append.queue_time_us":567147}
I20260709 14:52:03.461759  4986 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.772526  4986 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.773339  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:03.739121  5026 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:03.721719  5000 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:03.739486  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:03.730000  2611 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:03.734701  2612 rpcz_store.cc:276] 0709 14:52:01.187969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.190101 (+  2132us) service_pool.cc:224] Handling call
0709 14:52:03.162978 (+1972877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.372172 (+209194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.647066 (+274894us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54000ac150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.656426 (+  9360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4534021376}
I20260709 14:52:03.711215  4500 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.725562  2681 rpcz_store.cc:276] 0709 14:52:00.455095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:01.987871 (+1532776us) service_pool.cc:224] Handling call
0709 14:52:01.988383 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:02.266533 (+278150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.598806 (+1332273us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:03.606250 (+  7444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623991552}
I20260709 14:52:03.717760  4797 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:03.719357  4982 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.758682  2606 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.767961  4980 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:03.774485  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:03.772600  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:03.715332  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 606) took 2591 ms. Trace:
I20260709 14:52:03.772338  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:03.731856  4994 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.734189  4901 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:03.783257  3542 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:03.797557  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:03.789511  2760 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:03.793206  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.753517  4658 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:03.773515  4986 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:03.803414  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 396085us
I20260709 14:52:03.812311  3537 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:03.785254  5016 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.813314  3545 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:03.814416  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:03.733435  4889 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 14:52:03.796090  4797 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:03.786113  5000 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:03.724191  2685 rpcz_store.cc:276] 0709 14:52:00.661560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.264072 (+1602512us) service_pool.cc:224] Handling call
0709 14:52:02.295067 (+ 30995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.524435 (+229368us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:02.602646 (+ 78211us) log.cc:844] Serialized 14 byte log entry
0709 14:52:02.780394 (+177748us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800046c00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:03.208409 (+428015us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:03.304078 (+ 95669us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.304293 (+   215us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:03.304329 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.604370 (+300041us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:03.604459 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.604575 (+   116us) raft_consensus.cc:1685] finished
0709 14:52:03.604608 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.604955 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62743,"prepare.queue_time_us":78054,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":499500,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":3884,"spinlock_wait_cycles":530748160,"thread_start_us":496890,"threads_started":3,"wal-append.queue_time_us":10668}
W20260709 14:52:03.831930  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 444874us
W20260709 14:52:03.836357  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 107496us
I20260709 14:52:03.796564  3541 rpcz_store.cc:276] 0709 14:52:01.123547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.293042 (+1169495us) service_pool.cc:224] Handling call
0709 14:52:02.293516 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.663953 (+370437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.664028 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.665218 (+  1190us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.691132 (+1025914us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:03.691172 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.714994 (+ 23822us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.715046 (+    52us) raft_consensus.cc:1685] finished
0709 14:52:03.715063 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.715232 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160008,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":1393942,"threads_started":2,"wal-append.queue_time_us":871862}
I20260709 14:52:03.838390  3538 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:03.775553  4675 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.742336  4905 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.747284  4989 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:03.807818  4786 raft_consensus.cc:2764] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:03.756848  4884 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.814145  5026 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:03.793109  5023 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:03.839730  3543 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
W20260709 14:52:03.849906  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 148098us
I20260709 14:52:03.856775  4849 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:03.857793  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:03.848129  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:03.860071  4985 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:03.860175  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:03.860561  4985 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.861047  4985 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.851341  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:03.839346  4862 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.080s	user 0.013s	sys 0.006s
I20260709 14:52:03.790483  4855 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.862488  2757 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:03.863447  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:03.863703  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 991) took 1015 ms. Trace:
I20260709 14:52:03.863830  4855 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:03.864068  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:03.854216  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 973) took 2442 ms (client timeout 1937 ms). Trace:
I20260709 14:52:03.864343  4589 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:03.850567  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:03.819860  4978 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 14:52:03.883929  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 992) took 1013 ms. Trace:
I20260709 14:52:03.793882  4983 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:03.864079  2757 rpcz_store.cc:276] 0709 14:52:02.850131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.854883 (+  4752us) service_pool.cc:224] Handling call
0709 14:52:03.863499 (+1008616us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54000d0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.863641 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273050880}
I20260709 14:52:03.870333  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 964) took 2667 ms. Trace:
W20260709 14:52:03.865469  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 85163.9us
I20260709 14:52:03.871763  4562 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:03.871923   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:52:03.847116  4994 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 14:52:03.872318  4972 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:03.871819  5019 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:03.865270  4789 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:03.873598  5024 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 14:52:03.874905  4723 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:03.876127  4893 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:03.876586  5016 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.884842  2607 rpcz_store.cc:276] 0709 14:52:01.188806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.663416 (+1474610us) service_pool.cc:224] Handling call
0709 14:52:02.664293 (+   877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.665314 (+  1021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.665361 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.666701 (+  1340us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.232037 (+565336us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:03.232085 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.232163 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.855444 (+623281us) raft_consensus.cc:1685] finished
0709 14:52:03.855471 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.855636 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1014,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":2270,"thread_start_us":437616,"threads_started":1,"wal-append.queue_time_us":664994}
I20260709 14:52:03.877107  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:03.877686  4795 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:03.905565  4795 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.905721  4636 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.877941  5007 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:03.906869  2759 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:52:03.907356  5007 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:03.907595  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 887) took 1012 ms. Trace:
I20260709 14:52:03.907821  5007 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:03.908406  4983 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:03.908996  5007 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:03.908073  2759 rpcz_store.cc:276] 0709 14:52:02.895027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.890647 (+995620us) service_pool.cc:224] Handling call
0709 14:52:03.906456 (+ 15809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.907299 (+   843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.907533 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.879805  4917 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:03.911307  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.870188  2756 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:03.913012  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.913211  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1002) took 1030 ms. Trace:
W20260709 14:52:03.864605  2603 rpcz_store.cc:269] 0709 14:52:01.412128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.503532 (+1091404us) service_pool.cc:224] Handling call
0709 14:52:03.850334 (+1346802us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400280354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:03.854146 (+  3812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902838528}
I20260709 14:52:03.884398  2752 rpcz_store.cc:276] 0709 14:52:02.872505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.872696 (+   191us) service_pool.cc:224] Handling call
0709 14:52:02.879329 (+  6633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:02.879655 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:02.879686 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:02.880488 (+   802us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.347590 (+467102us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:03.347643 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.347784 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:03.883689 (+535905us) raft_consensus.cc:1685] finished
0709 14:52:03.883710 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:03.883858 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":466573,"threads_started":1,"wal-append.queue_time_us":658800}
I20260709 14:52:03.914333   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:52:03.869957  4589 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.914927  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.915743  2752 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:03.913748  2756 rpcz_store.cc:276] 0709 14:52:02.882807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.814919 (+932112us) service_pool.cc:224] Handling call
0709 14:52:03.845606 (+ 30687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.912857 (+ 67251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.913125 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.917207  2752 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:03.917235  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:03.917992  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 890) took 1021 ms. Trace:
I20260709 14:52:03.843847  4894 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:03.919270  5027 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:03.893540  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 154189us
I20260709 14:52:03.894111  3546 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:03.920300  4977 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:03.920915  4977 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:03.871973  5020 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:03.922804  5020 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.923285  5020 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:03.923834  4995 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:03.853669  4914 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:03.904909  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 156231us
I20260709 14:52:03.909653  4905 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:03.915100  4589 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.919229  4894 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.845252  4889 consensus_queue.cc:1243] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:52:03.887898  3545 raft_consensus.cc:2376] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:52:03.843070  4986 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.897851  2760 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:52:03.918396  2752 rpcz_store.cc:276] 0709 14:52:02.896116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.916550 (+1020434us) service_pool.cc:224] Handling call
0709 14:52:03.916946 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:03.917678 (+   732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.917909 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.913949  2759 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:03.933648  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.891355  4562 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.816288  4982 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:03.936304  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:03.890797  4903 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:03.891832  4972 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.951213  2755 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:03.893848  4901 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:03.979753  2681 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 14:52:03.952235  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48976.6us
I20260709 14:52:03.935014  4974 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:03.953734  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.626s (SENT) [suppressed 17 similar messages]
I20260709 14:52:03.967747  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 889) took 1071 ms. Trace:
I20260709 14:52:03.972801  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:03.935338  5007 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.922041  4977 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:03.960312  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57215.3us
I20260709 14:52:03.983058  2760 rpcz_store.cc:276] 0709 14:52:02.895791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.687580 (+791789us) service_pool.cc:224] Handling call
0709 14:52:03.967620 (+280040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:03.985709  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:03.957801  4995 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:03.982422  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:03.986698  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:03.974622  4562 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:03.998392  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 77115.6us
I20260709 14:52:03.983502  2685 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:52:03.999272  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:04.002424  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:03.979436  4893 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:04.003046  2683 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 14:52:04.003521  2760 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:03.980996  4972 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:03.979063  4901 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:03.969457  4975 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 14:52:04.013833  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:03.978650  4914 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:04.005900  5019 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:04.016438  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.017366  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.019157  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.019943  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.020289  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.020697  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.033869  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:04.021107  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:03.976032  4978 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 14:52:03.990237  4917 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:04.012984  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.028059  4723 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:04.032135  2607 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:04.036700  2605 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:04.038708  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1000) took 1147 ms. Trace:
I20260709 14:52:03.988090  5016 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 14:52:04.051954  2754 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:04.045409  4797 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:03.957854  5023 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 14:52:04.057346  3541 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
W20260709 14:52:04.060462  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83442.9us
I20260709 14:52:04.057713  2753 rpcz_store.cc:276] 0709 14:52:02.882033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.706102 (+824069us) service_pool.cc:224] Handling call
0709 14:52:03.706473 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:03.728031 (+ 21558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:03.728088 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:03.754681 (+ 26593us) log.cc:844] Serialized 26 byte log entry
0709 14:52:03.989134 (+234453us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:03.989203 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:03.989295 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:04.028872 (+ 39577us) raft_consensus.cc:1685] finished
0709 14:52:04.028900 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:04.029054 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45419,"raft.queue_time_us":20205,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":5269,"thread_start_us":154186,"threads_started":1,"wal-append.queue_time_us":233638}
I20260709 14:52:04.072485  4884 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:04.076341  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 199348us
I20260709 14:52:04.023316  4901 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:04.077114  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:04.007442  4893 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:04.108400  2751 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:04.084373  2606 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:04.086028  3545 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:04.065313  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:04.029790  4974 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 14:52:04.108263  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:04.089210  4973 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:04.052487  4995 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:04.109174  2603 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:04.091620  4903 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:04.151549  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1001) took 1269 ms. Trace:
I20260709 14:52:04.137761  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:04.135663  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 191836us
I20260709 14:52:04.030288  4983 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 14:52:04.149796  2758 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:04.152107  2751 rpcz_store.cc:276] 0709 14:52:02.882424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.802129 (+919705us) service_pool.cc:224] Handling call
0709 14:52:04.106603 (+304474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:04.108951 (+  2348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:04.151225 (+ 42274us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54000d0a50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:04.151459 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660657024}
I20260709 14:52:04.157232  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.159266  4986 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:04.087488  4917 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.163939  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:04.154397  5029 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
I20260709 14:52:04.091053  4914 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:04.138345  4989 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:04.180377  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 148621us
I20260709 14:52:04.151132  4903 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:04.177145  3545 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:04.172199  4675 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:04.189601  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50804.8us
I20260709 14:52:04.165289  5016 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:52:04.171514  5033 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:04.172739  4884 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:52:04.208564  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:04.191745  4903 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:04.218302  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 314180us
W20260709 14:52:04.222213  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 185127us
I20260709 14:52:04.222335  2683 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:04.223718  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86106.8us
I20260709 14:52:04.162117  4973 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.225718  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:04.093029  5026 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.226599  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:04.205231  4884 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:04.211223  5033 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:04.247565  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.207713  5035 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:04.239522  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:04.207871  4983 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 14:52:04.270678  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62224.4us
W20260709 14:52:04.271548  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45919us
I20260709 14:52:04.250723  4894 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:04.285717  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.209792  4985 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:04.203446  4893 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.284591  3542 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:04.289341  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 312547us
W20260709 14:52:04.291141  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64651.3us
I20260709 14:52:04.294884  4797 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:04.270344  4789 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 14:52:04.304124  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:04.308697  2753 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:04.285842  5016 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:52:04.319242  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:04.325024  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:04.325878  2760 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:04.329030  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108256us
I20260709 14:52:04.329100  2754 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:04.332437  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:04.332875  2755 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 2
I20260709 14:52:04.333206  2755 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:52:04.333527  2755 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:04.333818  4994 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:04.293726  5035 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 14:52:04.337118  2650 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.627s (SENT)
I20260709 14:52:04.337585  5035 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:04.338076  5035 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:04.291711  5039 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 14:52:04.343640  5037 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:04.344381  5037 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:04.344808  5037 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:04.328471  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.272130  4995 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.348701  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.349493  3542 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:04.209702  5019 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.358021  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:04.358412  5028 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.235s	user 0.000s	sys 0.003s
I20260709 14:52:04.359687  4914 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:04.359930  5036 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 14:52:04.338115  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:04.360296  5048 raft_consensus.cc:2764] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:52:04.360646  5042 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
W20260709 14:52:04.361512  5043 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 14:52:04.370877  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:04.363842  3536 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:04.399096  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.411636  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:04.403914  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 153959us
I20260709 14:52:04.406675  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:04.435094  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
W20260709 14:52:04.435969  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:04.321851  4797 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 14:52:04.441715  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:04.464514  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 135752us
I20260709 14:52:04.466030  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:04.467866  3542 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:04.469313  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:04.476786  5014 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 14:52:04.480208  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.481387  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.394515  4989 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:04.494302  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.524753  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.531741  2760 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:04.536209  5054 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:04.537285  5054 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:04.539184  5048 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:04.541945  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 87155.7us
I20260709 14:52:04.549800  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.491278  4986 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:04.554948  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 477890us
I20260709 14:52:04.519686  4989 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:04.563831  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:04.566165  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100273us
I20260709 14:52:04.533067  4797 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:04.489795  4917 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:04.572659  3545 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:04.562547  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:04.577715  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:04.578945  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:04.586285  4905 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:52:04.371745  4884 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:04.588063  4675 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:04.591105  4589 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.004s
W20260709 14:52:04.587555  5050 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 14:52:04.593308  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:04.547538  4894 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:04.655674  2603 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:52:04.663946  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.428277  5033 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:04.596769  4797 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:04.607435  4562 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.003s
I20260709 14:52:04.571398  4989 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:04.599211  4905 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:04.696880  5000 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.841s	user 0.014s	sys 0.000s
I20260709 14:52:04.697599  5000 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:04.701638  5000 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:04.704879  5000 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:04.707317  5000 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:04.710573  5000 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:04.651655  4973 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:04.745416  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:04.689751  4797 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:04.723668  5007 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.001s
W20260709 14:52:04.743155  4899 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.007s
I20260709 14:52:04.659224  4589 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:04.687170  4975 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 14:52:04.692898  4894 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:04.689553  5037 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:04.722958  4562 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:05.196291  2726 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.532s (SENT)
I20260709 14:52:05.252388  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:05.456593  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:05.184412  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 847330us
I20260709 14:52:05.245842   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:52:04.952240  4636 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 14:52:05.459259  2650 connection.cc:256] RPC call timeout handler was delayed by -1.1199175769999101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 14:52:05.459671  4636 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:05.459980  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0540103419999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 14:52:05.461066  3547 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:05.460804  4636 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:04.944775  5054 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:05.247995  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:05.463353  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 817) took 1120 ms. Trace:
W20260709 14:52:04.921057  3540 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.009s	sys 0.020s
I20260709 14:52:05.187299  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:04.926136  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 347262us
I20260709 14:52:04.920405  2760 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:05.346729  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 996) took 2438 ms (client timeout 1720 ms). Trace:
I20260709 14:52:05.463824  3547 rpcz_store.cc:276] 0709 14:52:04.342543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.423682 (+ 81139us) service_pool.cc:224] Handling call
0709 14:52:05.463063 (+1039381us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:05.463270 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250474240}
W20260709 14:52:04.942291  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 252835us
W20260709 14:52:04.977646  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 514169us
W20260709 14:52:04.965648  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.626s (SENT)
I20260709 14:52:05.319214  2756 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:05.468799  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:05.131959  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.021s
W20260709 14:52:05.464725  3540 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.015s	sys 0.033s
W20260709 14:52:05.469552  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.000s	sys 0.025s
W20260709 14:52:04.964987  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:05.468063  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 502442us
W20260709 14:52:05.131008  2686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.019s
W20260709 14:52:05.478629  2686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.000s	sys 0.024s
I20260709 14:52:04.813408  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:05.187299  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:05.466871  2758 rpcz_store.cc:269] 0709 14:52:02.880338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:02.891640 (+ 11302us) service_pool.cc:224] Handling call
0709 14:52:05.318344 (+2426704us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002058d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:05.318600 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419689216}
I20260709 14:52:04.936564  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:05.211651  2685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.005s	sys 0.007s
I20260709 14:52:05.335011  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:05.337981  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.011s	sys 0.011s
I20260709 14:52:05.232404  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:04.744462  4914 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 14:52:05.265447  3538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.001s	sys 0.048s
W20260709 14:52:04.868129  5060 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
W20260709 14:52:05.532445  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:05.497453  2685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.007s	sys 0.008s
I20260709 14:52:05.501161  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:05.498283  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.012s	sys 0.012s
W20260709 14:52:04.969756  5020 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.046s	user 0.014s	sys 0.000s
I20260709 14:52:05.499779  5007 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:04.724189  4905 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:52:05.449595  4972 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.397s	user 0.014s	sys 0.001s
W20260709 14:52:05.040521  5049 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.614s	user 0.000s	sys 0.002s
W20260709 14:52:05.168059  4977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.036s
W20260709 14:52:05.343075  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.012s	sys 0.010s
W20260709 14:52:05.574134  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 377864us
W20260709 14:52:05.440052  4989 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.003s
I20260709 14:52:05.495625  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:05.526649  3538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.004s	sys 0.058s
I20260709 14:52:05.585522  2677 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 14:52:05.511500  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:04.751940  5041 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:52:05.071349  5057 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 14:52:05.432238  5061 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.402s	user 0.000s	sys 0.004s
W20260709 14:52:05.574270  4977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.007s	sys 0.051s
W20260709 14:52:05.263758  5035 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.925s	user 0.013s	sys 0.000s
W20260709 14:52:04.958992  4901 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.824s	user 0.015s	sys 0.000s
W20260709 14:52:05.534210  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.21812e+06us
W20260709 14:52:05.234495  4994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.002s	sys 0.010s
I20260709 14:52:05.538174  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 856) took 1319 ms. Trace:
W20260709 14:52:05.108145  5042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.010s	sys 0.024s
I20260709 14:52:05.500451  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 710) took 1728 ms. Trace:
I20260709 14:52:05.462455  4636 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:05.592983  5041 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:05.316109  4786 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:05.585134  4972 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:05.556800  5020 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:05.584631  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:05.261421  4795 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.355s	user 0.012s	sys 0.004s
W20260709 14:52:05.244555  5047 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.938s	user 0.004s	sys 0.003s
I20260709 14:52:05.464793  4893 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:04.854396  5039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.042s	sys 0.079s
I20260709 14:52:05.545436  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:05.210891  5058 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.356s
I20260709 14:52:05.550194  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:04.887724  4723 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:05.495170  5028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.009s
I20260709 14:52:04.884960  4884 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:05.673702  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 811) took 1333 ms. Trace:
W20260709 14:52:05.494120  4995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.000s	sys 0.012s
I20260709 14:52:05.620729  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:04.816053  4985 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.067209  4982 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:05.002902  5026 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.401983  4562 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.203403  4986 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:04.744843  4917 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.787s
I20260709 14:52:05.564556  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:05.570479  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.015s	sys 0.035s
I20260709 14:52:05.315311  4980 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:52:05.719358  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.015s	sys 0.035s
W20260709 14:52:05.719900  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.29451e+06us
I20260709 14:52:05.574272  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:05.574803  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.013s	sys 0.011s
I20260709 14:52:05.578413  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:05.721638  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.47256e+06us
I20260709 14:52:05.721855  2684 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:05.585088  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:05.722862  2611 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:05.586449  3545 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.020s	sys 0.033s
W20260709 14:52:05.723768  3545 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.020s	sys 0.034s
I20260709 14:52:05.590389  2754 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:05.462242  4589 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:05.594342  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:05.602721  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 853) took 1643 ms. Trace:
I20260709 14:52:05.638865  2752 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:05.545436  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:05.605396  2685 rpcz_store.cc:276] 0709 14:52:04.218665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.218872 (+   207us) service_pool.cc:224] Handling call
0709 14:52:04.247832 (+ 28960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:04.248088 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:04.285182 (+ 37094us) log.cc:844] Serialized 14 byte log entry
0709 14:52:05.436367 (+1151185us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b4800541158. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:52:05.537623 (+101256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:05.537713 (+    90us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:05.537791 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.537899 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.538081 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":386559,"prepare.queue_time_us":28697,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":407544,"raft.run_cpu_time_us":11223,"raft.run_wall_time_us":713139,"spinlock_wait_cycles":56778624,"thread_start_us":875144,"threads_started":1}
I20260709 14:52:05.611450  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 855) took 1393 ms. Trace:
I20260709 14:52:05.604210  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:05.594489  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 892) took 2687 ms. Trace:
I20260709 14:52:05.728883  2756 rpcz_store.cc:276] 0709 14:52:02.896821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.916942 (+1020121us) service_pool.cc:224] Handling call
0709 14:52:05.318898 (+1401956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:05.468803 (+149905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.572519 (+103716us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002058d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:05.583992 (+ 11473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3223291264}
I20260709 14:52:05.621236  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:05.722498  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:05.483351  4899 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:05.726203  2726 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.629s (SENT)
W20260709 14:52:05.605100  5042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.010s	sys 0.039s
I20260709 14:52:05.634435  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1010) took 1197 ms. Trace:
I20260709 14:52:05.611594  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:05.621762  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1009) took 1311 ms. Trace:
I20260709 14:52:05.631916  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 819) took 1232 ms. Trace:
I20260709 14:52:05.630965  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:05.749128  2683 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:05.604476  4994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.002s	sys 0.023s
I20260709 14:52:05.594211  5066 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:52:05.603240  5035 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.602926  4989 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:05.630805  5070 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 14:52:05.624647  4636 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.628227  5068 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.730489  4986 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:05.640251  5041 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:05.612003  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:05.781072  2751 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:52:05.781296  3547 raft_consensus.cc:2368] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6.
I20260709 14:52:05.611963  5000 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:05.643004  5039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.062s	sys 0.107s
I20260709 14:52:05.695779  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 377) took 1623 ms. Trace:
I20260709 14:52:05.606689  5023 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:05.614763  4972 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:05.695497  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:05.617053  5020 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:05.611101  4786 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:05.567289  5007 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:05.673866  5028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.005s	sys 0.010s
W20260709 14:52:05.713800  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 252978us
I20260709 14:52:05.618178  2681 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:05.682049  4995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.000s	sys 0.013s
I20260709 14:52:05.733843  2610 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:05.619175  5077 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:05.605260  3540 rpcz_store.cc:276] 0709 14:52:03.772521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.772886 (+   365us) service_pool.cc:224] Handling call
0709 14:52:03.773308 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:03.773678 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:03.959147 (+185469us) log.cc:844] Serialized 14 byte log entry
0709 14:52:05.243984 (+1284837us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b24000f1238. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:52:05.246254 (+  2270us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b1800334de0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:05.499853 (+253599us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:05.499926 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:05.499992 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.500071 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.500279 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68653,"prepare.queue_time_us":201421,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":8051,"raft.queue_time_us":39608,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":198,"spinlock_wait_cycles":13405440,"thread_start_us":1640343,"threads_started":3,"wal-append.queue_time_us":253958}
I20260709 14:52:05.733258  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:05.746887  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142777us
I20260709 14:52:05.746778  3537 raft_consensus.cc:2376] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:05.746388  2753 rpcz_store.cc:276] 0709 14:52:04.436891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.437089 (+   198us) service_pool.cc:224] Handling call
0709 14:52:04.437760 (+   671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:04.487978 (+ 50218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.488027 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.526805 (+ 38778us) log.cc:844] Serialized 26 byte log entry
0709 14:52:04.849394 (+322589us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:04.849462 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:04.849584 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.634111 (+784527us) raft_consensus.cc:1685] finished
0709 14:52:05.634144 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.634349 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55486,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":21216,"thread_start_us":274604,"threads_started":1,"wal-append.queue_time_us":1078430}
I20260709 14:52:05.604743  4901 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:05.747653  2755 rpcz_store.cc:276] 0709 14:52:04.310089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.320286 (+ 10197us) service_pool.cc:224] Handling call
0709 14:52:04.332612 (+ 12326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:04.334417 (+  1805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.334455 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:04.357137 (+ 22682us) log.cc:844] Serialized 52 byte log entry
0709 14:52:04.663609 (+306472us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:04.663687 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:04.663777 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.431424 (+767647us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:05.621362 (+189938us) raft_consensus.cc:1685] finished
0709 14:52:05.621389 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.621673 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9333,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":10412416,"thread_start_us":225364,"threads_started":1,"wal-append.queue_time_us":457794}
I20260709 14:52:05.747879  3541 rpcz_store.cc:276] 0709 14:52:04.399385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.530718 (+131333us) service_pool.cc:224] Handling call
0709 14:52:04.557358 (+ 26640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:04.557939 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.558003 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.599682 (+ 41679us) log.cc:844] Serialized 26 byte log entry
0709 14:52:05.214802 (+615120us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:05.214881 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.214980 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.631599 (+416619us) raft_consensus.cc:1685] finished
0709 14:52:05.631634 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.631832 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4860,"raft.queue_time_us":1856,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":9555,"thread_start_us":452216,"threads_started":1,"wal-append.queue_time_us":958211}
I20260709 14:52:05.793444  5054 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.839891  5023 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:05.753471  2754 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 14:52:05.857040  2754 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:05.856398  5054 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.755374  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:05.858546  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 103299us
I20260709 14:52:05.859414  4972 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:05.859846  4972 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:05.860230  4972 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:05.759155  2608 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:52:05.839440  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:05.772679  2726 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.656s (SENT)
I20260709 14:52:05.863827  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:05.864982  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 92391.8us
I20260709 14:52:05.865355  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 380) took 1302 ms. Trace:
I20260709 14:52:05.865800  2608 rpcz_store.cc:276] 0709 14:52:04.562663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.562847 (+   184us) service_pool.cc:224] Handling call
0709 14:52:04.578984 (+ 16137us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:05.863136 (+1284152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.865040 (+  1904us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:05.865279 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2713358720}
W20260709 14:52:05.773597  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 152964us
I20260709 14:52:05.794291  2605 rpcz_store.cc:276] 0709 14:52:04.072636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.074632 (+  1996us) service_pool.cc:224] Handling call
0709 14:52:04.075114 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:04.183956 (+108842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.183989 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.214621 (+ 30632us) log.cc:844] Serialized 26 byte log entry
0709 14:52:04.425033 (+210412us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:04.425082 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:04.425181 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.695387 (+1270206us) raft_consensus.cc:1685] finished
0709 14:52:05.695423 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.695674 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20619,"raft.queue_time_us":104557,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7297,"thread_start_us":262094,"threads_started":2,"wal-append.queue_time_us":454260}
I20260709 14:52:05.856622  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:05.850589  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:05.879107  4675 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 14:52:05.879383  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146218us
I20260709 14:52:05.902127  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:05.880309  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:05.776348  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:05.903993  2752 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:05.787009  2760 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:52:05.904659  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:05.788619  2756 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 14:52:05.905126  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:05.881197  4995 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 14:52:05.906471  4905 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.907235  5082 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:05.907878  4995 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:05.908437  2677 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:52:05.908543  4995 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:05.772007  4589 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.909672  4589 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:05.911468  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:05.912029  4589 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.762686  4985 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:05.913167  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:05.826052  2611 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:05.762197  5075 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:05.760005  5026 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:05.757552  5074 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 14:52:05.808454  4903 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.582s	user 0.014s	sys 0.004s
I20260709 14:52:05.808073  5051 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 14:52:05.752044  5078 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.761579  4980 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.809449  5035 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:05.815735  5037 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.817947  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 852) took 1859 ms. Trace:
I20260709 14:52:05.896301  5066 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.857389  2754 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 14:52:05.965903  2754 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:05.674507  3538 rpcz_store.cc:276] 0709 14:52:04.340550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.372389 (+ 31839us) service_pool.cc:224] Handling call
0709 14:52:04.412590 (+ 40201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:04.489367 (+ 76777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.489418 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.557477 (+ 68059us) log.cc:844] Serialized 26 byte log entry
0709 14:52:05.603584 (+1046107us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:05.603636 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.603739 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.673370 (+ 69631us) raft_consensus.cc:1685] finished
0709 14:52:05.673403 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.673597 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25564,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":203,"thread_start_us":909175,"threads_started":1,"wal-append.queue_time_us":1052051}
I20260709 14:52:05.961458  2680 rpcz_store.cc:276] 0709 14:52:03.958993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.972496 (+ 13503us) service_pool.cc:224] Handling call
0709 14:52:03.986480 (+ 13984us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:04.099179 (+112699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.099240 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:04.134981 (+ 35741us) log.cc:844] Serialized 52 byte log entry
0709 14:52:04.585956 (+450975us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b1800389ec0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:52:05.046488 (+460532us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b1800389ec0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:05.541991 (+495503us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:05.542055 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.542169 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.817617 (+275448us) raft_consensus.cc:1685] finished
0709 14:52:05.817649 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.817860 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18148,"raft.queue_time_us":140405,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":15712,"spinlock_wait_cycles":31421440,"thread_start_us":495188,"threads_started":1,"wal-append.queue_time_us":1637864}
I20260709 14:52:05.779767  4636 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:05.867880  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:05.869238  4786 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:05.870059  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:05.871325  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:05.791466  5000 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:05.872726  2603 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:05.873791  2606 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:05.873929  5007 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:05.874553  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:05.875379  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:05.875536  5027 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:05.771389  5073 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:05.875901  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:05.877979  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:05.878268  4982 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:05.878605  5069 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.832734  4899 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.831645  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:06.022071  2685 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:05.727698  2682 rpcz_store.cc:276] 0709 14:52:04.218202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.218374 (+   172us) service_pool.cc:224] Handling call
0709 14:52:04.235237 (+ 16863us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:04.591068 (+355831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:04.591103 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.591634 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 14:52:05.473269 (+881635us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:05.473338 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.473459 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.611188 (+137729us) raft_consensus.cc:1685] finished
0709 14:52:05.611212 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.611387 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341638,"raft.queue_time_us":418386,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":23441,"thread_start_us":881081,"threads_started":1,"wal-append.queue_time_us":940918}
I20260709 14:52:05.793840  5068 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.726794  2678 rpcz_store.cc:276] 0709 14:52:03.959686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.972061 (+ 12375us) service_pool.cc:224] Handling call
0709 14:52:03.978687 (+  6626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:03.994011 (+ 15324us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:04.139428 (+145417us) log.cc:844] Serialized 14 byte log entry
0709 14:52:04.589856 (+450428us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:04.590470 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 14:52:04.590676 (+   206us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:04.590711 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:04.590794 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:05.602416 (+1011622us) raft_consensus.cc:1685] finished
0709 14:52:05.602442 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:05.602633 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400927,"prepare.queue_time_us":197243,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":447535,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":11126,"thread_start_us":143002,"threads_started":2,"wal-append.queue_time_us":156341}
I20260709 14:52:05.967167  5035 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:05.790668  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 227023us
I20260709 14:52:06.004508  5000 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:05.881989  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:06.109586  2681 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:05.886750  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:05.970093  4972 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:06.111235  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 224580us
I20260709 14:52:06.112444  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:05.806092  4989 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:05.916400  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:06.117933  3540 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:05.887328  3536 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:05.889021  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:05.890389  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:06.001773  5027 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:05.884207  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1003) took 1583 ms. Trace:
I20260709 14:52:05.888646  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.126622  2751 rpcz_store.cc:276] 0709 14:52:04.301201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.316662 (+ 15461us) service_pool.cc:224] Handling call
0709 14:52:04.332714 (+ 16052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:05.781753 (+1449039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:05.883821 (+102068us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:05.884120 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3324017280}
I20260709 14:52:05.645628  4795 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.886148  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:05.910243  4986 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:05.882115  5077 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:06.140702  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:05.935815  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.142719  2611 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:05.946544  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:05.782191  4562 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.891929  4914 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:05.957897  4980 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:05.958523  3546 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:05.960604  4995 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:05.840378  5016 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:05.907990  5082 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:05.879566  5058 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:05.886876  5023 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:05.896162  4901 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:05.935917  5075 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:06.173939  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:06.176995  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:06.002784  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:06.018532  2756 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 14:52:06.024583  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146692us
I20260709 14:52:06.031648  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:06.030730  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:05.878775  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:05.909029  4905 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:06.033373  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:06.013602  5073 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:06.024041  5007 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:06.001412  4855 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:06.192291  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:06.192709  4855 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:06.193239  5068 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.156863  4562 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:06.196415  4562 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:06.033233  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:06.199548  2752 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:06.007296  5020 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:06.108675  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 826) took 1171 ms. Trace:
W20260709 14:52:05.883188  2726 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.902s (SENT)
I20260709 14:52:05.995254  5037 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:05.995254  4884 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:06.013829  4786 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:05.951215  4903 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.133204  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:06.133544  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.156863  2605 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:52:06.158349  2679 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:06.158900  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:06.137382  5035 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:05.962987  5078 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:06.112027  4894 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:06.166529  2608 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 14:52:06.224382  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64797.5us
I20260709 14:52:06.022701  4589 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.131s
I20260709 14:52:06.124413  4994 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:06.200066  5073 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:06.179699  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 149089us
I20260709 14:52:06.180433  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 850) took 2209 ms. Trace:
I20260709 14:52:06.103642  4974 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 14:52:06.172355  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 390) took 1133 ms. Trace:
I20260709 14:52:05.995759  5066 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.181833  2680 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 3: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:06.190284  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:06.193120  2604 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:06.254716  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.109606  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:06.202167  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 323469us
W20260709 14:52:06.202476  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:06.208371  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:06.146499  5077 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:06.166419  5070 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:06.211063  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 195206us
I20260709 14:52:06.158892  2677 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:06.221392  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:06.182461  5075 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:06.234812  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.183475  4980 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:06.208065  3545 rpcz_store.cc:276] 0709 14:52:04.937879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.938226 (+   347us) service_pool.cc:224] Handling call
0709 14:52:04.951370 (+ 13144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:04.951701 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:05.724308 (+772607us) log.cc:844] Serialized 14 byte log entry
0709 14:52:06.042780 (+318472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.043133 (+   353us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:06.108188 (+ 65055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:06.108331 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:06.108554 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3768,"prepare.queue_time_us":801156,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":68379,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":31868,"thread_start_us":1044738,"threads_started":2}
I20260709 14:52:06.219406  4562 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:52:06.160876  4914 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:06.188026  5023 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 14:52:06.213182  5037 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.242540  5046 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:06.241824  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.251632  4636 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.235136  4901 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:06.169855  5016 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:06.340797  2605 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:06.272586  2759 rpcz_store.cc:276] 0709 14:52:05.039052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.496207 (+457155us) service_pool.cc:224] Handling call
0709 14:52:05.545852 (+ 49645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:05.546107 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:05.914562 (+368455us) log.cc:844] Serialized 14 byte log entry
0709 14:52:06.171609 (+257047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.171726 (+   117us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:06.171812 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:06.171985 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:06.172258 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11201,"prepare.queue_time_us":475835,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":28682,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"thread_start_us":554309,"threads_started":2}
I20260709 14:52:06.172338  3542 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:06.257824  2684 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 14:52:06.251861  5026 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:06.253012  4893 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.005s
W20260709 14:52:06.248318  5087 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 14:52:06.249522  5078 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.210047  5058 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:06.250025  4972 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:06.167500  4899 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 14:52:06.273222  3546 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 14:52:06.318645  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:06.319308  3538 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:06.314565  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48235.9us
I20260709 14:52:06.250594  5020 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:06.261089  4797 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.045s	user 0.015s	sys 0.000s
I20260709 14:52:06.240777  5088 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:06.322746  2679 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:06.325305  4982 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:06.326241  5095 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:06.444545  4972 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:06.466360  5016 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.471345  4972 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:06.474402  4972 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:06.324441  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134294us
I20260709 14:52:06.329057  2680 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:06.478856  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:06.261417  5041 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:06.209676  4983 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 14:52:06.256443  4986 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:06.033006  5069 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:06.342518  4980 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.271394  2686 rpcz_store.cc:276] 0709 14:52:03.958129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:03.971683 (+ 13554us) service_pool.cc:224] Handling call
0709 14:52:03.985326 (+ 13643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:04.002973 (+ 17647us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:04.189182 (+186209us) log.cc:844] Serialized 14 byte log entry
0709 14:52:05.939882 (+1750700us) log.cc:844] Serialized 14 byte log entry
0709 14:52:06.166920 (+227038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.167006 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:06.167119 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:06.167216 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:06.167364 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":316165,"prepare.queue_time_us":799252,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":2516,"raft.queue_time_us":456936,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":101927,"thread_start_us":1750923,"threads_started":5,"wal-append.queue_time_us":211547,"wal-append.run_cpu_time_us":2475,"wal-append.run_wall_time_us":1264595}
I20260709 14:52:06.365876  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:06.341715  4985 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:06.258775  4903 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:06.559628  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:06.562098  2679 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:06.376269  5097 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 14:52:06.377171  4995 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:06.564203  4995 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.565550  3547 raft_consensus.cc:2410] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:06.369258  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:06.435415  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 552250us
I20260709 14:52:06.567441  4983 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:06.568058  4983 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:06.569347  4983 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:06.438383  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 122845us
W20260709 14:52:06.356683  5098 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 14:52:06.448939  3540 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:06.572768  3540 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:06.572587  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:06.574357  3540 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 14:52:06.450088  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 228813us
I20260709 14:52:06.575773  5035 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.460233  5046 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:06.576192  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:06.577028  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:06.577235  5046 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:06.578168  2680 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:06.453881  2685 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:06.485859  5088 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:06.579536  4795 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.579968  4903 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:06.450654  2752 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:06.581139  5088 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:06.234596  4884 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:06.582875  5100 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 14:52:06.583827  4884 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:06.458521  2611 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:06.338733  5074 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 14:52:06.585038  5041 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:06.466233  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:06.585888  3543 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:52:06.586253  3538 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:06.371446  5027 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:06.586796  2608 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:06.434753  4901 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:06.587471  5027 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:06.468786  4982 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:06.475402  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:06.339663  4977 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 14:52:06.503659  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.486497  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:06.496302  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.470304  5034 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 14:52:06.553364  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:06.555212  5020 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:06.578630  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:06.456271  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:06.598513  2604 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:06.480837  4797 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:06.621320  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:06.620970  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:06.629746  2679 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:06.631728  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.634424  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:06.636564  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:06.674005  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:06.971195  2755 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.007745  2610 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:06.551894  5054 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.014489  2758 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:07.015249  2686 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
W20260709 14:52:07.019629  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 544264us
I20260709 14:52:07.036895  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:06.814198  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.616396  5041 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:06.457036  5082 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:07.051882  5095 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 14:52:07.077780  5110 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.452s	user 0.000s	sys 0.001s
I20260709 14:52:07.079288  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:06.586906  5058 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:06.631289  4899 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.079380  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:07.055414  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.085717  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 987) took 1198 ms. Trace:
I20260709 14:52:07.103341  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.103801  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:07.103432  2606 rpcz_store.cc:276] 0709 14:52:05.887561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.158504 (+270943us) service_pool.cc:224] Handling call
0709 14:52:06.180896 (+ 22392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:06.309438 (+128542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:06.309495 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.309567 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:06.309618 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.085243 (+775625us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:07.085336 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:07.085597 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294650112}
I20260709 14:52:06.927651  5037 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:06.786562  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:07.105782  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:06.634970  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 827) took 1607 ms (client timeout 1613 ms). Trace:
W20260709 14:52:07.106685  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:06.585914  5074 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:07.096349  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:07.097723  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.097841  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:07.088366  2679 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:06.970708  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 398215us
W20260709 14:52:06.724357  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156901us
W20260709 14:52:07.106899  3542 rpcz_store.cc:269] 0709 14:52:05.027934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.028252 (+   318us) service_pool.cc:224] Handling call
0709 14:52:06.634643 (+1606391us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54002bb454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:06.634881 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488893184}
I20260709 14:52:07.046265  5104 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:07.045946  4914 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:07.107985  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71227.1us
I20260709 14:52:07.107874  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.146442  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:07.131089  2726 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.937s (SENT)
I20260709 14:52:07.146457  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:07.132704  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:06.677376  5016 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:07.094578  4983 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.093972  2754 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:07.161352  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.161741  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:07.162380  2610 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:06.666101  4884 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:07.154140  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.154218  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:07.154767  3537 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.110179  5095 raft_consensus.cc:2764] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:07.155511  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.173405  2605 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:06.681569  4855 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:06.678339  4977 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:07.159873  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.160512  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:07.163604  2754 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 14:52:07.165107  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:07.172546  4889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.078s	sys 0.126s
I20260709 14:52:07.111218  5074 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:07.129952  5035 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.117743  4899 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.196257  2679 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.172828  5037 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
W20260709 14:52:07.324694  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65426.9us
W20260709 14:52:07.293612  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.030s	sys 0.044s
I20260709 14:52:07.189100  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:07.335233  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 364) took 1453 ms. Trace:
I20260709 14:52:07.335903  2677 rpcz_store.cc:276] 0709 14:52:05.881572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.912839 (+ 31267us) service_pool.cc:224] Handling call
0709 14:52:07.334954 (+1422115us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54002b2854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.335157 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561928960}
I20260709 14:52:07.137943  4901 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:07.192674  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.152245  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.201473  2680 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.162964  4723 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:07.167649  5086 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:07.156553  5016 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:07.200712  5041 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:07.149247  4982 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:07.155154  5104 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.294785  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:07.330698  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.030s	sys 0.052s
I20260709 14:52:07.238636  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.188720  2758 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
W20260709 14:52:07.189574  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58534.2us
W20260709 14:52:07.355186  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 208820us
W20260709 14:52:07.216426  2681 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.090s	user 0.014s	sys 0.000s
I20260709 14:52:07.180114  4884 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:07.219229  3409 connection.cc:256] RPC call timeout handler was delayed by -1.3075484729999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 14:52:07.231041  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.334491  2605 raft_consensus.cc:2376] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:52:07.363427  2609 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:07.211033  4786 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.963s	user 0.012s	sys 0.002s
I20260709 14:52:07.327617  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 823) took 2664 ms. Trace:
I20260709 14:52:07.201269  4977 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:07.351330  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 772693us
W20260709 14:52:07.366993  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 217626us
W20260709 14:52:07.354658  5000 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.207s	user 0.008s	sys 0.008s
I20260709 14:52:07.270896  5035 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.355823  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.269008  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.357151  2681 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:07.361974  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:07.344772  4980 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:07.214790  4889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.079s	sys 0.127s
I20260709 14:52:07.364413  3543 rpcz_store.cc:276] 0709 14:52:04.663533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.666229 (+  2696us) service_pool.cc:224] Handling call
0709 14:52:04.690914 (+ 24685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:06.586345 (+1895431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.327209 (+740864us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:07.327537 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4357321600}
I20260709 14:52:07.349416  5086 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:07.373306  4675 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.380905  5086 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:07.381301  4786 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:07.381143  4675 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.381871  4786 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:07.369055  2756 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:52:07.353519  5016 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.359455  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.371954  2679 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.353888  4723 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:07.394086  5073 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.072s	user 0.007s	sys 0.005s
W20260709 14:52:07.394997  5114 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.203s	user 0.000s	sys 0.002s
I20260709 14:52:07.393420  4723 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.396417  5104 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.399650  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.401769  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:07.402323  5105 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.848s	user 0.001s	sys 0.002s
I20260709 14:52:07.403864  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.406302  2677 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:07.372746  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.385525  4786 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.386737  2680 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.407346  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 857) took 1760 ms. Trace:
W20260709 14:52:07.373395  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 361) took 1493 ms (client timeout 1758 ms). Trace:
W20260709 14:52:07.417994  2681 rpcz_store.cc:269] 0709 14:52:05.880442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.881698 (+  1256us) service_pool.cc:224] Handling call
0709 14:52:07.373324 (+1491626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:07.416263  2683 rpcz_store.cc:276] 0709 14:52:05.647178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.107833 (+460655us) service_pool.cc:224] Handling call
0709 14:52:06.144196 (+ 36363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:06.157961 (+ 13765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:06.157994 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:06.165556 (+  7562us) log.cc:844] Serialized 52 byte log entry
0709 14:52:06.587536 (+421980us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b1800123040. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:07.361488 (+773952us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:07.361542 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.361624 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:07.406945 (+ 45321us) raft_consensus.cc:1685] finished
0709 14:52:07.406971 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:07.407274 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4886,"raft.queue_time_us":11952,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13114,"spinlock_wait_cycles":40530304,"thread_start_us":768256,"threads_started":1,"wal-append.queue_time_us":1190596}
I20260709 14:52:07.372236  5000 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:07.435297  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69622.4us
I20260709 14:52:07.436008  2677 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:07.444644  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57602.8us
I20260709 14:52:07.452483  2683 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 14:52:07.394739  5073 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.431643  5103 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:07.371206  5074 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:07.451833  5075 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.095s	user 0.012s	sys 0.003s
I20260709 14:52:07.478066  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.482048  2683 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:07.476958  5041 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.503029  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.488591  4917 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.488762  5074 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:07.513481  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.515612  2680 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:07.452253  4977 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:07.522388  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:07.523409  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59782.5us
I20260709 14:52:07.478024  5075 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.531613  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.532662  2680 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:07.492542  5041 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.534996  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.536005  2681 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.537212  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.538614  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.539271  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.540648  5000 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.545789  4884 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.546617  4884 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:07.549443  2683 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:07.551148  2757 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:07.552623  4977 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.558518  4899 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:07.559152  2679 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 14:52:07.518904  4917 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.561251  5041 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:07.561719  4917 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.559612  2679 pending_rounds.cc:85] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 2
I20260709 14:52:07.553942  5086 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:07.562448  2679 pending_rounds.cc:107] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 14:52:07.563422  2679 raft_consensus.cc:2889] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:07.564324  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:07.564468  5086 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.564849  4797 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.565577  5086 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.566399  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 367) took 1631 ms. Trace:
I20260709 14:52:07.566228  5086 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.567580  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:07.568845  2679 rpcz_store.cc:276] 0709 14:52:05.935098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.591467 (+656369us) service_pool.cc:224] Handling call
0709 14:52:06.629529 (+ 38062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:07.566084 (+936555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.566329 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:07.569703  2685 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:07.572521  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.575699  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:07.590757  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.580319  5126 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
W20260709 14:52:07.596238  3547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 615) took 1987 ms (client timeout 1611 ms). Trace:
W20260709 14:52:07.596760  3547 rpcz_store.cc:269] 0709 14:52:05.608668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.791205 (+182537us) service_pool.cc:224] Handling call
0709 14:52:07.595917 (+1804712us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400225454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.596146 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613204736}
I20260709 14:52:07.605829  2607 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 14:52:07.606901  2607 pending_rounds.cc:85] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 4
I20260709 14:52:07.607239  2607 pending_rounds.cc:107] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 14:52:07.607491  2607 raft_consensus.cc:2889] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:07.580101  5027 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.946s	user 0.010s	sys 0.004s
I20260709 14:52:07.580814  4914 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:07.614936  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:07.574306  5074 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 14:52:07.618964  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.583197  5104 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:52:07.625216  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:07.626366  2726 connection.cc:256] RPC call timeout handler was delayed by -1.5059194280000838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2258s for a response
W20260709 14:52:07.628053  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1331979949999322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 14:52:07.616444  5077 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.285s	user 0.015s	sys 0.003s
I20260709 14:52:07.618364  4914 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:07.626170  4980 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 14:52:07.580582  4899 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 14:52:07.606634  5035 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:07.608045  5126 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:07.583317  4917 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 14:52:07.644037  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.911s (SENT)
I20260709 14:52:07.632352  5059 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:07.625898  5000 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:07.634027  4884 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.649009  5134 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.653093  4884 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.653393  5134 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.655584  4884 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.632439  5086 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.657202  5075 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:07.643330  4914 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:07.660873  5073 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.635609  5104 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:07.665843  5104 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.667814  3547 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:07.669878  3543 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:07.670646  4917 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:52:07.671135  4917 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:07.672122  4917 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.673573  4917 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:07.674209  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 861) took 1758 ms. Trace:
I20260709 14:52:07.674626  3543 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:07.675484  3542 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 14:52:07.675983  5070 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.356s	user 0.010s	sys 0.006s
I20260709 14:52:07.676442  5070 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:07.676792  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:07.677080  3542 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:07.641109  5088 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 14:52:07.635330  5037 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 14:52:07.634213  5074 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:07.679419  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:07.679569  5074 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.680545  3545 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:07.682555  3536 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 14:52:07.684765  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 624) took 1792 ms (client timeout 2245 ms). Trace:
W20260709 14:52:07.685281  3536 rpcz_store.cc:269] 0709 14:52:05.915047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.123576 (+208529us) service_pool.cc:224] Handling call
0709 14:52:07.684471 (+1560895us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54004b64d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.684679 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3360748416}
I20260709 14:52:07.686400  5134 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:07.642957  5077 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.687670  2685 rpcz_store.cc:276] 0709 14:52:05.916285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.579315 (+663030us) service_pool.cc:224] Handling call
0709 14:52:06.579667 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:07.575140 (+995473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.673879 (+ 98739us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:07.674137 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2276196224}
I20260709 14:52:07.689330  3541 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.689761  4675 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:07.690658  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 622) took 1941 ms. Trace:
I20260709 14:52:07.691114  3541 rpcz_store.cc:276] 0709 14:52:05.749364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.156426 (+407062us) service_pool.cc:224] Handling call
0709 14:52:07.690406 (+1533980us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.690575 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3482713984}
W20260709 14:52:07.693293  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 824) took 2951 ms (client timeout 2208 ms). Trace:
I20260709 14:52:07.693420  3538 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:07.693936  3546 rpcz_store.cc:269] 0709 14:52:04.741989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:04.747055 (+  5066us) service_pool.cc:224] Handling call
0709 14:52:07.692941 (+2945886us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400003a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.693171 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134638976}
W20260709 14:52:07.695569  5113 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.376s	user 0.002s	sys 0.000s
I20260709 14:52:07.697326  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 363) took 1816 ms. Trace:
I20260709 14:52:07.699769  5066 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:07.702062  5016 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.675848  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:07.703559  3543 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:52:07.650103  5059 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:07.705461  5059 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.706580  5059 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:07.697734  2682 rpcz_store.cc:276] 0709 14:52:05.881259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.261866 (+380607us) service_pool.cc:224] Handling call
0709 14:52:06.262338 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:06.262793 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:06.262866 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:06.304302 (+ 41436us) log.cc:844] Serialized 26 byte log entry
0709 14:52:06.616102 (+311800us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:06.616163 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:06.616254 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:07.694857 (+1078603us) spinlock_profiling.cc:243] Waited 64.6 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:07.696971 (+  2114us) raft_consensus.cc:1685] finished
0709 14:52:07.696995 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:07.697256 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47447,"raft.queue_time_us":5244,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":40636,"spinlock_wait_cycles":148552064,"thread_start_us":311292,"threads_started":1,"wal-append.queue_time_us":1014896}
I20260709 14:52:07.736083  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.736258  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.737136  2679 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
W20260709 14:52:07.743410  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.788311  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.788669  4914 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.789537  2682 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:07.792097  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.793396  2685 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.731943  5016 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:07.740690  5066 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:07.797600  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.766923  4797 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.776520  5039 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:07.799827  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:07.802580  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:07.803292  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.804597  2606 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.804865  2607 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:07.807839  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.809369  2686 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:07.815666  2723 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
I20260709 14:52:07.816076  2723 leader_election.cc:403] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 14:52:07.816872  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.817929  2682 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:52:07.801390  5077 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.841185  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.844913  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:07.846045  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.847175  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:07.848668  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:07.850422  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.850551  5140 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7.
W20260709 14:52:07.851155  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.852558  5122 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:07.853041  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:07.853809  5073 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:07.850632  5070 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:07.854714  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:07.800504  5066 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:07.861936  2679 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:07.863482  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.846611  5088 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 14:52:07.856295  5082 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:07.867450  5066 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.867470  5082 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:07.869484  5137 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 14:52:07.871686  5082 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:07.868366  5115 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.544s	user 0.000s	sys 0.005s
I20260709 14:52:07.808488  5039 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:07.847097  5000 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 14:52:07.875643  5070 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:07.876116  5070 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:07.876195  5039 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:07.805801  4797 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:07.877323  5070 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:07.879141  5070 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.879257  4797 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:07.879544  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64049.7us
I20260709 14:52:07.880681  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.882089  5066 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:07.883538  2726 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.520s (SENT)
I20260709 14:52:07.884086  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:07.885270  2726 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.523s (SENT)
W20260709 14:52:07.886139  2726 leader_election.cc:336] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.910s (SENT)
W20260709 14:52:07.886976  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:07.887910  4786 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:07.889228  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.889433  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.890172  2677 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.890100  2680 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.893311  5066 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:07.893961  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:07.894666  2680 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:07.895431  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.897392  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.844599  5037 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 14:52:07.904325  2680 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:07.905467  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:07.906407  3543 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.906462  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.907204  2681 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.909000  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.909523  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:07.909775  2605 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.909456  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.974654  2606 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.976608  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:07.977691  5070 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:07.978137  5070 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:07.978497  5070 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:07.980738  4982 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:07.981280  4982 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:07.911588  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48330.3us
I20260709 14:52:07.983425  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:07.984340  4901 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:07.984879  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.985913  4901 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:07.986325  4901 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:07.986682  4901 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:07.989347  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 727) took 2355 ms (client timeout 1911 ms). Trace:
W20260709 14:52:07.989910  3537 rpcz_store.cc:269] 0709 14:52:05.633906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:05.867492 (+233586us) service_pool.cc:224] Handling call
0709 14:52:07.989081 (+2121589us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.989257 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701842688}
I20260709 14:52:07.991254  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:07.991909  3537 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:07.992305  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:07.993243  5077 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:07.993652  5077 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:07.994005  5077 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:07.995798  3540 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:07.997355  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:07.998099  2760 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:08.000684  5106 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.455s	user 0.004s	sys 0.000s
I20260709 14:52:07.982309  4982 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:08.006294  4982 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:08.009012  5121 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.626s	user 0.002s	sys 0.000s
I20260709 14:52:08.009641  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:08.010423  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 859) took 2344 ms. Trace:
I20260709 14:52:08.011152  5104 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:08.011214  2678 rpcz_store.cc:276] 0709 14:52:05.666052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.120887 (+454835us) service_pool.cc:224] Handling call
0709 14:52:06.121305 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:06.583382 (+462077us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:06.583437 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.583512 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:06.583565 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.010108 (+1426543us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400445a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:08.010160 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:08.010348 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2422,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":1061385088}
I20260709 14:52:08.017306  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:07.962136  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.962455  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:07.908720  5068 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 14:52:07.886740  2726 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.986s (SENT)
I20260709 14:52:07.908824  3538 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:07.964538  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:08.031008  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 144289us
I20260709 14:52:08.031323  3543 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.882426  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:07.966545  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.032898  2680 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:08.033934  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.035211  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:08.035995  2680 raft_consensus.cc:2376] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:08.037187  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:08.038179  2680 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:07.966988  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 903) took 1523 ms. Trace:
I20260709 14:52:08.038697  2756 rpcz_store.cc:276] 0709 14:52:06.443494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.443670 (+   176us) service_pool.cc:224] Handling call
0709 14:52:07.966689 (+1523019us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.966868 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639124736}
I20260709 14:52:07.967031  3547 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:07.968134  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:07.968528  2681 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:07.968588  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:07.969337  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:07.969851  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:08.043304  2682 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:08.043936  2679 raft_consensus.cc:2376] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6. Re-sending same reply.
I20260709 14:52:08.029644  2607 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.971225  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:07.971879  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:08.045822  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 163461us
W20260709 14:52:08.046241  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54004.7us
I20260709 14:52:07.910382  3546 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:07.972873  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 911) took 1298 ms. Trace:
I20260709 14:52:08.047088  2757 rpcz_store.cc:276] 0709 14:52:06.674738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.675475 (+   737us) service_pool.cc:224] Handling call
0709 14:52:07.972686 (+1297211us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:07.972810 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976073984}
I20260709 14:52:08.046325  2677 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:07.973651  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.047998  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.048554  3542 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:08.082209  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:08.019378  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:08.098186  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83950us
W20260709 14:52:08.100370  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86214.7us
W20260709 14:52:08.020875  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141707us
I20260709 14:52:08.029644  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:08.120884  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.121799  3543 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:08.126839  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78922.4us
I20260709 14:52:08.127359  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.128111  2680 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:08.130359  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.136165  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.136804  3541 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:08.137049  2611 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:07.848269  4914 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:08.140686  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.129855  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:08.142023  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.142967  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.143613  3547 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:08.144300  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.144938  2681 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:08.145745  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.146418  3546 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:08.149073  2760 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:08.149439  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:08.149788  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 913) took 1474 ms. Trace:
I20260709 14:52:08.150121  2760 rpcz_store.cc:276] 0709 14:52:06.675464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.726024 (+ 50560us) service_pool.cc:224] Handling call
0709 14:52:07.997165 (+1271141us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:08.149712 (+152547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210674048}
W20260709 14:52:08.138630  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:08.151819  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:08.153066  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.155297  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.158207  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:08.147308  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.141472  2751 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:08.163848  3545 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:07.961757  4675 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:08.167264  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.168776  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.168845  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:08.020136  5007 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:08.175083  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.119736  5068 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:08.126502  5066 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:08.183176  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.143010  5037 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:08.189462  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:08.194199  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64446.4us
I20260709 14:52:08.194430  2611 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:08.191799  2751 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:08.195899  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.186455  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.197283  2756 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:08.207676  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.208873  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:52:08.210198  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.219619  2760 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:08.222998  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.224120  3541 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:08.176563  5140 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:08.177960  5075 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:08.136245  4899 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 14:52:08.242664  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.139568  5103 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 14:52:08.247318  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:08.191543  4786 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:08.192106  5066 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.261448  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.266227  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.223405  5088 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:08.277892  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.267606  3541 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:52:08.188915  5126 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 14:52:08.261912  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.283831  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:08.307327  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.313825  3541 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:08.315322  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.330647  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.330461  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.331223  2760 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:08.332058  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.332782  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.312000  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.335913  3536 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:08.337301  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.328856  2603 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 14:52:08.330394  2605 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:52:08.331499  2604 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:08.319746  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.341975  2751 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 2.
I20260709 14:52:08.343487  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.345181  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.334451  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.348858  2760 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:08.287554  5066 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:08.360925  3537 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:08.359066  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.363261  3536 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:08.367281  5058 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:08.371430  5077 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:08.373152  2611 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:08.233421  5070 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:08.378051  5070 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:08.381227  5104 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:08.381852  5104 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:08.296645  5068 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:08.385236  5068 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:08.254743  5046 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:08.362500  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.193377  4901 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:08.331357  2607 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:08.279316  5007 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:08.374522  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.401695  2756 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:08.402581  3547 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:08.396512  2677 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:08.417356  3543 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:08.425642  2760 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:08.427881  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.428756  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.429440  2760 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:08.429963  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.392096  5046 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.434926  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.436405  3543 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:08.437589  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.438793  3547 raft_consensus.cc:2368] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:08.431512  5046 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:08.444872  5126 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:08.445490  5126 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:08.448531  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 329982us
I20260709 14:52:08.451642  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 360) took 2630 ms. Trace:
I20260709 14:52:08.394726  4797 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:08.389384  5088 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:08.457129  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.452565  2684 rpcz_store.cc:276] 0709 14:52:05.821393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.144450 (+323057us) service_pool.cc:224] Handling call
0709 14:52:06.161467 (+ 17017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:06.342572 (+181105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.451070 (+2108498us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:08.451540 (+   470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220645248}
I20260709 14:52:08.458950  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:08.419421  5007 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.467242  3536 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
W20260709 14:52:08.491604  2753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.017s	sys 0.068s
W20260709 14:52:08.526377  2753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.017s	sys 0.075s
I20260709 14:52:08.528318  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.482719  2760 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:52:08.515278  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.529985  3543 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:52:08.531374  3543 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:08.407917  5070 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:08.472939  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.528039  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.534456  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.534849  2611 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:08.534875  3537 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:08.533514  2576 leader_election.cc:400] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6.
I20260709 14:52:08.535939  2576 leader_election.cc:403] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 14:52:08.473498  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:08.543294  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:08.527326  3536 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:52:08.473206  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:08.454499  5077 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:08.552726  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 79611.1us
I20260709 14:52:08.553071  2684 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:08.563560  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:08.564038  3537 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:08.564983  4901 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:08.471973  5074 raft_consensus.cc:2764] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 14:52:08.552407  5126 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:08.572577  5126 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:08.396063  5104 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:08.580400  5103 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:08.581141  5103 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:08.582207  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.586546  2686 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 14:52:08.593575  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.594806  2751 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:08.595975  5120 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.234s	user 0.003s	sys 0.009s
I20260709 14:52:08.575907  5104 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:08.598688  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.601048  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 910) took 1927 ms. Trace:
I20260709 14:52:08.603781  2684 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 14:52:08.472879  4797 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:08.606451  5136 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.971s	user 0.009s	sys 0.000s
I20260709 14:52:08.608060  4797 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:08.515810  4786 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:08.608666  3543 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 14:52:08.609068  4786 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:08.610042  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1019) took 2167 ms. Trace:
I20260709 14:52:08.610358  4786 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:08.610579  2759 rpcz_store.cc:276] 0709 14:52:06.442877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.443044 (+   167us) service_pool.cc:224] Handling call
0709 14:52:06.500997 (+ 57953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:06.588224 (+ 87227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:06.588265 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:06.588318 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:06.588353 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.609715 (+2021362us) spinlock_profiling.cc:243] Waited 86.9 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:08.609775 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:08.609947 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199973248}
I20260709 14:52:08.611519  4901 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:08.611974  4786 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:08.529825  5007 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:08.612367  4901 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:08.601819  2752 rpcz_store.cc:276] 0709 14:52:06.674234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.674472 (+   238us) service_pool.cc:224] Handling call
0709 14:52:06.675005 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:06.677627 (+  2622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:06.677674 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:06.678391 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 14:52:07.149129 (+470738us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:07.149191 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:07.149276 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:08.007349 (+858073us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:52:08.594967 (+587618us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:08.600722 (+  5755us) raft_consensus.cc:1685] finished
0709 14:52:08.600758 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:08.600956 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1941,"raft.queue_time_us":3666,"raft.run_cpu_time_us":14281,"raft.run_wall_time_us":1328347,"spinlock_wait_cycles":94130816,"thread_start_us":466961,"threads_started":1,"wal-append.queue_time_us":645331}
I20260709 14:52:08.616351  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 944) took 1082 ms. Trace:
I20260709 14:52:08.616878  2683 rpcz_store.cc:276] 0709 14:52:07.534006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:07.538966 (+  4960us) service_pool.cc:224] Handling call
0709 14:52:08.616064 (+1077098us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:08.616266 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20879744}
I20260709 14:52:08.617712  2752 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:52:08.619350  4914 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:08.620136  4914 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:08.622048  5117 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.241s	user 0.016s	sys 0.062s
I20260709 14:52:08.609287  3543 pending_rounds.cc:85] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 3
I20260709 14:52:08.622579  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.624828  4901 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:08.618736  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1026) took 1250 ms. Trace:
I20260709 14:52:08.625989  2752 rpcz_store.cc:276] 0709 14:52:07.368704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.616818 (+1248114us) service_pool.cc:224] Handling call
0709 14:52:08.617361 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.618325 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.618652 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.626153  5066 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:08.628113  4995 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:08.629405  5066 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:08.628079  2752 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:08.631440  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 914) took 1095 ms. Trace:
I20260709 14:52:08.631985  2752 rpcz_store.cc:276] 0709 14:52:07.536111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.627291 (+1091180us) service_pool.cc:224] Handling call
0709 14:52:08.627815 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:08.631028 (+  3213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.631324 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.634141  2752 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:08.648895  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 918) took 1064 ms. Trace:
I20260709 14:52:08.651891  2752 rpcz_store.cc:276] 0709 14:52:07.584242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.633132 (+1048890us) service_pool.cc:224] Handling call
0709 14:52:08.633797 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.634628 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.648802 (+ 14174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.622720  3543 pending_rounds.cc:107] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 14:52:08.653445  3541 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:08.659652  3543 raft_consensus.cc:2889] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:08.666877  3547 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:08.667732  3542 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.000s
I20260709 14:52:08.668177  3542 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:08.676194  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:08.677374  2756 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:08.677049  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.677937  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 915) took 1138 ms. Trace:
I20260709 14:52:08.678323  2756 rpcz_store.cc:276] 0709 14:52:07.539716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.544006 (+1004290us) service_pool.cc:224] Handling call
0709 14:52:08.677861 (+133855us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.582191  5103 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:08.699260  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:08.700842  3537 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:08.691680  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.693382  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:08.703562  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.018s
W20260709 14:52:08.704212  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.008s	sys 0.018s
W20260709 14:52:08.704481  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 530082us
I20260709 14:52:08.704699  2760 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:08.691232  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1022) took 1654 ms. Trace:
I20260709 14:52:08.705420  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 400) took 1068 ms. Trace:
I20260709 14:52:08.705904  2760 rpcz_store.cc:276] 0709 14:52:07.637397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.671805 (+1034408us) service_pool.cc:224] Handling call
0709 14:52:08.704426 (+ 32621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.705130 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.705353 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.693974  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.707420  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.702747  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.708192  2611 raft_consensus.cc:2376] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7. Re-sending same reply.
I20260709 14:52:08.709319  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.706735  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:08.712466  2650 leader_election.cc:400] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:08.713016  2650 leader_election.cc:403] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:52:08.527184  4723 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:08.718163  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:08.719625  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.720223  2757 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:08.722339  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 397) took 1088 ms. Trace:
I20260709 14:52:08.723382  2757 rpcz_store.cc:276] 0709 14:52:07.634494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.669914 (+1035420us) service_pool.cc:224] Handling call
0709 14:52:08.720017 (+ 50103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:08.721984 (+  1967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.722263 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.722278  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.726212  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.667325  4914 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:08.728539  2756 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:08.729732  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 407) took 1079 ms. Trace:
I20260709 14:52:08.731398  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:08.730486  2756 rpcz_store.cc:276] 0709 14:52:07.650253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.679237 (+1028984us) service_pool.cc:224] Handling call
0709 14:52:08.728240 (+ 49003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:08.729323 (+  1083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.729647 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:08.711577  5016 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:08.734265  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.746135  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.748106  2610 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:08.736372  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:08.721313  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:08.738435  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 491216us
W20260709 14:52:08.751252  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:08.752254  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:08.753131  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:08.753881  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:08.754082  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 224915us
I20260709 14:52:08.754757  2604 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:08.238001   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:52:08.739588  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 455831us
I20260709 14:52:08.760437  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:08.740556  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 544701us
I20260709 14:52:08.762084  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:08.766127  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:08.766677  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:08.724156  4899 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:08.707824  5041 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:08.765846  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.718729  4636 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:08.746129  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.726998  4797 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 14:52:08.716856  5077 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:08.717093  5074 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:08.705386  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:08.723974  5000 raft_consensus.cc:2764] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:08.705811  2758 rpcz_store.cc:276] 0709 14:52:07.037666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:07.357435 (+319769us) service_pool.cc:224] Handling call
0709 14:52:07.357821 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:07.358347 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:08.007964 (+649617us) log.cc:844] Serialized 14 byte log entry
0709 14:52:08.179512 (+171548us) log.cc:844] Serialized 14 byte log entry
0709 14:52:08.382156 (+202644us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b180035cb60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:08.690726 (+308570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:08.690799 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:08.690889 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.690999 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:08.691155 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":703284,"prepare.queue_time_us":682333,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":8752,"raft.queue_time_us":224489,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":144,"spinlock_wait_cycles":74301824,"thread_start_us":478953,"threads_started":2,"wal-append.queue_time_us":338586}
I20260709 14:52:08.722770  4723 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.906754  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:08.727357  4989 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.181s
W20260709 14:52:08.772642  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56763us
I20260709 14:52:08.774319  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:08.771178  2756 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 14:52:08.777330  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:09.030516  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 253152us
I20260709 14:52:09.033142  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:08.768739  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:08.952001  2611 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:08.954005  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:09.035605  2605 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:08.956281  2608 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 14:52:08.724659  5058 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 14:52:08.755679  5016 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.746367  4914 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 14:52:08.964635  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:08.962692  2755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.036s	sys 0.070s
W20260709 14:52:09.019855  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 113286us
W20260709 14:52:09.120399  2755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.042s	sys 0.075s
I20260709 14:52:09.121731  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1021) took 2097 ms. Trace:
I20260709 14:52:08.771910  4901 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:09.021203  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 315855us
I20260709 14:52:09.133663  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.039160  2758 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [4.4-5.5]
I20260709 14:52:09.135867  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 414) took 1254 ms. Trace:
I20260709 14:52:09.136438  2758 rpcz_store.cc:276] 0709 14:52:07.881763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.022081 (+1140318us) service_pool.cc:224] Handling call
0709 14:52:09.038929 (+ 16848us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:09.135341 (+ 96412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.135769 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:09.097895  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.102685  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 413) took 1247 ms. Trace:
I20260709 14:52:09.139261  2756 rpcz_store.cc:276] 0709 14:52:07.855610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.770500 (+914890us) service_pool.cc:224] Handling call
0709 14:52:08.770936 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:09.028834 (+257898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.102560 (+ 73726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:09.141044  2756 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:09.105265  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 336586us
I20260709 14:52:08.771689  4995 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:52:08.770259  5152 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6.
I20260709 14:52:08.906334  5041 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:08.776216  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.767334  5088 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:09.122128  2755 rpcz_store.cc:276] 0709 14:52:07.024022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:07.215386 (+191364us) service_pool.cc:224] Handling call
0709 14:52:07.298793 (+ 83407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:07.299183 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:08.002437 (+703254us) log.cc:844] Serialized 14 byte log entry
0709 14:52:09.121177 (+1118740us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:09.121252 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:09.121304 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.121491 (+   187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.121667 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":865728,"prepare.queue_time_us":703561,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":192753,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":19306,"thread_start_us":954738,"threads_started":2,"wal-append.queue_time_us":439501}
W20260709 14:52:09.123796  2681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.018s	sys 0.037s
I20260709 14:52:09.124116  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:08.952603  5070 raft_consensus.cc:2764] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 14:52:08.952011  4994 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:52:09.180343  5058 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:09.144431  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1028) took 1124 ms. Trace:
I20260709 14:52:08.961949  4899 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:09.111078  4675 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:09.149972  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.012s	sys 0.041s
I20260709 14:52:09.022904  4982 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:09.207084  4899 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.211009  4675 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:09.211741  4675 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:09.212759  5124 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:09.213387  5124 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:09.213827  5124 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:09.215386  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:09.217621  2606 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:08.780956  5075 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.352s
W20260709 14:52:09.122984  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.176491  2612 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:08.966337  5077 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:09.177570  2681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.018s	sys 0.037s
I20260709 14:52:09.179700  2755 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:09.250639  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1030) took 1109 ms. Trace:
I20260709 14:52:09.251106  2755 rpcz_store.cc:276] 0709 14:52:08.140892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.175458 (+1034566us) service_pool.cc:224] Handling call
0709 14:52:09.179456 (+  3998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:09.250278 (+ 70822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.250541 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:09.022703  4723 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:09.183321  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 428120us
I20260709 14:52:09.198939  5039 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:09.133729  5073 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.134039  5069 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:09.114410  5156 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:09.196070  2756 rpcz_store.cc:276] 0709 14:52:08.020419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.140244 (+1119825us) service_pool.cc:224] Handling call
0709 14:52:09.140680 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:09.141604 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.144328 (+  2724us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:09.180694  4994 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:09.208025  4982 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:09.146653  4797 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:09.152527  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 415) took 1258 ms. Trace:
I20260709 14:52:09.304255  2758 rpcz_store.cc:276] 0709 14:52:07.882134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.137831 (+1255697us) service_pool.cc:224] Handling call
0709 14:52:09.138321 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:09.138617 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:09.138668 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:09.138742 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:09.138794 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.140013 (+  1219us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.140171 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:09.199198  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.013s	sys 0.041s
I20260709 14:52:09.160391  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:09.306118  2611 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:09.259853  5051 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 14:52:09.232457  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.013s	sys 0.070s
W20260709 14:52:09.329090  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.013s	sys 0.070s
W20260709 14:52:09.150085  5020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.759s	user 0.031s	sys 0.042s
I20260709 14:52:09.175086  5041 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:09.242331  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.015s	sys 0.034s
I20260709 14:52:09.127393  5016 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:09.257303  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:09.232982  5007 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.610s	user 0.013s	sys 0.000s
W20260709 14:52:09.247673  5158 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 14:52:09.334440  5020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.941s	user 0.031s	sys 0.043s
W20260709 14:52:09.274622  2753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.037s	sys 0.042s
I20260709 14:52:09.232648  5058 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:09.193856  5164 raft_consensus.cc:2764] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 14:52:09.258266  5104 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.660s	user 0.014s	sys 0.000s
I20260709 14:52:09.165782  5000 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:09.308214  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:09.338225  4797 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:09.394055  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86000.4us
W20260709 14:52:09.309576  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.310551  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.253801  5077 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:09.251439  4983 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:09.330406  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.030s	sys 0.079s
W20260709 14:52:09.335686  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.016s	sys 0.035s
I20260709 14:52:09.260535  5039 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:09.332409  2607 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.937s	user 0.011s	sys 0.003s
I20260709 14:52:09.418972  2607 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:09.269270  5075 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:09.241494  5163 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 14:52:09.296641  4995 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:09.293254  3409 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.711s (SENT)
W20260709 14:52:09.351068  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 93860.2us
I20260709 14:52:09.424463  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.424865  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:09.327553  4982 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:52:09.353945  2753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.041s	sys 0.046s
W20260709 14:52:09.348042  4786 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.001s
I20260709 14:52:09.330952  5165 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.407498  2756 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:09.353818  5166 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.448396  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:09.449379  2756 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:09.347687  5007 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:09.412860  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.030s	sys 0.080s
W20260709 14:52:09.414462  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 270678us
I20260709 14:52:09.371752  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:09.413172  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 887811us
I20260709 14:52:09.418259  2758 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 14:52:09.458863  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.460268  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.457274  2758 pending_rounds.cc:85] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:09.463608  2758 pending_rounds.cc:107] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 14:52:09.463971  2758 raft_consensus.cc:2889] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:09.410591  4636 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:09.427227  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 116829us
W20260709 14:52:09.459834  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.013s	sys 0.047s
W20260709 14:52:09.476351  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.013s	sys 0.047s
I20260709 14:52:09.476737  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:09.477218  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 716877us
W20260709 14:52:09.439342  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.450793  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:09.478374  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.479559  2756 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 14:52:09.370213  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.019s	sys 0.023s
W20260709 14:52:09.482153  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.020s	sys 0.024s
I20260709 14:52:09.482059  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.353391  5160 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:09.453179  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 425146us
I20260709 14:52:09.418826  5159 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:52:09.490723  2756 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:09.462677  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:09.458911  5152 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:09.433197  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139999us
W20260709 14:52:09.493432  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 359922us
I20260709 14:52:09.486395  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:09.482777  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 704829us
I20260709 14:52:09.499723  2758 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:09.420325  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 398) took 1092 ms. Trace:
I20260709 14:52:09.500481  2607 rpcz_store.cc:276] 0709 14:52:08.328066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.330349 (+  2283us) service_pool.cc:224] Handling call
0709 14:52:09.420046 (+1089697us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:09.500985  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 850) took 1187 ms. Trace:
I20260709 14:52:09.484803  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:09.436141  5104 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.501374  3547 rpcz_store.cc:276] 0709 14:52:08.313296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.551017 (+237721us) service_pool.cc:224] Handling call
0709 14:52:08.551392 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:08.667455 (+116063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.500748 (+833293us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400203354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:09.500923 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264450432}
I20260709 14:52:09.502337  5104 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:52:09.503407  5104 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:09.503759  5140 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:09.503880  5104 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:09.504611  5140 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:09.504683  5104 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:09.497129  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.596s (SENT) [suppressed 40 similar messages]
W20260709 14:52:09.509788  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.525416  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:09.530771  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70558.8us
I20260709 14:52:09.568436  2756 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:09.434305  4995 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.391251  4994 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:09.492069  5069 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.473851  5165 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.458295  5034 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
W20260709 14:52:09.605770  2605 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.004s
I20260709 14:52:09.655465  2605 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:09.458698  5153 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 14:52:09.611599  4636 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.495541  5154 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 14:52:09.612514  5180 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:52:09.614168  5172 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 14:52:09.614933  5122 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:09.616794  4995 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:09.617210  5126 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.044s	user 0.014s	sys 0.002s
W20260709 14:52:09.617516  5169 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 14:52:09.617856  5069 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:09.622004  5068 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.236s	user 0.013s	sys 0.001s
I20260709 14:52:09.622867  5164 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.625785  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 388) took 1652 ms. Trace:
I20260709 14:52:09.629676  5104 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.631399  5007 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:09.496279  5152 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:09.484630  5039 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.486340  5156 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.146s
I20260709 14:52:09.509244  5058 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.491601  5035 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 14:52:09.633687  5073 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.459028  5166 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.608987  5160 raft_consensus.cc:2764] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 14:52:09.763353  4723 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.495173  5159 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:09.765681  5159 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:09.767469  2609 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:09.703120  5179 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.770242  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 996) took 1968 ms. Trace:
I20260709 14:52:09.769994  5179 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.599288  4797 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.647028  5165 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.776921  5068 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.777431  5165 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:09.777639  5068 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.579871  5082 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 14:52:09.650359  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1025) took 2282 ms. Trace:
W20260709 14:52:09.590533  2610 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.001s
I20260709 14:52:09.792251  2610 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:09.793041  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1003) took 1073 ms. Trace:
I20260709 14:52:09.793474  2610 rpcz_store.cc:276] 0709 14:52:08.719695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.722543 (+  2848us) service_pool.cc:224] Handling call
0709 14:52:09.792955 (+1070412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48896}
W20260709 14:52:09.668265  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 142904us
I20260709 14:52:09.795452  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 626) took 3781 ms. Trace:
I20260709 14:52:09.796130  3540 rpcz_store.cc:276] 0709 14:52:06.013654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.575266 (+561612us) service_pool.cc:224] Handling call
0709 14:52:07.995517 (+1420251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:07.996296 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.795136 (+1798840us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54004b37d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:09.795362 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3265340288}
I20260709 14:52:09.682890  2756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:09.663709  5167 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:09.700641  2681 rpcz_store.cc:276] 0709 14:52:07.973292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.187614 (+214322us) service_pool.cc:224] Handling call
0709 14:52:08.188135 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.392202 (+204067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.392240 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:08.416544 (+ 24304us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.249575 (+833031us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:09.249700 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.249831 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.625459 (+375628us) raft_consensus.cc:1685] finished
0709 14:52:09.625489 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.625698 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109167,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":7240,"thread_start_us":903287,"threads_started":2,"wal-append.queue_time_us":1005696}
W20260709 14:52:09.704087  3409 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.104s (SENT)
I20260709 14:52:09.799098  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:09.705912  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.679059  5157 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:09.674291  5122 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:09.678123  5180 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:09.677692  5170 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:52:09.686508  5074 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.045s
W20260709 14:52:09.697929  5037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.436s	user 0.022s	sys 0.029s
I20260709 14:52:09.678678  5176 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:09.729295  5174 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 14:52:09.730258  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.732337  5181 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:09.737112  5046 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.295s	user 0.015s	sys 0.002s
I20260709 14:52:09.740109  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1023) took 2687 ms. Trace:
W20260709 14:52:09.838457  5074 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.017s	sys 0.045s
I20260709 14:52:09.719883  5164 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.744330  5126 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.745726  5104 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:09.720528  5152 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:09.752630  5039 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:09.740270  4675 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.528s	user 0.015s	sys 0.000s
I20260709 14:52:09.756708  5073 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.757226  5166 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:09.637248  5175 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:09.469545  5075 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.779304  2609 rpcz_store.cc:276] 0709 14:52:07.801592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:07.801735 (+   143us) service_pool.cc:224] Handling call
0709 14:52:08.622452 (+820717us) spinlock_profiling.cc:243] Waited 924 us on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521f8c 00007fa16b521ef6 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:09.769898 (+1147446us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002ff8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:09.770145 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2632399360}
I20260709 14:52:09.724305  5183 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.674410  5161 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
W20260709 14:52:09.788633  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 326019us
I20260709 14:52:09.789157  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1017) took 3688 ms. Trace:
I20260709 14:52:09.795351  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:09.800477  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96430.1us
W20260709 14:52:09.677489  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.031s	sys 0.018s
W20260709 14:52:09.885955  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.031s	sys 0.019s
W20260709 14:52:09.812372  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 327708us
W20260709 14:52:09.888123  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.790292  2759 rpcz_store.cc:276] 0709 14:52:07.368331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.612176 (+1243845us) service_pool.cc:224] Handling call
0709 14:52:08.612637 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.613088 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.613121 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:08.613753 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.329891 (+716138us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:09.329946 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.330055 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.650079 (+320024us) raft_consensus.cc:1685] finished
0709 14:52:09.650106 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.650284 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6691,"raft.queue_time_us":17206,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":32577,"thread_start_us":715599,"threads_started":1,"wal-append.queue_time_us":857662}
I20260709 14:52:09.817037  2681 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:09.885617  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.884865  2753 rpcz_store.cc:276] 0709 14:52:06.100707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:06.102113 (+  1406us) service_pool.cc:224] Handling call
0709 14:52:06.102669 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:06.119076 (+ 16407us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:07.970401 (+1851325us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b18003dcc00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:52:08.528955 (+558554us) log.cc:844] Serialized 14 byte log entry
0709 14:52:09.427122 (+898167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:09.459962 (+ 32840us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.460153 (+   191us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:09.460193 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.460273 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.759021 (+298748us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b18003dcc00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:09.788806 (+ 29785us) raft_consensus.cc:1685] finished
0709 14:52:09.788841 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.789065 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26994,"prepare.queue_time_us":1877942,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":846,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":34616,"spinlock_wait_cycles":20770048,"thread_start_us":1283935,"threads_started":2,"wal-append.queue_time_us":802987}
W20260709 14:52:09.825098  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 343158us
W20260709 14:52:09.787355  4914 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.006s
I20260709 14:52:09.831693  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:09.836675  2682 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:09.841691  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:09.787662  5020 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:09.800925  5167 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:09.912254  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 250500us
I20260709 14:52:09.798519  4977 raft_consensus.cc:2764] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 14:52:09.845904  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 359588us
I20260709 14:52:09.846683  2679 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:09.817790  5165 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 14:52:09.850888  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 353765us
I20260709 14:52:09.841607  5077 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.685047  4995 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.944500  5197 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:09.945310  4995 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.946272  5197 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:09.948012  4995 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:09.837949  5037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.590s	user 0.023s	sys 0.030s
I20260709 14:52:09.852442  2751 rpcz_store.cc:276] 0709 14:52:07.054037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.595716 (+1541679us) service_pool.cc:224] Handling call
0709 14:52:08.596286 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.596582 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.596614 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:08.597224 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 14:52:08.964223 (+366999us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:08.964296 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:08.964418 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.616850 (+652432us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:09.739694 (+122844us) raft_consensus.cc:1685] finished
0709 14:52:09.739724 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.740020 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8208640,"thread_start_us":366377,"threads_started":1,"wal-append.queue_time_us":672709}
I20260709 14:52:09.948279  5197 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:09.864183  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 337713us
W20260709 14:52:09.951421  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.952831  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1005) took 1232 ms. Trace:
I20260709 14:52:09.829418  5157 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:09.875209  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:09.876701  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:09.838131  5170 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.881011  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:09.853483  5176 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:09.885613  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.912180  5193 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:09.914052  4914 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
W20260709 14:52:09.915307  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 120103us
I20260709 14:52:09.918227  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 412) took 2063 ms. Trace:
I20260709 14:52:09.910640  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.909361  5020 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:09.860896  5074 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:09.930018  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:09.844511  5051 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 14:52:09.806643  5179 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:09.936606  2726 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.676s (SENT)
I20260709 14:52:09.877983  5126 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.948916  5037 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:09.854032  4982 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:09.914433  5016 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
W20260709 14:52:09.869486  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 391191us
I20260709 14:52:09.953358  2608 rpcz_store.cc:276] 0709 14:52:08.720289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.119477 (+399188us) service_pool.cc:224] Handling call
0709 14:52:09.130216 (+ 10739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:09.199521 (+ 69305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:09.199556 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:09.206709 (+  7153us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.676588 (+469879us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:09.676648 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.676761 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.952563 (+275802us) raft_consensus.cc:1685] finished
0709 14:52:09.952589 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.952756 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68662,"raft.queue_time_us":76917,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":116,"thread_start_us":469153,"threads_started":1,"wal-append.queue_time_us":652113}
W20260709 14:52:09.940284  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:09.853916  5181 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.941367  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 919) took 2319 ms. Trace:
I20260709 14:52:10.037070  2752 rpcz_store.cc:276] 0709 14:52:07.622356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.657760 (+1035404us) service_pool.cc:224] Handling call
0709 14:52:08.658367 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.658772 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.658814 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:08.659536 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.456169 (+796633us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:09.456237 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.456387 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.941096 (+484709us) raft_consensus.cc:1685] finished
0709 14:52:09.941120 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.941288 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":732447,"threads_started":1,"wal-append.queue_time_us":761674}
I20260709 14:52:09.845543  5000 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:09.879091  5124 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.665s	user 0.015s	sys 0.000s
W20260709 14:52:09.939079  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:10.045423  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:09.990149  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 104602us
I20260709 14:52:10.048007  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.990152  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:10.051005  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:10.051399  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61281.6us
I20260709 14:52:10.052551  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:09.990334  2760 rpcz_store.cc:276] 0709 14:52:07.855221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.729389 (+874168us) service_pool.cc:224] Handling call
0709 14:52:08.763750 (+ 34361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:08.775287 (+ 11537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.775322 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:08.775913 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.392161 (+616248us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:09.581160 (+188999us) log.cc:844] Serialized 14 byte log entry
0709 14:52:09.913534 (+332374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.913637 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:09.913689 (+    52us) raft_consensus.cc:1685] finished
0709 14:52:09.913709 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:09.918125 (+  4416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98019,"prepare.queue_time_us":183222,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":479713,"raft.run_cpu_time_us":473,"raft.run_wall_time_us":71829,"thread_start_us":524616,"threads_started":2,"wal-append.queue_time_us":693633}
I20260709 14:52:09.994975  3540 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:09.997881  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61324.4us
W20260709 14:52:10.056351  2726 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.715s (SENT)
I20260709 14:52:09.858793  5046 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:09.883581  4675 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:09.955821  5190 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:52:09.957305  5103 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:10.068228  5195 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:09.929811  5104 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:10.068498  5000 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.068753  5074 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:10.069155  5104 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:10.069322  5195 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:10.071152  5194 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:10.070981  5059 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:10.071861  5104 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:09.934949  5184 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:10.072906  5187 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 14:52:10.072710  5184 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.073695  5198 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.074290  5198 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:09.940241  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:10.029732  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:10.027156  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116619us
I20260709 14:52:10.033789  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 389) took 2060 ms. Trace:
I20260709 14:52:09.931785  5189 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.973723  4995 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:52:10.041018  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 111170us
W20260709 14:52:10.062008  2726 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.735s (SENT)
W20260709 14:52:10.081270  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51666.2us
I20260709 14:52:09.870637  5122 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.084619  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:10.079728  2677 rpcz_store.cc:276] 0709 14:52:07.973588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:08.423596 (+450008us) service_pool.cc:224] Handling call
0709 14:52:08.424188 (+   592us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:08.424791 (+   603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:08.424840 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:08.450865 (+ 26025us) log.cc:844] Serialized 52 byte log entry
0709 14:52:09.886436 (+1435571us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:09.991938 (+105502us) log.cc:844] Serialized 14 byte log entry
0709 14:52:10.032985 (+ 41047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:10.033146 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:10.033215 (+    69us) raft_consensus.cc:1685] finished
0709 14:52:10.033235 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:10.033674 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13466,"raft.queue_time_us":48342,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":35690,"thread_start_us":1315030,"threads_started":2,"wal-append.queue_time_us":1354160}
I20260709 14:52:10.086203  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:10.087575  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:10.081807  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:10.065307  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:09.941955  5192 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.146s
I20260709 14:52:10.033470  5007 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:10.039513  5034 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:10.093748  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 176021us
I20260709 14:52:09.942019  5175 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 14:52:10.105989  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:10.101337  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:10.108866  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43664.5us
I20260709 14:52:10.095328  2680 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
W20260709 14:52:10.106943  2726 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.809s (SENT)
I20260709 14:52:10.110226  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:10.112949  2726 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.911s (SENT)
I20260709 14:52:10.113520  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:10.068810  5103 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:10.058633  4723 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:10.065693  4786 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:10.128492  5035 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:09.960673  5180 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:10.132443  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52308.7us
I20260709 14:52:10.116947  2685 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:10.040906  5016 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.104511  5020 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:10.079380  4983 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.005s
I20260709 14:52:10.077653  5058 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:09.954483  5140 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.287117  4983 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:52:10.192600  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86694.6us
I20260709 14:52:10.249171  5034 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 14:52:10.065270  5179 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:10.251008  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 201559us
W20260709 14:52:10.302464  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:10.254276  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:10.093189  5165 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:10.116591  4914 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:10.115741  5156 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:10.259065  4989 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:10.146593  5166 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:09.954550  5157 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:10.440622  5206 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 14:52:10.429708  5075 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:10.267393  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 157832us
I20260709 14:52:10.447932  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:10.448453  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.448773  2757 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:10.449373  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.450853  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.076025  5167 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:10.117475  5160 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.163s
I20260709 14:52:10.280627  5073 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:10.281068  5191 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:10.281306  5126 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:10.281656  5177 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:52:10.282178  5203 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:10.284852  5198 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:10.191051  5184 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:10.097147  4995 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:10.118820  5104 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.150s
I20260709 14:52:10.481073  5192 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:52:10.075109  5163 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:10.499631  5163 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:10.499724  5126 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.500142  5163 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:10.500509  5126 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:09.992894  5176 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.064652  5124 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:10.292830  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:10.084254  5059 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:10.305397  5154 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:10.308212  5164 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:10.288048  5041 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.933s	user 0.016s	sys 0.000s
I20260709 14:52:10.316313  4675 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:10.370958  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:10.141656  5007 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:10.374310  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:10.368608  5179 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:10.384891  4983 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:10.386888  5200 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 14:52:10.388552  5170 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:10.323081  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:10.282210  4982 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:10.180037  5194 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
I20260709 14:52:10.438532  4989 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.452008  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.277793  5066 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:52:10.453272  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:10.453025  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.449343  2754 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:10.455071  5165 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.455370  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.457856  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:10.460021  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:10.320341  5051 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 14:52:10.465076  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:10.463438  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:10.471580  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:10.425292  5020 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:10.477015  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 180886us
W20260709 14:52:10.478725  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:10.429157  5156 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:10.244258  5037 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:10.488065  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:10.492522  5219 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.493224  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:10.495322  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:10.497258  5181 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:10.497107  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:10.498687  5189 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:52:10.560220  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:10.516366  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 194986us
W20260709 14:52:10.645350  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 391182us
I20260709 14:52:10.741775  2757 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
W20260709 14:52:10.657666  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:10.770744  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.053614  2678 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:10.908728  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:11.079145  2607 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:11.081002  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:10.908661  2686 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:10.907249  2605 raft_consensus.cc:2376] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7. Re-sending same reply.
W20260709 14:52:11.084491  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 791731us
I20260709 14:52:10.898566  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.052696  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:10.872443  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102911us
I20260709 14:52:11.086625  2610 raft_consensus.cc:2376] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9. Re-sending same reply.
W20260709 14:52:10.867353  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:11.077098  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.094744  2608 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 14:52:11.101257  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.092793  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:11.093946  2753 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:11.087226  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.077236  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:10.670246  5222 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:10.907485  2760 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:10.672741  5153 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:10.883411  2683 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:10.494463  4797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.016s
I20260709 14:52:11.118222  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:10.692173  5215 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:11.088430  2751 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:52:11.093549  2726 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.975s (SENT)
W20260709 14:52:11.090461  5157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.020s	sys 0.056s
W20260709 14:52:10.670820  5145 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.951s	user 0.008s	sys 0.003s
I20260709 14:52:10.519178  5191 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.109135  2611 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:11.163892  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:10.973790  5170 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:11.103668  2679 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:11.109304  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:10.872915  5237 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.153s	user 0.002s	sys 0.002s
I20260709 14:52:11.099924  5070 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:11.173797  5070 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:11.174276  5070 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:10.923556  4983 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.177471  4983 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.098042  5041 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.131089  5222 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.180171  5041 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.181315  5041 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:52:11.182317  5222 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.090173  5016 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.183483  3547 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:11.184173  5218 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.184806  5218 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.185331  5070 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.185990  5059 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:11.186609  5059 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:10.506458  5000 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.637s
I20260709 14:52:11.127661  5153 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:11.187731  5059 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.188573  5070 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.189082  5153 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.190351  5124 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:11.144713  5157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.021s	sys 0.056s
I20260709 14:52:10.923888  5165 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.192711  5165 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:11.193843  5163 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.693s	user 0.006s	sys 0.007s
W20260709 14:52:11.194689  5159 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.428s	user 0.009s	sys 0.006s
I20260709 14:52:11.195194  5159 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.195335  5170 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:11.169983  5027 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 14:52:11.196564  5027 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:11.196981  5027 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:11.197752  5159 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.198386  5170 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.111805  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:10.502790  5126 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:10.519158  5203 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.232168  5203 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.089998  4995 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.235349  4723 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:10.482231  5077 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.235823  5126 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.103466  2609 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:11.236543  5126 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:11.237375  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:11.238269  2609 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:10.922708  5179 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:10.672132  5069 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:52:11.126968  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:11.255370  3541 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:11.126888  2607 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:52:11.128017  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:11.256215  3536 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:11.083876  5167 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:11.124379  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:10.526597  5140 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:11.094691  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 629712us
I20260709 14:52:11.289005  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.134013  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:11.134812  2752 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:11.141112  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:11.141422  2612 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:11.152850  2758 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 14:52:11.154604  2754 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 14:52:11.293052  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 199516us
I20260709 14:52:11.092478  5156 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:11.157231  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64535us
I20260709 14:52:11.291816  2612 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:10.881856  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 166553us
I20260709 14:52:11.091046  5051 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.165083  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.326491  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:11.326741  2754 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:11.326593  3546 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:11.118702  4797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.031s	sys 0.063s
I20260709 14:52:11.096915  5103 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.165792  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:11.167693  5191 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:11.172012  5227 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:52:11.172266  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.367877  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.368642  2611 raft_consensus.cc:2368] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:11.368829  3541 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:11.172835  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63609.3us
I20260709 14:52:11.188241  5000 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:11.097615  4989 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:10.923429  5219 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.140918  4977 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.111853  5176 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.112372  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.114126  5194 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:11.212260  4994 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 14:52:11.202566  5152 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.265s	user 0.008s	sys 0.003s
W20260709 14:52:11.205197  5039 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.322s	user 0.016s	sys 0.000s
I20260709 14:52:11.133291  5215 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:11.217530  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 399) took 1141 ms. Trace:
I20260709 14:52:11.226152  2605 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:52:11.194353  5163 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:11.230942  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.119345  2604 raft_consensus.cc:2376] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:11.242170  2678 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:11.268308  3543 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:11.234453  5068 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:11.315301  2726 leader_election.cc:336] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.258s (SENT)
I20260709 14:52:11.327368  2758 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:11.366632  3542 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:11.323649  2612 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.354385  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:11.461524  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:11.370355  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:11.378162  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 265842us
I20260709 14:52:11.376176  2754 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:11.402004  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:11.432801  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 143877us
I20260709 14:52:11.463677  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:11.404564  3536 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
W20260709 14:52:11.405826  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 309644us
I20260709 14:52:11.425050  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:11.426342  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.254696  5203 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.465741  2611 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:11.235860  4723 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:11.425266  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.465803  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.326812  4786 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:11.484889  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.326311  5104 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:11.491914  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.494002  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.496675  3536 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:11.467005  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.468649  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:11.464685  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.269248  5069 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:11.472738  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102512us
I20260709 14:52:11.502077  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:11.504077  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:11.472772  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.354892  5230 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.475073  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.475327  4797 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:11.464829  2686 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:11.497589  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:11.474388  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 159165us
I20260709 14:52:11.508451  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.445792  5126 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.456705  2681 rpcz_store.cc:276] 0709 14:52:10.076564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:10.076741 (+   177us) service_pool.cc:224] Handling call
0709 14:52:10.077132 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:10.077655 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:10.140752 (+ 63097us) log.cc:844] Serialized 14 byte log entry
0709 14:52:10.463297 (+322545us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:10.471165 (+  7868us) log.cc:844] Serialized 26 byte log entry
0709 14:52:10.471354 (+   189us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:10.471390 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:10.471468 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:11.217207 (+745739us) raft_consensus.cc:1685] finished
0709 14:52:11.217234 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:11.217429 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45852,"prepare.queue_time_us":389436,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":40379,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":9741,"thread_start_us":283660,"threads_started":2,"wal-append.queue_time_us":964063}
I20260709 14:52:11.543066  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:11.544234  2681 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:11.512187  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:11.512739  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44229.9us
I20260709 14:52:11.465600  5152 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.515411  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:11.518057  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.518591  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.554987  2679 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:11.378432  5000 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:11.447140  5103 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:11.327644  5216 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:11.532609  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.533979  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.558745  2754 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:11.559413  2607 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:11.560554  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.529824  2684 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:11.536203  5205 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:11.536834  5222 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:11.538076  4797 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:11.370986  5161 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.142s
I20260709 14:52:11.328471  5035 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.186s
I20260709 14:52:11.538960  5234 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:11.353793  5051 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:11.539920  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.591712  3541 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:11.515316  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.468099  5184 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.557742  2681 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:11.564390  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.434512  4977 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.566984  5103 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:11.567610  2682 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:11.521593  5077 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:11.567097  5233 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:11.570391  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.605808  2754 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:11.607084  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:11.506690  5218 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.531702  5203 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:11.579291  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.356302  5214 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.216s
I20260709 14:52:11.580526  2610 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:11.583976  2758 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:52:11.584965  2679 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 14:52:11.588357  3542 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:11.528184  5160 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:11.527611  5104 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:11.593724  3536 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:11.500209  5068 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 14:52:11.577986  5205 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:11.621588  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:11.622040  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.529695  5230 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.599578  5236 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.602313  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:11.675009  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:11.563550  4995 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:11.570278  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.541069  5039 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 14:52:11.610270  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.598709  5103 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:11.540237  5215 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.614562  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:11.696980  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 82508.8us
I20260709 14:52:11.610244  5203 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:11.615542  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.551371  5066 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.615765  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.616425  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.568950  4675 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:11.690629  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.567417  5069 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:11.693707  5240 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 14:52:11.675063  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:11.682901  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.690588  5068 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:11.601162  4899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.002s	sys 0.024s
I20260709 14:52:11.690095  5165 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:11.752205  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:11.798498  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 300958us
I20260709 14:52:11.773422  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:11.800352  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.693753  2608 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:11.775341  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.803929  2754 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:11.804579  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 286632us
I20260709 14:52:11.805160  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.806173  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:11.808138  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.704732  2607 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:11.777130  3542 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:11.694629  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 433987us
I20260709 14:52:11.669154  5077 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:11.815478  5165 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.816416  5231 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:11.816519  5077 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.817252  4995 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.817535  5231 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:11.817893  4995 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.818353  5231 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:11.690959  5217 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:11.788157  5059 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.602221  5184 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.777077  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:11.781318  5035 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 14:52:11.792637  4899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.002s	sys 0.024s
I20260709 14:52:11.790153  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.697595  5037 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:11.762125  5203 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:11.691392  5191 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.853333  2756 raft_consensus.cc:606] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.701911  5233 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:11.811273  5069 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:11.783952  5215 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:11.669914  5205 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:11.787693  5039 consensus_queue.cc:1243] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:11.819268  5217 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:11.846949  2758 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:11.846287  5037 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:11.883119  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.706537  5103 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:11.799963  5068 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.892846  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:11.851181  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.893682  2758 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:11.669238  5160 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.855078  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.814672  5224 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:11.669188  5104 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:11.775463  5066 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:11.866482  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 358093us
I20260709 14:52:11.820288  5046 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.834749  5176 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.703191  5000 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.911823  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.913606  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:11.834690  5165 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.819491  5059 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.844201  5192 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:11.883332  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:11.778342  4797 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:11.889559  5217 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:11.903117  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 748) took 2053 ms. Trace:
I20260709 14:52:11.917387  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:11.925557  3540 rpcz_store.cc:276] 0709 14:52:09.849677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.867213 (+ 17536us) service_pool.cc:224] Handling call
0709 14:52:09.905644 (+ 38431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:10.055670 (+150026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:11.897314 (+1841644us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:11.903040 (+  5726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204706560}
I20260709 14:52:11.902349  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.897912  5219 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.901067  2754 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:11.918524  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:11.923072  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:11.893481  4995 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:11.894443  5163 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.699481  5236 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:11.871047  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.902334  5051 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:11.931286  2686 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:11.937353  2753 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:11.947485  2758 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:11.923138  5254 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.960914  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.961983  2758 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:11.949963  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 150606us
I20260709 14:52:11.923691  4786 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:11.956174  2686 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:11.957350  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:11.966267  3540 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:11.922799  5192 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:11.927850  5140 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:11.925271  5231 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:11.932709  4989 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:11.934448  5222 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:11.973028  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.984141  2758 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:11.985663  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:11.979323  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:11.992556  3540 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
W20260709 14:52:12.006361  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 198330us
I20260709 14:52:11.993135  3540 pending_rounds.cc:85] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 3
I20260709 14:52:12.009469  3540 pending_rounds.cc:107] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 14:52:12.009828  3540 raft_consensus.cc:2889] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:12.009969  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:11.961580  5198 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:11.919018  5176 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.027315  5242 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 14:52:12.044201  2652 leader_election.cc:400] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:12.025578  5214 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 14:52:12.008718  5222 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.045533  5222 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:12.050304  4995 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.048285  5255 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.044764  5262 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:11.983188  5248 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.224s	user 0.000s	sys 0.002s
I20260709 14:52:11.970767  5254 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.027065  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:11.917217  5233 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:11.968438  5192 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.060521  3536 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 3. Current term is 4. Ops: []
I20260709 14:52:12.045759  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 412) took 1285 ms. Trace:
I20260709 14:52:12.052832  4989 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:12.073999  3541 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:12.077050  5176 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.085532  2608 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:12.088716  2679 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 9. Ops: []
W20260709 14:52:12.091074  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.092752  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:12.093133  2726 leader_election.cc:336] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.596s (SENT)
I20260709 14:52:12.096068  2606 rpcz_store.cc:276] 0709 14:52:10.759864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:10.760016 (+   152us) service_pool.cc:224] Handling call
0709 14:52:10.898968 (+138952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:10.899436 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:10.899470 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:10.974010 (+ 74540us) log.cc:844] Serialized 26 byte log entry
0709 14:52:11.201175 (+227165us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:11.201232 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:11.201314 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.045510 (+844196us) raft_consensus.cc:1685] finished
0709 14:52:12.045532 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.045690 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1859,"raft.queue_time_us":561,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":33553,"thread_start_us":96193,"threads_started":1,"wal-append.queue_time_us":445018}
W20260709 14:52:12.085651  5249 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 14:52:12.084718  5219 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.039321  5035 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:11.994644  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 103295us
I20260709 14:52:12.102638  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:12.066445  4995 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.107479  5068 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:12.107542  5246 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.231s	user 0.001s	sys 0.001s
W20260709 14:52:12.109139  5251 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 14:52:12.110204  5059 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.111313  3540 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:12.106218  5184 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:12.113618  5156 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.001s
I20260709 14:52:11.991750  5231 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.100929  5253 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:12.144332  3540 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:12.121179  4786 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:12.122926  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:12.099865  4989 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:12.133733  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.156651  2608 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:12.103297  5051 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:12.110769  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.092348  5262 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.138167  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.138937  5068 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:52:12.140964  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.163792  5191 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:12.142086  5165 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 14:52:12.146966  3540 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:12.148029  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.084348  5255 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.175331  5007 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.091789  4723 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.166544  5254 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:12.164203  5167 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.881s	user 0.010s	sys 0.004s
I20260709 14:52:12.161393  4989 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:12.168778  2684 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:12.174410  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.149068  5253 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:12.173012  5257 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 14:52:12.200515  2606 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:12.203931  3540 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:12.123634  5219 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.160837  5039 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:12.208396  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.160568  5066 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:12.164805  5264 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:12.212905  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.225009  2608 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:12.176851  5191 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:12.146641  5059 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 14:52:12.178037  4786 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.233253  4786 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:12.236702  4994 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.000s
I20260709 14:52:12.147360  5156 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.208881  5159 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.169050  5051 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.230644  5059 consensus_queue.cc:1243] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:12.207665  5255 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.186796  5007 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.146438  5035 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:12.226578  4723 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.227984  5160 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.253588  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:12.260152  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.241358  4786 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.227623  5039 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.275348  5268 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 14:52:12.276467  5222 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.277300  5153 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.280014  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.284318  2612 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.287408  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.288612  2608 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:12.292218  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.292906  5253 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.293327  2686 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:12.289947  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.295336  4995 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.295169  5217 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:12.296661  5179 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.009s	user 0.007s	sys 0.007s
I20260709 14:52:12.260841  5007 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.297371  5179 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:12.298990  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.300348  2757 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:12.302690  5007 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:12.304472  5027 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.107s	user 0.014s	sys 0.003s
I20260709 14:52:12.306427  5027 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.305567  5020 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.206s	user 0.011s	sys 0.005s
I20260709 14:52:12.313010  5156 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.313488  4994 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:12.314568  4994 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:12.314985  4994 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:12.315976  4994 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.317358  4994 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:12.244827  5232 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.320410  2608 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:12.243559  5191 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.246349  5159 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:12.347852  2611 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 14:52:12.353549  5239 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.156s	user 0.004s	sys 0.006s
I20260709 14:52:12.319586  5232 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.357822  5262 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.358047  5027 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.360821  5236 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:12.360983  4675 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.003s
I20260709 14:52:12.361524  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.359988  5219 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.340763  5051 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:12.366097  3538 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:12.366703  5217 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.378340  5231 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.380167  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 863) took 1033 ms. Trace:
I20260709 14:52:12.380625  3536 rpcz_store.cc:276] 0709 14:52:11.347340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.767795 (+420455us) service_pool.cc:224] Handling call
0709 14:52:11.768326 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:12.094419 (+326093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.379870 (+285451us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:12.380095 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671417088}
W20260709 14:52:12.384349  5203 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 14:52:12.384894  5203 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:12.389382  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.390515  5230 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:12.377029  5185 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.579s	user 0.001s	sys 0.017s
I20260709 14:52:12.394449  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1007) took 3673 ms. Trace:
I20260709 14:52:12.395040  2603 rpcz_store.cc:276] 0709 14:52:08.720809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.179004 (+458195us) service_pool.cc:224] Handling call
0709 14:52:09.179455 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:09.179786 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:09.179824 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:09.180450 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 14:52:09.608919 (+428469us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:09.608976 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:09.609083 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.362508 (+2753425us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:12.393994 (+ 31486us) raft_consensus.cc:1685] finished
0709 14:52:12.394029 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.394353 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39404928,"thread_start_us":427982,"threads_started":1,"wal-append.queue_time_us":603181}
I20260709 14:52:12.397121  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1041) took 1249 ms. Trace:
I20260709 14:52:12.397660  2760 rpcz_store.cc:276] 0709 14:52:11.148311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.148692 (+   381us) service_pool.cc:224] Handling call
0709 14:52:11.149299 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:11.203483 (+ 54184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:11.203538 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:11.225466 (+ 21928us) log.cc:844] Serialized 26 byte log entry
0709 14:52:11.776447 (+550981us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:11.776522 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.396684 (+620162us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:12.396742 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.396791 (+    49us) raft_consensus.cc:1685] finished
0709 14:52:12.396807 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.397055 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1050,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":179,"spinlock_wait_cycles":123143552,"thread_start_us":404550,"threads_started":1,"wal-append.queue_time_us":545471}
I20260709 14:52:12.400134  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.397111  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:12.403563  2680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 395) took 2581 ms (client timeout 1574 ms). Trace:
I20260709 14:52:12.327409  5103 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.392400  5152 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:12.407759  2680 rpcz_store.cc:269] 0709 14:52:09.821807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.831362 (+  9555us) service_pool.cc:224] Handling call
0709 14:52:12.403246 (+2571884us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540024ead4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:12.403423 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313974016}
I20260709 14:52:12.402585  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:12.392016  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.409849  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.404518  2760 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:12.405999  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.412298  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:12.417869  5212 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 2.016s	user 0.005s	sys 0.013s
I20260709 14:52:12.422785  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1036) took 2532 ms. Trace:
I20260709 14:52:12.423431  2759 rpcz_store.cc:276] 0709 14:52:09.890658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.894326 (+  3668us) service_pool.cc:224] Handling call
0709 14:52:09.894935 (+   609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:09.909649 (+ 14714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:09.909684 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:09.934288 (+ 24604us) log.cc:844] Serialized 26 byte log entry
0709 14:52:10.061120 (+126832us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:10.061308 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:10.061614 (+   306us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.406344 (+2344730us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:12.422347 (+ 16003us) raft_consensus.cc:1685] finished
0709 14:52:12.422381 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.422703 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14134,"raft.queue_time_us":20483,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":13332,"spinlock_wait_cycles":53008640,"thread_start_us":44015,"threads_started":1,"wal-append.queue_time_us":280841}
I20260709 14:52:12.425845  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:12.418740  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.430137  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.372524  5192 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.405333  5205 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.003s
I20260709 14:52:12.430846  2680 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:12.432212  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.434382  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.435106  2760 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:12.436379  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.438283  2760 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 5: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:12.375303  2686 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:12.440846  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:12.441377  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.442127  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.442762  2680 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.444907  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.445788  2759 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:12.409914  2679 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:12.447048  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.451731  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:12.442729  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.452723  2760 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.455719  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.457091  2603 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:12.460232  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.463447  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.470533  2759 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:12.472292  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.475522  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.476462  2603 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:12.407433  4675 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:12.477051  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1019) took 1073 ms. Trace:
I20260709 14:52:12.487600  2603 rpcz_store.cc:276] 0709 14:52:11.403304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.463034 (+1059730us) service_pool.cc:224] Handling call
0709 14:52:12.476987 (+ 13953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.490504  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.445614  5041 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:12.445818  5222 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.491339  2759 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:12.500768  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.502485  2759 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:12.489447  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.493913  3538 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:12.452214  4786 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:12.432969  5039 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:12.474341  5069 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.005s
I20260709 14:52:12.506424  2603 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.467428  5073 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:12.433038  5205 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.559226  5073 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.559746  5073 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:12.537510  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.529433  5203 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:12.561750  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.562893  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:12.564762  2759 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:12.536490  2680 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:12.499048  5041 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.565678  5041 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.560997  5205 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:12.626394  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.617666  2681 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:12.568099  2682 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:12.577775  2603 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:12.613093  2760 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:12.631567  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:12.633409  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.633862  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.634588  2680 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:12.635450  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.627324  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.636457  2680 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:12.626423  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:12.626147  4786 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:12.638789  4786 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.640751  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 422) took 1116 ms. Trace:
I20260709 14:52:12.641247  2603 rpcz_store.cc:276] 0709 14:52:11.524544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.489031 (+964487us) service_pool.cc:224] Handling call
0709 14:52:12.640673 (+151642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.642163  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.637480  3538 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.643343  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:12.645359  2680 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:12.570762  5274 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 14:52:12.647135  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:12.626693  5215 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.007s
I20260709 14:52:12.647758  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.521612  4675 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:12.657709  5066 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:12.658352  5066 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.659447  2680 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:12.647480  5191 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:12.656525  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 393) took 2836 ms. Trace:
I20260709 14:52:12.657450  4675 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.522590  5179 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.573606  5103 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.660456  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.607394  5192 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:12.664058  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.664588  2685 rpcz_store.cc:276] 0709 14:52:09.819851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.820737 (+   886us) service_pool.cc:224] Handling call
0709 14:52:12.649564 (+2828827us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:12.656442 (+  6878us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481881344}
I20260709 14:52:12.665241  2603 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:12.679319  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.680193  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1024) took 1275 ms. Trace:
I20260709 14:52:12.680567  2603 rpcz_store.cc:276] 0709 14:52:11.404929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.678985 (+1274056us) service_pool.cc:224] Handling call
0709 14:52:12.680116 (+  1131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.681512  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.666554  2612 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:12.648000  5205 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:12.682502  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 417) took 1920 ms. Trace:
I20260709 14:52:12.683379  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:12.684052  3538 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:12.645761  5155 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 14:52:12.711598  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:12.712098  4675 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:12.684578  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1040) took 1096 ms. Trace:
I20260709 14:52:12.713227  2603 rpcz_store.cc:276] 0709 14:52:11.587904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.681277 (+1093373us) service_pool.cc:224] Handling call
0709 14:52:12.684508 (+  3231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.715314  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.716317  5066 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.717015  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 421) took 1192 ms. Trace:
I20260709 14:52:12.706521  2612 rpcz_store.cc:276] 0709 14:52:10.761892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.109658 (+347766us) service_pool.cc:224] Handling call
0709 14:52:11.141277 (+ 31619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:12.283829 (+1142552us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b58000b8dd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5a20aa 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4
0709 14:52:12.682191 (+398362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.682447 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885344128}
I20260709 14:52:12.718727  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.719238  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.719830  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.720907  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1027) took 1247 ms. Trace:
I20260709 14:52:12.713058  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.720875  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.724156  2603 rpcz_store.cc:276] 0709 14:52:11.524147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.714810 (+1190663us) service_pool.cc:224] Handling call
0709 14:52:12.716885 (+  2075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.724314  3543 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:12.721742  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 414) took 1960 ms. Trace:
I20260709 14:52:12.721832  2612 rpcz_store.cc:276] 0709 14:52:11.473465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.718571 (+1245106us) service_pool.cc:224] Handling call
0709 14:52:12.720752 (+  2181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.724258  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.736581  2605 rpcz_store.cc:276] 0709 14:52:10.760885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.085101 (+324216us) service_pool.cc:224] Handling call
0709 14:52:11.085492 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:11.456807 (+371315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.721466 (+1264659us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540016dad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:12.721675 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319835520}
I20260709 14:52:12.739306  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.739871  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.740576  2603 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:12.737974  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.741245  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.741708  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1035) took 1155 ms. Trace:
I20260709 14:52:12.742208  2605 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:12.742884  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1031) took 1157 ms. Trace:
I20260709 14:52:12.743357  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.744328  3538 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:12.742421  2612 rpcz_store.cc:276] 0709 14:52:11.586590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.737452 (+1150862us) service_pool.cc:224] Handling call
0709 14:52:12.741630 (+  4178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.745842  3538 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:12.744171  2603 rpcz_store.cc:276] 0709 14:52:11.585686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.738888 (+1153202us) service_pool.cc:224] Handling call
0709 14:52:12.742782 (+  3894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.746978  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:12.747220  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.750192  2605 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:12.750947  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1042) took 1162 ms. Trace:
I20260709 14:52:12.751461  2605 rpcz_store.cc:276] 0709 14:52:11.588364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.746902 (+1158538us) service_pool.cc:224] Handling call
0709 14:52:12.750866 (+  3964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.752665  3543 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:12.748427  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.754487  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:12.678550  5191 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:12.756052  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:12.757234  5191 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.758313  4675 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:12.758750  4675 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.759182  4675 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.760746  2680 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:12.761355  5255 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:12.684629  5039 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.757623  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:12.764874  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81630.9us
I20260709 14:52:12.767107  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:12.747746  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.769104  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 427) took 1240 ms. Trace:
I20260709 14:52:12.769835  2603 rpcz_store.cc:276] 0709 14:52:11.528229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.748112 (+1219883us) service_pool.cc:224] Handling call
0709 14:52:12.769006 (+ 20894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.779764  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.742352  5203 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:12.743357  5066 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:12.764535  5179 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:12.777454  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.763387  5039 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.718730  4989 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.783226  5155 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:12.801867  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 876) took 1306 ms. Trace:
I20260709 14:52:12.785322  5192 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.811326  2685 rpcz_store.cc:276] 0709 14:52:11.496851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.707392 (+1210541us) service_pool.cc:224] Handling call
0709 14:52:12.709835 (+  2443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:12.710070 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:12.710440 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 14:52:12.801236 (+ 90796us) log.cc:844] Serialized 14 byte log entry
0709 14:52:12.801431 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:12.801490 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:12.801549 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.801636 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.801790 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1701,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":234,"thread_start_us":89866,"threads_started":1}
I20260709 14:52:12.792110  5203 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.792508  5066 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:12.800776  5069 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:12.806360  5231 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:12.827020  3537 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:12.827585  3537 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:12.827782  2685 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 14:52:12.811307  5155 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:12.809976  5179 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:12.830327  3537 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.821985  5203 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:12.825851  5066 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:12.801762  5205 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:12.832535  3537 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.820046  5007 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 14:52:12.831146  5155 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:12.816406  5192 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.857304  5179 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:12.862449  5197 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:12.864013  5197 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:12.864347  5205 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:12.865253  5039 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.868984  5041 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.873679  5271 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.632s	user 0.003s	sys 0.000s
I20260709 14:52:12.833408  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 880) took 1226 ms. Trace:
I20260709 14:52:12.772038  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.925760  2685 rpcz_store.cc:276] 0709 14:52:11.603866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.826975 (+1223109us) service_pool.cc:224] Handling call
0709 14:52:12.827433 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:12.828263 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.829822 (+  1559us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.927357  2603 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:12.752767  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.931960  2605 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:12.933617  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.934973  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1032) took 1348 ms. Trace:
I20260709 14:52:12.935606  2612 rpcz_store.cc:276] 0709 14:52:11.585982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.777034 (+1191052us) service_pool.cc:224] Handling call
0709 14:52:12.934642 (+157608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.936795  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.869292  4977 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:12.930505  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:12.968420  5103 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.972506  5231 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.972280  4989 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:12.973523  5041 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:12.941700  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 406) took 1118 ms. Trace:
I20260709 14:52:12.937179  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.940101  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 959) took 2477 ms. Trace:
I20260709 14:52:12.942559  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1041) took 1354 ms. Trace:
I20260709 14:52:12.943578  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 394) took 3122 ms. Trace:
I20260709 14:52:12.941238  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 938) took 1023 ms. Trace:
I20260709 14:52:12.952953  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 937) took 1042 ms. Trace:
I20260709 14:52:12.933645  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.987664  2604 rpcz_store.cc:276] 0709 14:52:11.588168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.932785 (+1344617us) service_pool.cc:224] Handling call
0709 14:52:12.942397 (+  9612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.987545  2678 rpcz_store.cc:276] 0709 14:52:10.462434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:10.462556 (+   122us) service_pool.cc:224] Handling call
0709 14:52:12.939403 (+2476847us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:12.939771 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334479104}
I20260709 14:52:12.992029  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:12.992462  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:12.992789  5069 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:12.993395  2604 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:12.994009  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 429) took 1170 ms. Trace:
I20260709 14:52:12.993862  5069 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:12.990082  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1037) took 2939 ms (client timeout 1716 ms). Trace:
W20260709 14:52:13.013999  2752 rpcz_store.cc:269] 0709 14:52:10.050399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:10.050600 (+   201us) service_pool.cc:224] Handling call
0709 14:52:12.986806 (+2936206us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000d0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:12.989731 (+  2925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2489443072}
I20260709 14:52:13.019266  2752 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:12.994663  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:12.994452  2604 rpcz_store.cc:276] 0709 14:52:11.823147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.991429 (+1168282us) service_pool.cc:224] Handling call
0709 14:52:12.993916 (+  2487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:12.988319  2683 rpcz_store.cc:276] 0709 14:52:09.821151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.821422 (+   271us) service_pool.cc:224] Handling call
0709 14:52:09.837644 (+ 16222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:11.116915 (+1279271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.942753 (+1825838us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400377154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:12.943277 (+   524us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404205952}
W20260709 14:52:12.998926  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.987284  2684 rpcz_store.cc:276] 0709 14:52:11.822741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.918162 (+ 95421us) service_pool.cc:224] Handling call
0709 14:52:12.941123 (+1022961us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:12.941393 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538699520}
I20260709 14:52:13.003933  2605 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:13.007882  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:12.996121  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 275005us
I20260709 14:52:13.009804  5103 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.010787  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:13.013041  5192 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.019032  5286 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.000981  5231 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.036751  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.038362  2605 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:12.988541  2760 rpcz_store.cc:276] 0709 14:52:11.910206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.632320 (+722114us) service_pool.cc:224] Handling call
0709 14:52:12.635229 (+  2909us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:12.642034 (+  6805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:12.642082 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:12.649611 (+  7529us) log.cc:844] Serialized 52 byte log entry
0709 14:52:12.764426 (+114815us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:12.764470 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.764548 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.952074 (+187526us) raft_consensus.cc:1685] finished
0709 14:52:12.952128 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.952613 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12514,"raft.queue_time_us":6393,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":4378,"thread_start_us":67868,"threads_started":1,"wal-append.queue_time_us":143497}
W20260709 14:52:12.999011  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.040814  5192 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.044616  5192 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.042627  5231 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:13.040567  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:12.990584  2612 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:13.031811  2678 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:13.035482  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.032805  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:13.033181  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1052) took 1057 ms. Trace:
I20260709 14:52:13.023557  5069 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.038760  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:13.039217  5041 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:13.050740  2752 rpcz_store.cc:276] 0709 14:52:11.971589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.017643 (+1046054us) service_pool.cc:224] Handling call
0709 14:52:13.018617 (+   974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:13.027316 (+  8699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.029228 (+  1912us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.050037  2603 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:13.056928  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.061664  2755 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:13.064910  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.36806e+06us
I20260709 14:52:13.065254  4989 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:13.066112  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1021) took 1662 ms (client timeout 1911 ms). Trace:
I20260709 14:52:13.066550  4989 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.069676  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.070159  4989 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.063706  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:13.072081  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.073963  3546 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 14:52:13.076030  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68700.1us
I20260709 14:52:13.076568  5184 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:13.077153  5184 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:13.077597  5103 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.059661  5041 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:13.080322  5041 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:13.081126  5058 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 14:52:13.083990  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.084343  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:13.077745  5184 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:12.988402  2759 rpcz_store.cc:276] 0709 14:52:11.918007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.726952 (+808945us) service_pool.cc:224] Handling call
0709 14:52:12.765126 (+ 38174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:12.765549 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:12.765581 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:12.790078 (+ 24497us) log.cc:844] Serialized 26 byte log entry
0709 14:52:12.938662 (+148584us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:12.938801 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:12.940482 (+  1681us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:12.940588 (+   106us) raft_consensus.cc:1685] finished
0709 14:52:12.940609 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:12.940896 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":334,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":12573,"thread_start_us":121132,"threads_started":1,"wal-append.queue_time_us":85416}
I20260709 14:52:13.093168  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 875) took 1601 ms. Trace:
I20260709 14:52:13.089174  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.048980  5192 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:13.099597  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.079195  4989 raft_consensus.cc:2764] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 14:52:13.087752  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:13.109055  2609 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 14:52:13.124671  2611 rpcz_store.cc:269] 0709 14:52:11.403944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.553724 (+149780us) service_pool.cc:224] Handling call
0709 14:52:13.065398 (+1511674us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b540042dbd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.065790 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694455040}
W20260709 14:52:13.096428  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58049.2us
I20260709 14:52:13.077489  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.091552  2760 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:13.101717  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1049) took 1401 ms. Trace:
I20260709 14:52:13.097702  3537 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 14:52:13.127345  2682 rpcz_store.cc:276] 0709 14:52:11.491955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.090527 (+598572us) service_pool.cc:224] Handling call
0709 14:52:12.091650 (+  1123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:12.630795 (+539145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.090523 (+459728us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400269154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.092888 (+  2365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1095180160}
I20260709 14:52:13.139169  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.136721  3537 pending_rounds.cc:85] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:13.096869  5197 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:13.140826  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.089430  4982 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:13.101909  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:13.090091  5170 raft_consensus.cc:993] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:52:13.144135  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 941) took 1015 ms. Trace:
I20260709 14:52:13.049033  5286 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.092226  5235 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:13.136168  2753 rpcz_store.cc:276] 0709 14:52:11.699911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.023286 (+323375us) service_pool.cc:224] Handling call
0709 14:52:13.099868 (+1076582us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:13.100329 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.100938 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.100961 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.101056 (+    95us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:13.101142 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.101234 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.101423 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3536768}
I20260709 14:52:13.143390  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1023) took 1729 ms. Trace:
I20260709 14:52:13.147464  2760 rpcz_store.cc:276] 0709 14:52:12.122566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.086993 (+964427us) service_pool.cc:224] Handling call
0709 14:52:13.087842 (+   849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.135787 (+ 47945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.138217 (+  2430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.086076  4786 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 14:52:13.147778  3537 pending_rounds.cc:107] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 14:52:13.232123  2612 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:13.164283  3546 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:13.240221  2726 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.816s (SENT)
I20260709 14:52:13.240993  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:13.241932  5192 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:13.243073  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100865us
I20260709 14:52:13.242892  5192 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.245363  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.246176  5192 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.246333  2677 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:13.252578  5218 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:13.253310  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.253844  5218 consensus_queue.cc:1243] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:13.253847  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 881) took 1582 ms. Trace:
I20260709 14:52:13.254551  2685 rpcz_store.cc:276] 0709 14:52:11.671251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.927875 (+1256624us) service_pool.cc:224] Handling call
0709 14:52:12.929256 (+  1381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:12.930325 (+  1069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:12.930384 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:13.030177 (+ 99793us) log.cc:844] Serialized 26 byte log entry
0709 14:52:13.059840 (+ 29663us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:13.059886 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.059996 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:13.253577 (+193581us) raft_consensus.cc:1685] finished
0709 14:52:13.253596 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.253758 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37243,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":509,"thread_start_us":27439,"threads_started":1,"wal-append.queue_time_us":219508}
I20260709 14:52:13.257961  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:13.256397  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.236418  3537 raft_consensus.cc:2889] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:13.144914  5197 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.254563  2677 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:13.207940  2678 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:13.158396  2684 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:13.154331  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1036) took 1567 ms (client timeout 1816 ms). Trace:
I20260709 14:52:13.149222  2609 rpcz_store.cc:276] 0709 14:52:11.404586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.107907 (+1703321us) service_pool.cc:224] Handling call
0709 14:52:13.108574 (+   667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:13.133466 (+ 24892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.133839 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:13.155880  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 419) took 2052 ms (client timeout 1641 ms). Trace:
I20260709 14:52:13.165944  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:13.157503  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.273451  5165 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:13.161500  5161 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:13.168928  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1040) took 2029 ms. Trace:
W20260709 14:52:13.275822  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 198599us
I20260709 14:52:13.273475  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 858) took 3302 ms. Trace:
W20260709 14:52:13.274132  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.286123  3537 rpcz_store.cc:276] 0709 14:52:09.971260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.978542 (+  7282us) service_pool.cc:224] Handling call
0709 14:52:09.999236 (+ 20694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:12.826900 (+2827664us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54005a3820. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:52:13.263754 (+436854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.273355 (+  9601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132293888}
I20260709 14:52:13.275097  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:13.277091  2608 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1020) took 1873 ms (client timeout 2039 ms). Trace:
W20260709 14:52:13.269316  2610 rpcz_store.cc:269] 0709 14:52:11.103378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.104171 (+   793us) service_pool.cc:224] Handling call
0709 14:52:13.155314 (+2051143us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400280354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.155625 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659354880}
I20260709 14:52:13.286391  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.150938  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:13.279827  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 879) took 1704 ms (client timeout 1716 ms). Trace:
W20260709 14:52:13.152619  5073 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.592s	user 0.025s	sys 0.000s
I20260709 14:52:13.290913  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 939) took 1364 ms. Trace:
I20260709 14:52:13.156581  4982 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:13.148623  5235 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:13.285146  5161 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:13.296482  2681 rpcz_store.cc:269] 0709 14:52:11.575092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.601963 (+ 26871us) service_pool.cc:224] Handling call
0709 14:52:13.279391 (+1677428us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54002b3254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.279594 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887123712}
W20260709 14:52:13.307235  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51205.1us
W20260709 14:52:13.292284  2608 rpcz_store.cc:269] 0709 14:52:11.403616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.289447 (+885831us) service_pool.cc:224] Handling call
0709 14:52:13.276577 (+987130us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.276826 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20554112}
I20260709 14:52:13.269373  5197 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:13.310336  2611 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 14:52:13.303445  5088 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 14:52:13.312007  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 197418us
I20260709 14:52:13.300822  2754 rpcz_store.cc:276] 0709 14:52:11.918407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.964273 (+ 45866us) service_pool.cc:224] Handling call
0709 14:52:13.098236 (+1133963us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:13.282578 (+184342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40500480}
W20260709 14:52:13.268154  2606 rpcz_store.cc:269] 0709 14:52:11.586797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.110024 (+523227us) service_pool.cc:224] Handling call
0709 14:52:13.153613 (+1043589us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.154018 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73288448}
I20260709 14:52:13.277908  5058 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.299845  5286 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:13.319842  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1025) took 1912 ms. Trace:
I20260709 14:52:13.320396  5198 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:13.320135  5058 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.321151  5198 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.321426  5166 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.317047  5073 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.322738  4914 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:13.324421  5073 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:13.325951  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86068.4us
I20260709 14:52:13.287410  2751 rpcz_store.cc:276] 0709 14:52:11.139565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.139802 (+   237us) service_pool.cc:224] Handling call
0709 14:52:11.169125 (+ 29323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:11.199255 (+ 30130us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:11.463216 (+263961us) log.cc:844] Serialized 14 byte log entry
0709 14:52:11.842229 (+379013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:11.842331 (+   102us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:11.842408 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.168183 (+1325775us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.168302 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.168679 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1478,"prepare.queue_time_us":398898,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":3209,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":7525,"spinlock_wait_cycles":68654464,"thread_start_us":630394,"threads_started":2,"wal-append.queue_time_us":577279,"wal-append.run_cpu_time_us":1689,"wal-append.run_wall_time_us":1055450}
I20260709 14:52:13.327565  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.329373  5161 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.330163  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.331369  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 876) took 1037 ms. Trace:
I20260709 14:52:13.331838  3536 rpcz_store.cc:276] 0709 14:52:12.293893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.278756 (+984863us) service_pool.cc:224] Handling call
0709 14:52:13.331231 (+ 52475us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.329512  4995 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.334254  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1031) took 5177 ms. Trace:
I20260709 14:52:13.334131  4995 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.334641  2755 rpcz_store.cc:276] 0709 14:52:08.157419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.252053 (+1094634us) service_pool.cc:224] Handling call
0709 14:52:09.300039 (+ 47986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.062444 (+3762405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.333825 (+271381us) spinlock_profiling.cc:243] Waited 3.76 s on lock 0x7b54000e6354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.334148 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8639543680}
W20260709 14:52:13.336432  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.338079  4982 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.340219  5159 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:13.342715  5181 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:13.343355  5181 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.331406  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.343812  5181 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:13.344461  2681 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:13.345808  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.347348  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:13.348810  5232 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.349726  5232 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:13.350001  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.351423  5232 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:13.352854  5232 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.317461  2604 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:13.354370  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 423) took 1829 ms. Trace:
I20260709 14:52:13.354862  2604 rpcz_store.cc:276] 0709 14:52:11.524899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.260067 (+1735168us) service_pool.cc:224] Handling call
0709 14:52:13.313536 (+ 53469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.353981 (+ 40445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.354281 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.356698  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:13.356858  5194 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:13.360110  2681 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:13.361289  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 882) took 1597 ms. Trace:
I20260709 14:52:13.358222  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.362691  3541 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:13.357996  5161 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:13.366436  2681 rpcz_store.cc:276] 0709 14:52:11.764153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.359171 (+1595018us) service_pool.cc:224] Handling call
0709 14:52:13.359741 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.360900 (+  1159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.361194 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.361328  2604 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:13.317873  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:13.320468  2611 rpcz_store.cc:276] 0709 14:52:11.405259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.209596 (+1804337us) service_pool.cc:224] Handling call
0709 14:52:13.303515 (+ 93919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:13.313172 (+  9657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.317174 (+  4002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.375633  2682 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:13.383977  4995 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.384748  5244 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.385195  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 962) took 1284 ms. Trace:
I20260709 14:52:13.386000  4636 raft_consensus.cc:1081] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
W20260709 14:52:13.386535  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89100.9us
I20260709 14:52:13.384006  2611 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:13.387825  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1039) took 1800 ms. Trace:
I20260709 14:52:13.388006  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:13.388211  2611 rpcz_store.cc:276] 0709 14:52:11.587688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.369705 (+1782017us) service_pool.cc:224] Handling call
0709 14:52:13.383635 (+ 13930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.387498 (+  3863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.387754 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.389556  4977 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:13.389981  2611 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 2. Current term is 3. Ops: []
I20260709 14:52:13.390777  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1027) took 1343 ms. Trace:
I20260709 14:52:13.392859  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.393980  5244 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:13.395159  5244 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:13.395951  5244 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:52:13.396340  5244 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:13.397414  5244 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.397861  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:13.398418  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.321630  5198 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:13.403563  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.391203  2611 rpcz_store.cc:276] 0709 14:52:12.047301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.389143 (+1341842us) service_pool.cc:224] Handling call
0709 14:52:13.389624 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.390446 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.390688 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.408314  5234 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:13.408936  5286 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.409179  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1030) took 1344 ms. Trace:
I20260709 14:52:13.409778  5286 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.408082  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.418320  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:13.420691  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93374.4us
I20260709 14:52:13.385702  2681 rpcz_store.cc:276] 0709 14:52:12.101365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.368010 (+1266645us) service_pool.cc:224] Handling call
0709 14:52:13.384088 (+ 16078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.384621 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.384652 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.384739 (+    87us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:13.384836 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.384918 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.385091 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.409904  2607 rpcz_store.cc:276] 0709 14:52:12.064781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.407598 (+1342817us) service_pool.cc:224] Handling call
0709 14:52:13.408178 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.408543 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.408581 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.408646 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:13.408687 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.408766 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.408938 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.402384  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.437007  3536 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:13.437544  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:13.438675  3536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:52:13.439296  5226 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:13.439538  5166 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 14:52:13.442998  4675 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.009s
I20260709 14:52:13.439674  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:13.451375  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56780.5us
I20260709 14:52:13.452021  5159 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:13.452363  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:13.452615  5159 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.453104  5159 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:13.453716  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:13.440531  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121823us
I20260709 14:52:13.455557  5224 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:13.456772  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.438676  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.431923  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.448391  5226 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:13.454698  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.381914  2603 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:13.459241  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 425) took 1931 ms. Trace:
I20260709 14:52:13.377839  5155 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.463220  2606 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:13.459794  2603 rpcz_store.cc:276] 0709 14:52:11.527653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.929738 (+1402085us) service_pool.cc:224] Handling call
0709 14:52:13.459151 (+529413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.469473  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.470463  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.471135  2677 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:13.471957  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 407) took 1306 ms. Trace:
I20260709 14:52:13.472294  2677 rpcz_store.cc:276] 0709 14:52:12.165603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.470062 (+1304459us) service_pool.cc:224] Handling call
0709 14:52:13.470641 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.471619 (+   978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.471863 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.472949  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:13.473821  5066 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.010s
I20260709 14:52:13.475327  5007 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:13.474952  4675 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.476506  2679 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 8. Ops: []
I20260709 14:52:13.477869  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 883) took 1359 ms. Trace:
I20260709 14:52:13.478047  5122 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:13.478327  2685 rpcz_store.cc:276] 0709 14:52:12.118364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.457973 (+1339609us) service_pool.cc:224] Handling call
0709 14:52:13.477059 (+ 19086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.477397 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.477434 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.477514 (+    80us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:13.477579 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.477632 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.477800 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.479399  5192 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:13.482409  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.483799  5051 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:13.480623  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 878) took 1918 ms. Trace:
I20260709 14:52:13.467742  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1026) took 1420 ms. Trace:
I20260709 14:52:13.489655  2606 rpcz_store.cc:276] 0709 14:52:12.047008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.380952 (+1333944us) service_pool.cc:224] Handling call
0709 14:52:13.447940 (+ 66988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.466799 (+ 18859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.467158 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.491166  5058 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.501205  4675 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.457470  5103 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:13.502406  5103 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.502915  5103 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:13.503093  5037 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.504854  5037 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:13.505584  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.476189  5007 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:13.508394  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 860) took 2183 ms. Trace:
I20260709 14:52:13.510522  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.511708  5222 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:13.512269  5222 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:13.513397  5222 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:13.515398  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76855.8us
I20260709 14:52:13.505473  5000 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:13.517367  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.519083  5166 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.486081  2679 rpcz_store.cc:276] 0709 14:52:11.572244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.807914 (+1235670us) service_pool.cc:224] Handling call
0709 14:52:12.824913 (+ 16999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.477005 (+652092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.479377 (+  2372us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.480546 (+  1169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497359616}
I20260709 14:52:13.463907  5226 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:13.524269  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.526248  5194 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:52:13.474519  5066 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.519851  2611 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:13.465950  5244 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:13.526633  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1039) took 2411 ms. Trace:
I20260709 14:52:13.484720  5051 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.546200  2757 rpcz_store.cc:276] 0709 14:52:11.114891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.115031 (+   140us) service_pool.cc:224] Handling call
0709 14:52:11.115434 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:12.300882 (+1185448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.526033 (+1225151us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000695d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.526416 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724292992}
I20260709 14:52:13.527089  4995 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:13.603343  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:13.604164  2755 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:13.605001  5046 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.601188  5164 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.605726  5046 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.606518  5164 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.588697  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.607734  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:13.608574  2752 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:13.608747  5234 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:13.589154  2610 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:13.589088  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 71850.2us
I20260709 14:52:13.589581  5235 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.595114  5166 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.595773  5165 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:13.557727  5222 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:13.555361  5224 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.600260  5066 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:13.601142  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:13.602213  2760 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:13.599633  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 970) took 1495 ms. Trace:
I20260709 14:52:13.630404  2683 rpcz_store.cc:276] 0709 14:52:12.104039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.458730 (+1354691us) service_pool.cc:224] Handling call
0709 14:52:13.568175 (+109445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.599223 (+ 31048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.599256 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.599301 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:13.599353 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.599416 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.599543 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:13.620672  5166 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:13.621340  5203 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.781s	user 0.014s	sys 0.009s
I20260709 14:52:13.621872  5140 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:13.650883  5140 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.509032  3545 rpcz_store.cc:276] 0709 14:52:11.325000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:11.325175 (+   175us) service_pool.cc:224] Handling call
0709 14:52:11.354430 (+ 29255us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:11.487924 (+133494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:11.487976 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:11.527798 (+ 39822us) log.cc:844] Serialized 52 byte log entry
0709 14:52:11.846817 (+319019us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:11.846894 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.507478 (+1660584us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400203354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.507555 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:13.507634 (+    79us) raft_consensus.cc:1685] finished
0709 14:52:13.507652 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.508083 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":808,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":17179,"spinlock_wait_cycles":305547520,"thread_start_us":278401,"threads_started":1,"wal-append.queue_time_us":395170,"wal-append.run_cpu_time_us":5587,"wal-append.run_wall_time_us":1439959}
I20260709 14:52:13.598567  5159 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.640038  5224 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.654013  5166 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.655665  5159 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:13.656716  5159 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:13.657047  5166 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.659040  5181 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:13.640790  4977 consensus_queue.cc:1175] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.3-6.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:52:13.640368  2754 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:13.663041  5203 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.664655  5075 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:13.665926  5075 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:13.666046  5203 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.666345  5155 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.667949  5198 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:13.673178  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116310us
I20260709 14:52:13.623194  5126 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:13.675009  2683 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:13.677083  2682 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:13.677361  5140 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.678222  5007 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:13.624845  5051 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:13.678755  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 750) took 3828 ms. Trace:
I20260709 14:52:13.679225  3547 rpcz_store.cc:276] 0709 14:52:09.850519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.853134 (+  2615us) service_pool.cc:224] Handling call
0709 14:52:09.853658 (+   524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:11.183950 (+1330292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.678247 (+2494297us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003200d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:13.678561 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3057640320}
I20260709 14:52:13.679723  5051 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:13.680518  5051 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.681321  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.655815  5224 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.682077  2607 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:13.682330  3540 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:13.682991  5007 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:52:13.683117  5218 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.683483  5007 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:13.683851  3540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 874) took 1498 ms (client timeout 1607 ms). Trace:
I20260709 14:52:13.683816  5218 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.685909  5051 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:13.687465  5164 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.689023  2679 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:13.691900  5046 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.692101  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.694324  4977 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:13.643775  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:13.694965  4977 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:13.642808  5226 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.695767  5140 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:13.695967  4977 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.697103  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 18) took 4014 ms. Trace:
I20260709 14:52:13.697602  2756 rpcz_store.cc:276] 0709 14:52:09.682512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:09.682707 (+   195us) service_pool.cc:224] Handling call
0709 14:52:13.377870 (+3695163us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b4c00140780. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16c912a20 00007fa16c923a29 00007fa16c8e9644 00007fa16c910f2d 00007fa16c910b93 00007fa16e640c92 00007fa16e64084a 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691
0709 14:52:13.696682 (+318812us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.696808 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4731121280}
I20260709 14:52:13.698693  5226 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.701584  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1050) took 1967 ms. Trace:
I20260709 14:52:13.702174  2758 rpcz_store.cc:276] 0709 14:52:11.700319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.009007 (+308688us) service_pool.cc:224] Handling call
0709 14:52:13.666399 (+1657392us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:13.666880 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.667228 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.667262 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:13.667334 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:13.667368 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.667437 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:13.667595 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36475648}
I20260709 14:52:13.703905  5286 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.639101  5069 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:13.707278  5069 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.710064  5069 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:13.711355  5235 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:13.717988  5297 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.129s	user 0.003s	sys 0.000s
W20260709 14:52:13.626057  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 115767us
I20260709 14:52:13.726078  5194 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:52:13.626652  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.729137  5194 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:13.635911  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.636613  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.640779  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:13.731293  2606 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:52:13.731638  2610 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:13.730707  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:13.722417  5218 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:13.639365  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.736151  2604 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:52:13.684624  5007 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.641477  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:13.738752  2606 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:13.739898  2604 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:13.746910  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.747856  2604 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:13.684463  3540 rpcz_store.cc:269] 0709 14:52:12.185047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.212531 (+ 27484us) service_pool.cc:224] Handling call
0709 14:52:13.683610 (+1471079us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002dc3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.683769 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3347199872}
I20260709 14:52:13.750672  2758 raft_consensus.cc:1217] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Deduplicated request from leader. Original: 4.3->[6.4-8.5]   Dedup: 6.4->[8.5-8.5]
I20260709 14:52:13.756028  2758 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 14:52:13.758368  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:13.758845  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 117498us
I20260709 14:52:13.759943  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.760434  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:13.761838  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.765700  2755 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:13.766943  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.771001  2760 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:13.738529  5007 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:13.670814  5198 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.780234  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.783982  2607 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:52:13.785684  2611 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:52:13.782081  5166 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:13.787871  5166 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:13.789409  5166 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:13.791774  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:13.792147  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.792563  2758 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:13.793738  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:13.793835  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:13.782143  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.798034  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:13.805125  2754 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:13.799978  2758 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:13.812482  2756 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:13.814777  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:13.815706  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.824681  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.779356  4914 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:13.779560  5069 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:13.827291  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.778950  5155 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:13.782272  5181 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:13.845842  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:13.837592  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:13.808809  5286 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:13.749076  5103 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:13.667152  5075 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:13.855865  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.824393  5046 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 14:52:13.879937  2686 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:13.880316  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60392.6us
W20260709 14:52:13.890324  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.854156  2752 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:13.897490  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 414) took 1681 ms. Trace:
W20260709 14:52:13.898638  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 203841us
I20260709 14:52:13.898125  2679 rpcz_store.cc:276] 0709 14:52:12.216080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.520535 (+1304455us) service_pool.cc:224] Handling call
0709 14:52:13.688696 (+168161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:13.689578 (+   882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:13.897122 (+207544us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400423950. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:13.897389 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385441536}
W20260709 14:52:13.900972  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 443668us
W20260709 14:52:13.912956  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 119194us
I20260709 14:52:13.928139  2678 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:13.895952  2682 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:13.844283  5155 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.834414  4914 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:13.919816  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.845067  5069 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:13.926601  2607 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 14:52:13.936913  5197 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:13.895119  4982 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:13.820797  5244 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
W20260709 14:52:14.000325  5122 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.000s
I20260709 14:52:14.002272  5238 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:14.002801  2679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7584380efde04ee289b42acaea2e4742"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:52:14.002976  5238 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:14.005270  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.006119  3545 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:14.007187  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.007910  5244 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:14.008741  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.009662  3545 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:14.010031  2677 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:14.011932  5140 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:14.012454  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 410) took 1846 ms. Trace:
I20260709 14:52:14.013006  2677 rpcz_store.cc:276] 0709 14:52:12.166504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.473108 (+1306604us) service_pool.cc:224] Handling call
0709 14:52:14.009775 (+536667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.010617 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:14.012188 (+  1571us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400269b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:14.012371 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233109888}
I20260709 14:52:14.014812  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.016523  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:13.951426  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 137938us
I20260709 14:52:14.017450  3543 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:14.017635  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:14.019073  5238 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:14.020735  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:13.961215  2611 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:13.825567  4989 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 14:52:14.022802  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:13.913108  5218 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:14.024588  2756 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:14.020846  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 964) took 1918 ms. Trace:
W20260709 14:52:14.026441  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.883906  5286 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:13.884440  5126 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 14:52:13.970739  5198 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:13.960892  5155 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:13.985637  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.986443  5181 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.991389  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:13.991667  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:13.992215  5074 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:14.067343  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:13.996043  5184 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.764s	user 0.012s	sys 0.003s
I20260709 14:52:13.974257  4636 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:13.996663  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:52:13.998538  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:14.073665  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75277us
I20260709 14:52:14.021226  5103 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:14.018424  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:14.010540  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:13.913910  5075 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:14.033138  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:14.034392  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:14.027992  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:14.040887  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:14.006121  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:14.025404  2681 rpcz_store.cc:276] 0709 14:52:12.102379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.446262 (+1343883us) service_pool.cc:224] Handling call
0709 14:52:13.556233 (+109971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.001775 (+445542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.001815 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:14.001894 (+    79us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:14.001948 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:14.020589 (+ 18641us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:14.020641 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:14.020780 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11092,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":1023244672}
I20260709 14:52:13.972505  4914 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:14.066215  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:14.100446  2607 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 7
W20260709 14:52:14.069209  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:13.972286  5069 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:13.992368  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:14.071727  5074 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:14.082305  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:14.082945  5181 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:14.083870  5194 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:14.084615  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:14.106360  2603 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:14.106936  2604 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:14.107220  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 72945.9us
I20260709 14:52:14.102097  2610 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:14.103022  3540 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:14.100395  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:14.047242  5286 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:14.064056  5238 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:14.122777  2682 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:14.065930  5198 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:14.124491  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:14.078135  5140 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:14.134660  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 127149us
I20260709 14:52:14.126386  3538 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:14.131456  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:14.123030  5122 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:14.073400  5218 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:14.133594  3547 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:14.141340  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:14.111506  5074 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:14.111538  5194 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:14.105545  4989 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:14.145295  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:14.129302  5180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.010s
I20260709 14:52:14.150295  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1038) took 1571 ms. Trace:
I20260709 14:52:14.126384  5126 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 14:52:14.099658  5232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.007s	sys 0.015s
W20260709 14:52:14.148138  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54179.8us
I20260709 14:52:14.174813  2603 rpcz_store.cc:276] 0709 14:52:12.568894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.467424 (+898530us) service_pool.cc:224] Handling call
0709 14:52:13.583413 (+115989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.106865 (+523452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:14.139882 (+ 33017us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:14.140100 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185788672}
W20260709 14:52:14.175259  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 209083us
W20260709 14:52:14.125686  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47119.8us
I20260709 14:52:14.177101  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:14.168386  2759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.012s	sys 0.016s
W20260709 14:52:14.203711  2759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.012s	sys 0.016s
W20260709 14:52:14.168737  2751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.005s	sys 0.023s
I20260709 14:52:14.105566  4636 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:14.132989  5088 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 14:52:14.191170  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:14.150250  5122 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:14.187496  5232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.008s	sys 0.016s
I20260709 14:52:14.101480  5184 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:14.173759  5037 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.668s	user 0.015s	sys 0.001s
I20260709 14:52:14.134639  5140 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:14.195724  2677 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 14:52:14.124068  2609 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.012s
W20260709 14:52:14.232942  2609 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.015s	sys 0.013s
I20260709 14:52:14.140578  5103 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 14:52:14.167793  5194 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:14.166560  5180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.008s	sys 0.011s
W20260709 14:52:14.213742  2757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.011s	sys 0.007s
I20260709 14:52:14.166560  5238 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:14.177740  2430 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:14.232198  2682 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 14:52:14.205029  2751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.023s
W20260709 14:52:14.233915  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.015s	sys 0.014s
W20260709 14:52:14.250692  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.016s	sys 0.014s
W20260709 14:52:14.235415  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.018s	sys 0.027s
I20260709 14:52:14.241854  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:14.252308  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.019s	sys 0.027s
W20260709 14:52:14.252857  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 767772us
I20260709 14:52:14.253051  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:14.255829  2679 raft_consensus.cc:606] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:14.256523  2679 raft_consensus.cc:614] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:14.257539  2685 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:14.206508  5164 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 14:52:14.264214  3545 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:14.208338  5192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.012s	sys 0.035s
I20260709 14:52:14.189703  5041 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 14:52:14.199764  5244 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 14:52:14.277556  3540 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:14.281829  5226 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:14.282349  5238 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:14.284792  4636 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:14.256973  5088 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:14.286114  5088 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:14.286546  5088 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:14.287968  5235 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:14.288496  5184 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:14.288697  5235 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:14.289441  5197 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:14.264981  5051 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 14:52:14.290235  5051 consensus_queue.cc:1243] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 14:52:14.290987  5302 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 14:52:14.291437  2435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 164) took 28110 ms (client timeout 15000 ms). Trace:
I20260709 14:52:14.292868  5035 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:14.293366  5122 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:14.295151  5310 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:14.295805  5310 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:14.293557  5035 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:14.312819  2677 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:14.314551  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:14.310981  2605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.003s	sys 0.020s
W20260709 14:52:14.316718  2605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.003s	sys 0.020s
I20260709 14:52:14.206728  5046 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 14:52:14.311741  2753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.007s	sys 0.020s
W20260709 14:52:14.320196  2753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.007s	sys 0.020s
W20260709 14:52:14.325613  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51374.3us
W20260709 14:52:14.270869  5205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.000s	sys 0.060s
W20260709 14:52:14.335383  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64222.5us
W20260709 14:52:14.275614  5192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.013s	sys 0.036s
I20260709 14:52:14.341035  3542 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:14.276322  5222 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:14.348521  5222 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:14.349007  5222 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:14.275871  4901 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:14.341243  5235 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:14.342295  5181 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:14.292939  5218 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:14.345676  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:14.336283  5205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.000s	sys 0.061s
W20260709 14:52:14.347935  5039 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.024s	sys 0.078s
W20260709 14:52:14.377602  5300 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 14:52:14.378711  2680 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:14.381009  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 877) took 2847 ms. Trace:
I20260709 14:52:14.381477  2680 rpcz_store.cc:276] 0709 14:52:11.533362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.762350 (+1228988us) service_pool.cc:224] Handling call
0709 14:52:12.762703 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.379471 (+1616768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:14.380583 (+  1112us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:14.380884 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3715224960}
I20260709 14:52:14.270820  5037 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:14.383265  5037 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:14.384519  5037 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:14.385991  5037 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:14.352522  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 110804us
I20260709 14:52:14.350809  4901 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:14.388077  4901 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:14.390388  5244 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:14.378000  5039 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.024s	sys 0.078s
I20260709 14:52:14.391134  5244 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:14.392416  5035 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:14.393070  5035 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:52:14.393739  5244 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:14.395346  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:14.396895  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:14.397529  2752 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:14.291963  2435 rpcz_store.cc:269] 0709 14:51:46.061400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:46.064939 (+  3539us) service_pool.cc:224] Handling call
0709 14:52:14.153351 (+28088412us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:52:03.515407 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:52:03.516216 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:52:03.516306 (+    90us) write_op.cc:435] Acquired schema lock
0709 14:52:03.516338 (+    32us) tablet.cc:662] Decoding operations
0709 14:52:03.689438 (+173100us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:52:03.689518 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 14:52:03.689548 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 14:52:03.696461 (+  6913us) tablet.cc:701] Row locks acquired
0709 14:52:03.696531 (+    70us) write_op.cc:260] PREPARE: finished
0709 14:52:03.696795 (+   264us) write_op.cc:270] Start()
0709 14:52:03.697030 (+   235us) write_op.cc:276] Timestamp: P: 1783608723696702 usec, L: 0
0709 14:52:03.697063 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 14:52:03.698086 (+  1023us) log.cc:844] Serialized 5193 byte log entry
0709 14:52:05.631808 (+1933722us) op_driver.cc:464] REPLICATION: finished
0709 14:52:06.201483 (+569675us) write_op.cc:301] APPLY: starting
0709 14:52:06.218122 (+ 16639us) tablet.cc:1367] starting BulkCheckPresence
0709 14:52:06.304396 (+ 86274us) tablet.cc:1370] finished BulkCheckPresence
0709 14:52:06.304579 (+   183us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:52:06.469607 (+165028us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:52:06.469993 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 14:52:06.470043 (+    50us) write_op.cc:312] APPLY: finished
0709 14:52:06.470564 (+   521us) log.cc:844] Serialized 94 byte log entry
0709 14:52:06.470819 (+   255us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:06.547423 (+ 76604us) write_op.cc:454] Released schema lock
0709 14:52:06.619779 (+ 72356us) write_op.cc:341] FINISH: Updating metrics
0709 14:52:07.007459 (+387680us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:07.016228 (+  8769us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48386,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":483630,"apply.run_cpu_time_us":13820,"apply.run_wall_time_us":561342,"num_ops":13,"prepare.queue_time_us":360061,"prepare.run_cpu_time_us":20009,"prepare.run_wall_time_us":337771,"raft.queue_time_us":884300,"raft.run_cpu_time_us":46056,"raft.run_wall_time_us":315948,"replication_time_us":1934487,"spinlock_wait_cycles":1920,"thread_start_us":1613870,"threads_started":4,"wal-append.queue_time_us":270873,"wal-append.run_cpu_time_us":27808,"wal-append.run_wall_time_us":3576101}]]}
I20260709 14:52:14.401072  5310 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:14.351231  4989 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 14:52:14.361526  3547 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:14.360137  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:14.377779  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:14.361577  2604 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:14.276647  5126 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 14:52:14.339095  4786 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:14.402604  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:14.410248  2755 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.004s	sys 0.013s
W20260709 14:52:14.420266  2755 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.004s	sys 0.013s
I20260709 14:52:14.423408  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:14.426192  2604 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:14.441478  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.436834  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:14.453392  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:14.451327  5161 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.010s	sys 0.017s
I20260709 14:52:14.398653  5155 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:14.424616  4786 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:14.501224  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:14.492998  2612 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.006s	sys 0.019s
W20260709 14:52:14.519908  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47605.7us
I20260709 14:52:14.524887  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:14.525894  2603 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:14.377794  5192 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 14:52:14.438354  5244 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 14:52:14.529631  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:14.528152  2612 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.006s	sys 0.020s
W20260709 14:52:14.488090  5267 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.014s	sys 0.013s
W20260709 14:52:14.487756  5161 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.010s	sys 0.018s
W20260709 14:52:14.505164  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.003s	sys 0.023s
I20260709 14:52:14.416051  5058 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 14:52:14.465963  5140 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:14.539650  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:14.482542  5205 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:14.547868  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:14.601958  2610 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 10
I20260709 14:52:14.525276  5310 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 14:52:14.351732  5198 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:14.546247  5088 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:14.554737  2607 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:14.572095  3545 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:14.572234  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:14.572811  5286 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:14.422124  5035 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 14:52:14.466785  5051 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:14.577106  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:14.585376  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.004s	sys 0.023s
I20260709 14:52:14.585745  2678 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:14.593184  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.016s	sys 0.005s
I20260709 14:52:14.575604  2430 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:52:14.559962  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89238.3us
I20260709 14:52:14.492719  5226 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.163s
W20260709 14:52:14.746049  2760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.003s	sys 0.030s
I20260709 14:52:14.755368  2752 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:52:14.955395  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.037s
W20260709 14:52:15.066660  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.001s	sys 0.043s
W20260709 14:52:15.067610  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.016s	sys 0.015s
W20260709 14:52:14.750613  3538 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.004s
W20260709 14:52:15.067885  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 738982us
I20260709 14:52:15.068845  3538 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:14.940279  2751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.023s	sys 0.013s
I20260709 14:52:14.922636  4977 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:15.070137  2751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.029s	sys 0.014s
W20260709 14:52:15.070964  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:15.071174  5203 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:15.072292  5140 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:15.007272  5159 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:15.073101  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:15.073750  5159 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.074678  5051 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:14.727097  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.004s	sys 0.036s
I20260709 14:52:15.078238  5051 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:15.069569  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 782) took 1123 ms. Trace:
I20260709 14:52:15.144877  3538 rpcz_store.cc:276] 0709 14:52:13.946350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.970441 (+ 24091us) service_pool.cc:224] Handling call
0709 14:52:15.069487 (+1099046us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:14.813871  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.000s	sys 0.019s
I20260709 14:52:15.021286  5235 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:15.080554  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:15.078305  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.014s	sys 0.053s
W20260709 14:52:15.065280  2760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.014s	sys 0.044s
I20260709 14:52:15.083820  2430 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 4 to 6, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:14.649909  5194 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:14.554682  5007 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.752s	user 0.007s	sys 0.003s
I20260709 14:52:15.108553  5197 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:15.087734  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:14.682237  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:14.679500  5041 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:14.681031  5205 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:14.728729  2650 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.003s	sys 0.015s
W20260709 14:52:14.585727  5267 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.016s	sys 0.014s
W20260709 14:52:14.586041  5166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.014s	sys 0.040s
W20260709 14:52:14.738592  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.014s	sys 0.023s
W20260709 14:52:15.194126  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.019s	sys 0.024s
W20260709 14:52:15.090752  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.008s	sys 0.009s
I20260709 14:52:14.614635  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:15.195169  3547 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:14.923695  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.005s	sys 0.009s
W20260709 14:52:14.879503  2757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.016s	sys 0.041s
I20260709 14:52:15.066324  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1048) took 1078 ms. Trace:
I20260709 14:52:14.503371  4786 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:15.110561  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:15.196130  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.009s	sys 0.009s
I20260709 14:52:15.197129  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1037) took 2653 ms. Trace:
I20260709 14:52:15.074692  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:15.197486  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:15.198151  2752 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:15.199365  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:15.199839  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:15.200350  2752 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:15.200798  3545 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:15.201758  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:14.771695  5307 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 14:52:15.195930  2757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.016s	sys 0.045s
W20260709 14:52:15.126466  2603 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.002s
I20260709 14:52:15.203655  2603 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:15.139249  5155 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:14.646090  5069 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.004s
I20260709 14:52:14.584331  5244 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:15.060551  5034 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 14:52:14.832387  5299 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 14:52:15.137209  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.003s	sys 0.016s
I20260709 14:52:15.134187  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:15.077222  5159 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:15.140714  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.000s	sys 0.036s
W20260709 14:52:15.141160  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.021s	sys 0.016s
I20260709 14:52:15.108217  5046 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:15.143610  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.005s	sys 0.010s
W20260709 14:52:15.226374  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.006s	sys 0.011s
W20260709 14:52:15.146281  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.005s	sys 0.020s
W20260709 14:52:15.139508  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.002s	sys 0.017s
I20260709 14:52:15.154596  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:15.124650  5180 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.020s	sys 0.033s
W20260709 14:52:15.169059  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.033s	sys 0.088s
W20260709 14:52:15.204116  5166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.023s	sys 0.071s
I20260709 14:52:15.167891  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1025) took 3102 ms. Trace:
W20260709 14:52:15.172159  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.028s	sys 0.039s
W20260709 14:52:15.258889  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.029s	sys 0.039s
W20260709 14:52:15.259001  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.003s	sys 0.019s
W20260709 14:52:15.174343  2650 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.008s	sys 0.020s
I20260709 14:52:15.259869  3540 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:15.260041  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.21251e+06us
W20260709 14:52:15.260777  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.037s	sys 0.088s
W20260709 14:52:15.261212  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.07006e+06us
W20260709 14:52:15.262156  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.05974e+06us
W20260709 14:52:15.149539  5238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.003s	sys 0.016s
W20260709 14:52:15.179574  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:15.191300  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:15.174917  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94524.2us
I20260709 14:52:15.136942  5035 consensus_queue.cc:1243] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 14:52:15.195803  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.005s	sys 0.012s
W20260709 14:52:14.969982  5314 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 14:52:15.204952  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:15.187346  5322 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 14:52:15.196136  2607 rpcz_store.cc:276] 0709 14:52:13.987632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.025519 (+ 37887us) service_pool.cc:224] Handling call
0709 14:52:15.066225 (+1040706us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:15.206202  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:15.276611  3538 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:15.206645  2752 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 14:52:15.209488  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.003s	sys 0.016s
W20260709 14:52:15.209583  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.008s	sys 0.014s
W20260709 14:52:15.277808  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.008s	sys 0.014s
I20260709 14:52:15.204700  2430 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 3 to 5, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:52:15.211529  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 413) took 3012 ms. Trace:
I20260709 14:52:15.197528  2609 rpcz_store.cc:276] 0709 14:52:12.543490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.467370 (+923880us) service_pool.cc:224] Handling call
0709 14:52:13.569045 (+101675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.595133 (+ 26088us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:14.237926 (+642793us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.196730 (+958804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:15.196790 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:15.196857 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.196933 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.197061 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":663449,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"thread_start_us":1566806,"threads_started":2}
W20260709 14:52:15.214022  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.021s	sys 0.016s
W20260709 14:52:15.213857  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.000s	sys 0.036s
I20260709 14:52:15.216899  2610 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:15.225564  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:15.203507  5041 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:15.194278  5039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.014s	sys 0.040s
W20260709 14:52:15.263674  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:15.187525  5170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.028s	sys 0.054s
W20260709 14:52:15.275087  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:15.272958  5238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.004s	sys 0.016s
I20260709 14:52:15.282018  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:15.314260  2609 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:15.206362  5309 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:15.239964  5122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.012s	sys 0.023s
W20260709 14:52:15.284227  2753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.020s	sys 0.017s
I20260709 14:52:15.169442  5235 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:15.167352  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.009s	sys 0.012s
W20260709 14:52:15.267899  5180 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.020s	sys 0.037s
I20260709 14:52:15.288587  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:15.171911  5197 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:15.213855  5069 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:15.208796  5034 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:15.255470  5007 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:15.280764  2685 rpcz_store.cc:276] 0709 14:52:12.199017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.479624 (+1280607us) service_pool.cc:224] Handling call
0709 14:52:13.480063 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.263048 (+782985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.211087 (+948039us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54002b2854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:15.211419 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787369344}
W20260709 14:52:15.272614  5058 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.016s	sys 0.018s
I20260709 14:52:15.307492  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:15.391811  3547 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:15.392518  2603 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:15.392791  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 894) took 1104 ms. Trace:
I20260709 14:52:15.299281  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:15.257315  2608 rpcz_store.cc:276] 0709 14:52:12.046592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.370577 (+1323985us) service_pool.cc:224] Handling call
0709 14:52:13.395341 (+ 24764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.395717 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:13.771086 (+375369us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.148762 (+1377676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:15.148866 (+   104us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:15.148972 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.149097 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.149349 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":775696,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":1221,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":34069,"thread_start_us":1648417,"threads_started":2}
I20260709 14:52:15.228406  5192 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:52:15.282634  4901 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.000s	sys 0.026s
I20260709 14:52:15.296370  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:15.342732  2753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.031s	sys 0.017s
I20260709 14:52:15.323648  2677 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:15.410045  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1050) took 1066 ms. Trace:
W20260709 14:52:15.373732  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.016s	sys 0.015s
I20260709 14:52:15.410912  2684 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:15.357648  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:15.297600  5039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.016s	sys 0.042s
I20260709 14:52:15.373518  2607 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:15.315994  4914 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.183s	user 0.015s	sys 0.001s
I20260709 14:52:15.393383  3547 rpcz_store.cc:276] 0709 14:52:14.288176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.497171 (+208995us) service_pool.cc:224] Handling call
0709 14:52:15.392697 (+895526us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:15.395248  2612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.050s	sys 0.062s
W20260709 14:52:15.424649  2612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.050s	sys 0.062s
I20260709 14:52:15.302977  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:15.388475  2430 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:52:15.328590  5170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.527s	user 0.030s	sys 0.057s
W20260709 14:52:15.373562  5122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.018s	sys 0.031s
I20260709 14:52:15.410569  2604 rpcz_store.cc:276] 0709 14:52:14.343992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.279995 (+936003us) service_pool.cc:224] Handling call
0709 14:52:15.409948 (+129953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:15.433485  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:15.439723  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:15.418334  5058 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.017s	sys 0.020s
I20260709 14:52:15.290278  5046 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:15.379168  5195 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:15.319653  5155 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:15.442380  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:15.442787  5324 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 14:52:15.443766  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 424) took 3898 ms. Trace:
I20260709 14:52:15.444006  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:15.444432  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:15.446164  2610 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:15.433861  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 954) took 1843 ms. Trace:
W20260709 14:52:15.409240  4901 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.000s	sys 0.026s
I20260709 14:52:15.385886  5333 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:15.406179  5192 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:15.408223  5051 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:15.398774  5007 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:15.351931  5309 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:15.486474  2607 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:15.499554  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1053) took 1154 ms. Trace:
I20260709 14:52:15.682410  2610 rpcz_store.cc:276] 0709 14:52:14.345038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.302494 (+957456us) service_pool.cc:224] Handling call
0709 14:52:15.499487 (+196993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:15.443531  4914 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:15.743320  2686 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:15.745221  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 425) took 1944 ms (client timeout 1523 ms). Trace:
W20260709 14:52:15.745837  2686 rpcz_store.cc:269] 0709 14:52:13.800609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.969707 (+169098us) service_pool.cc:224] Handling call
0709 14:52:15.744863 (+1775156us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:15.745096 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3790305536}
I20260709 14:52:15.462034  5058 consensus_queue.cc:1243] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 14:52:15.749230  5312 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:15.749831  5312 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:15.496299  5244 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:15.755139  5244 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:15.501451  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 867) took 3669 ms. Trace:
I20260709 14:52:15.690937  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 427) took 1872 ms. Trace:
I20260709 14:52:15.504276  5155 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.771884  5155 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:15.505878  5164 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:15.506044  5330 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 14:52:15.507447  5203 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.203s
I20260709 14:52:15.390307  5205 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:15.773242  5164 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:15.777037  5164 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:15.510690  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:15.780396  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:15.781744  2677 raft_consensus.cc:2368] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10.
I20260709 14:52:15.511281  5034 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.784114  5034 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:15.512925  4989 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.203s
I20260709 14:52:15.716594  5046 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:15.514232  5195 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:15.716965  5238 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:15.516883  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 889) took 2119 ms. Trace:
I20260709 14:52:15.517246  5333 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:15.518850  5051 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.520157  4786 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:15.524488  5069 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:15.722709  5075 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.616s	user 0.019s	sys 0.000s
I20260709 14:52:15.529860  5286 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:15.531770  5314 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 14:52:15.847314  2752 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:15.723308  5342 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:15.848176  5342 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:15.499734  2753 rpcz_store.cc:276] 0709 14:52:13.590941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.606980 (+ 16039us) service_pool.cc:224] Handling call
0709 14:52:13.615161 (+  8181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.639025 (+ 23864us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:14.320825 (+681800us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.433180 (+1112355us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.433385 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:15.433439 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:15.433539 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.433628 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.433787 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15008,"prepare.queue_time_us":805850,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":139,"raft.queue_time_us":179449,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"thread_start_us":1643938,"threads_started":2,"wal-append.queue_time_us":1209990}
I20260709 14:52:15.849162  2609 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:15.549582  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:15.587620  3540 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:15.855398  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1058) took 1508 ms (client timeout 2004 ms). Trace:
I20260709 14:52:15.855682  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 892) took 1568 ms. Trace:
W20260709 14:52:15.595932  5074 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.410s	user 0.010s	sys 0.005s
I20260709 14:52:15.725044  5310 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:15.524863  2430 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 4, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:15.596704  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:15.807781  5195 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:15.617885  5037 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.231s	user 0.014s	sys 0.004s
W20260709 14:52:15.891002  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.008s	sys 0.014s
I20260709 14:52:15.504770  5088 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:15.512537  5041 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:15.690330  5226 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:15.708120  5346 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 14:52:15.708287  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 421) took 1918 ms (client timeout 1982 ms). Trace:
W20260709 14:52:15.899119  2755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.004s	sys 0.018s
W20260709 14:52:15.899747  2755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.004s	sys 0.018s
W20260709 14:52:15.899381  2684 rpcz_store.cc:269] 0709 14:52:13.790216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.815299 (+ 25083us) service_pool.cc:224] Handling call
0709 14:52:15.707881 (+1892582us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:15.708179 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3664066304}
I20260709 14:52:15.513615  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:15.516151  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 959) took 1790 ms. Trace:
I20260709 14:52:15.516433  2756 raft_consensus.cc:1217] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-6.4]   Dedup: 6.4->[]
W20260709 14:52:15.516825  5298 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 14:52:15.458782  3538 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
W20260709 14:52:15.920822  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.003s	sys 0.010s
I20260709 14:52:15.920944  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 895) took 1632 ms. Trace:
W20260709 14:52:15.921416  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.003s	sys 0.010s
I20260709 14:52:15.921011  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:15.519129  5192 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:15.760293  2678 rpcz_store.cc:276] 0709 14:52:13.818409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.006617 (+188208us) service_pool.cc:224] Handling call
0709 14:52:14.026264 (+ 19647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:15.066236 (+1039972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.689460 (+623224us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540024ead4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:15.690786 (+  1326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286055808}
I20260709 14:52:15.762501  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1054) took 1417 ms. Trace:
I20260709 14:52:15.497305  2612 rpcz_store.cc:276] 0709 14:52:11.527309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.270664 (+1743355us) service_pool.cc:224] Handling call
0709 14:52:13.371346 (+100682us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:13.443740 (+ 72394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.443778 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:13.444483 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 14:52:13.752608 (+308125us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:14.584611 (+832003us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.425199 (+840588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.425324 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:15.425394 (+    70us) raft_consensus.cc:1685] finished
0709 14:52:15.425410 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.425615 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79171,"prepare.queue_time_us":574915,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":365122,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":49776,"thread_start_us":1879939,"threads_started":3,"wal-append.queue_time_us":898918}
I20260709 14:52:15.759229  3542 rpcz_store.cc:276] 0709 14:52:11.832123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:12.549218 (+717095us) service_pool.cc:224] Handling call
0709 14:52:12.552953 (+  3735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.347772 (+1794819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.500963 (+1153191us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:15.501333 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3882437888}
I20260709 14:52:15.808013  3537 rpcz_store.cc:276] 0709 14:52:13.397950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.399210 (+  1260us) service_pool.cc:224] Handling call
0709 14:52:13.399561 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:13.400058 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:13.754765 (+354707us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.103172 (+1348407us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:15.104091 (+   919us) log.cc:844] Serialized 26 byte log entry
0709 14:52:15.104488 (+   397us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:15.104549 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.104691 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:15.516625 (+411934us) raft_consensus.cc:1685] finished
0709 14:52:15.516653 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.516819 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35799,"prepare.queue_time_us":744253,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":434573,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":298,"thread_start_us":1604550,"threads_started":2,"wal-append.queue_time_us":1099677}
I20260709 14:52:15.831843  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:15.832733  3545 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.004s	sys 0.017s
I20260709 14:52:15.807595  5238 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.841413  4786 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:15.842937  5333 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:15.845600  5051 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:15.849311  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 952) took 2259 ms. Trace:
W20260709 14:52:15.544071  5347 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 14:52:15.503774  5194 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:15.964857  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:15.966274  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1047) took 1623 ms. Trace:
W20260709 14:52:15.856110  2609 rpcz_store.cc:269] 0709 14:52:14.346676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.281514 (+934838us) service_pool.cc:224] Handling call
0709 14:52:15.855278 (+573764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:15.724774  5197 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:15.856235  3540 rpcz_store.cc:276] 0709 14:52:14.287485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.109808 (+822323us) service_pool.cc:224] Handling call
0709 14:52:15.855603 (+745795us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:15.858317  2680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.009s	sys 0.010s
I20260709 14:52:15.730540  5235 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:15.728896  5222 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.379s	user 0.016s	sys 0.000s
W20260709 14:52:15.885210  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 335723us
W20260709 14:52:15.971788  2680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.009s	sys 0.011s
I20260709 14:52:15.652899  5007 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:15.947850  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.010s	sys 0.014s
I20260709 14:52:15.972474  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:15.973201  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.007s	sys 0.008s
W20260709 14:52:15.973534  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.007s	sys 0.008s
W20260709 14:52:15.974228  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 598430us
W20260709 14:52:15.884378  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.008s	sys 0.013s
W20260709 14:52:15.788362  5159 consensus_queue.cc:1175] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 6 status { last_received { term: 5 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:52:15.833117  4914 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:15.892263  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.008s	sys 0.014s
I20260709 14:52:15.901420  2754 rpcz_store.cc:276] 0709 14:52:13.725303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.924972 (+199669us) service_pool.cc:224] Handling call
0709 14:52:13.980287 (+ 55315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:14.261486 (+281199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.261538 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:14.292590 (+ 31052us) log.cc:844] Serialized 52 byte log entry
0709 14:52:15.259517 (+966927us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:15.259596 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.378137 (+118541us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:15.378226 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:15.515731 (+137505us) raft_consensus.cc:1685] finished
0709 14:52:15.515756 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.516068 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4970,"raft.queue_time_us":154,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8443,"spinlock_wait_cycles":618790144,"thread_start_us":966390,"threads_started":1,"wal-append.queue_time_us":1203056}
I20260709 14:52:15.914006  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 442) took 2175 ms. Trace:
W20260709 14:52:15.917128  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.042s	sys 0.100s
I20260709 14:52:15.917495  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1048) took 1574 ms. Trace:
W20260709 14:52:16.000566  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.042s	sys 0.100s
I20260709 14:52:16.000615  2608 rpcz_store.cc:276] 0709 14:52:14.343232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.447731 (+1104499us) service_pool.cc:224] Handling call
0709 14:52:15.917406 (+469675us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:15.918219  2603 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 14:52:16.001720  2603 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:15.920040  2756 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 14:52:15.922382  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.006s
W20260709 14:52:15.737624  4977 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.667s	user 0.015s	sys 0.000s
I20260709 14:52:15.921527  3538 rpcz_store.cc:276] 0709 14:52:14.288509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.174085 (+885576us) service_pool.cc:224] Handling call
0709 14:52:15.920871 (+746786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:15.884536  5310 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:15.926083  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.005s	sys 0.007s
I20260709 14:52:15.750739  5312 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:15.891045  5286 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:15.931087  2607 rpcz_store.cc:276] 0709 14:52:14.345342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.412209 (+1066867us) service_pool.cc:224] Handling call
0709 14:52:15.762364 (+350155us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:15.932809  3545 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.004s	sys 0.017s
I20260709 14:52:15.935103  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:15.936515  2682 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.006s	sys 0.007s
W20260709 14:52:16.026510  2682 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.006s	sys 0.007s
W20260709 14:52:15.938755  2759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.004s	sys 0.008s
I20260709 14:52:16.026623  2684 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:16.027149  2759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.004s	sys 0.009s
I20260709 14:52:15.935393  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:15.948418  2751 rpcz_store.cc:276] 0709 14:52:13.590238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.601708 (+ 11470us) service_pool.cc:224] Handling call
0709 14:52:13.602091 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:13.602550 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:13.602588 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:13.603378 (+   790us) log.cc:844] Serialized 52 byte log entry
0709 14:52:14.259676 (+656298us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:14.329218 (+ 69542us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.070887 (+741669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.072891 (+  2004us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:15.848984 (+776093us) raft_consensus.cc:1685] finished
0709 14:52:15.849007 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.849231 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":295356,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":14515,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":9377,"thread_start_us":1387825,"threads_started":2,"wal-append.queue_time_us":898299}
I20260709 14:52:15.952391  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1061) took 2228 ms. Trace:
I20260709 14:52:16.028080  2755 rpcz_store.cc:276] 0709 14:52:13.723608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.823222 (+ 99614us) service_pool.cc:224] Handling call
0709 14:52:13.823693 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.829758 (+  6065us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:14.429680 (+599922us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.900377 (+1470697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:15.923674 (+ 23297us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:15.951496 (+ 27822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.951680 (+   184us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.952114 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691689,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":252601,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":81280,"thread_start_us":1956297,"threads_started":3,"wal-append.queue_time_us":1107337}
W20260709 14:52:15.940623  2758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.000s	sys 0.011s
W20260709 14:52:16.031060  2758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.001s	sys 0.011s
W20260709 14:52:16.033337  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.014s	sys 0.009s
W20260709 14:52:16.033905  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.016s	sys 0.009s
W20260709 14:52:16.034271  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 772288us
W20260709 14:52:15.963807  3541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.002s	sys 0.012s
I20260709 14:52:16.035290  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:16.035657  3541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.003s	sys 0.013s
W20260709 14:52:15.967170  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:15.967794  2612 rpcz_store.cc:276] 0709 14:52:14.342892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.931391 (+1588499us) service_pool.cc:224] Handling call
0709 14:52:15.966158 (+ 34767us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:15.972951  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.011s	sys 0.014s
W20260709 14:52:16.040956  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:15.975603  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.008s	sys 0.014s
W20260709 14:52:15.895438  2757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.011s	sys 0.011s
I20260709 14:52:15.981659  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:15.891124  5074 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:16.050153  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77791us
W20260709 14:52:15.907157  2760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.010s	sys 0.001s
W20260709 14:52:16.051052  2760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.010s	sys 0.002s
I20260709 14:52:16.051344  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:15.983564  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.017s	sys 0.011s
W20260709 14:52:16.003293  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1066) took 1526 ms (client timeout 1805 ms). Trace:
I20260709 14:52:16.014482  3547 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:16.055577  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 893) took 1767 ms. Trace:
I20260709 14:52:16.056027  3547 rpcz_store.cc:276] 0709 14:52:14.287842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.964228 (+1676386us) service_pool.cc:224] Handling call
0709 14:52:16.014170 (+ 49942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.055086 (+ 40916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.055495 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:16.054538  2603 rpcz_store.cc:269] 0709 14:52:14.476932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.272122 (+795190us) service_pool.cc:224] Handling call
0709 14:52:16.003183 (+731061us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:16.017148  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.006s	sys 0.007s
I20260709 14:52:16.017719  3537 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:16.060245  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 790) took 1805 ms. Trace:
W20260709 14:52:16.060359  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.017s	sys 0.016s
W20260709 14:52:16.060808  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.017s	sys 0.016s
I20260709 14:52:16.023025  2606 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 14:52:16.060709  3537 rpcz_store.cc:276] 0709 14:52:14.254916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.938376 (+1683460us) service_pool.cc:224] Handling call
0709 14:52:15.981254 (+ 42878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.059903 (+ 78649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.060179 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.068557  2606 pending_rounds.cc:85] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 2
I20260709 14:52:16.032388  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1062) took 2308 ms. Trace:
I20260709 14:52:16.069015  2606 pending_rounds.cc:107] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 14:52:16.033079  5035 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 14:52:16.043430  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.038508  5342 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:16.033609  4982 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:16.037586  4786 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:16.050493  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 897) took 1761 ms. Trace:
I20260709 14:52:16.050496  2609 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:16.067039  5218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.014s	sys 0.019s
I20260709 14:52:15.938364  5226 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:16.061273  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.017s	sys 0.011s
I20260709 14:52:16.062105  2751 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 9
W20260709 14:52:16.064916  2757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.011s	sys 0.012s
W20260709 14:52:16.065528  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 931385us
I20260709 14:52:16.065549  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.069449  2606 raft_consensus.cc:2889] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:15.998615  2605 rpcz_store.cc:276] 0709 14:52:13.738503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.738643 (+   140us) service_pool.cc:224] Handling call
0709 14:52:13.739001 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.739298 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:14.319714 (+580416us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.909941 (+1590227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:15.913310 (+  3369us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:15.913583 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.913688 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:15.913886 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31110,"prepare.queue_time_us":584546,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":374836,"raft.run_wall_time_us":41023,"spinlock_wait_cycles":16112384,"thread_start_us":2104245,"threads_started":3,"wal-append.queue_time_us":1476064}
W20260709 14:52:16.067014  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:16.022734  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.006s	sys 0.025s
I20260709 14:52:16.071823  2603 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:16.060285  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:16.054703  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 955) took 2463 ms. Trace:
I20260709 14:52:16.075289  2612 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:15.988060  5222 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:16.081552  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46415.5us
I20260709 14:52:16.082248  2683 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:16.056535  5198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.033s	sys 0.124s
W20260709 14:52:16.080894  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.742s (SENT) [suppressed 24 similar messages]
I20260709 14:52:16.084627  2430 catalog_manager.cc:5697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 1 to 3, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:52:16.002566  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.006s	sys 0.006s
I20260709 14:52:16.089227  3540 rpcz_store.cc:276] 0709 14:52:14.289139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.970344 (+1681205us) service_pool.cc:224] Handling call
0709 14:52:15.970791 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.044530 (+ 73739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:16.049937 (+  5407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.050007 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:16.050071 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.050154 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.050416 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:16.097059  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.006s	sys 0.006s
I20260709 14:52:16.069108  2758 rpcz_store.cc:276] 0709 14:52:13.723945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.879904 (+155959us) service_pool.cc:224] Handling call
0709 14:52:13.952886 (+ 72982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:14.022440 (+ 69554us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:14.325430 (+302990us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.031685 (+1706255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.031781 (+    96us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:16.031867 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.031982 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.032285 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":532309,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":229472,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"thread_start_us":1959242,"threads_started":3,"wal-append.queue_time_us":1542648}
I20260709 14:52:16.101373  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1069) took 1602 ms. Trace:
I20260709 14:52:16.093973  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 988) took 1791 ms. Trace:
I20260709 14:52:16.056565  5166 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:16.289863  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:16.127898  3537 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 14:52:16.120328  5140 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.047s	user 0.010s	sys 0.007s
I20260709 14:52:16.290689  2677 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:16.123222  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1051) took 2134 ms. Trace:
I20260709 14:52:16.120960  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.102330  2608 pending_rounds.cc:85] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 2
W20260709 14:52:16.073807  5170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.012s	sys 0.033s
I20260709 14:52:16.289026  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1046) took 1965 ms. Trace:
I20260709 14:52:16.311506  2612 rpcz_store.cc:276] 0709 14:52:14.323846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.039882 (+1716036us) service_pool.cc:224] Handling call
0709 14:52:16.074996 (+ 35114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:16.288648 (+213652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.288960 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.289958  2609 rpcz_store.cc:276] 0709 14:52:14.499097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.981230 (+1482133us) service_pool.cc:224] Handling call
0709 14:52:16.101271 (+120041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.300976  2754 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:16.292268  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:16.313388  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.011s	sys 0.029s
I20260709 14:52:16.296093  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:16.296211  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.007s	sys 0.007s
I20260709 14:52:16.314958  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1060) took 1966 ms. Trace:
I20260709 14:52:16.295043  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 958) took 2702 ms. Trace:
I20260709 14:52:16.297529  2608 pending_rounds.cc:107] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 14:52:16.291405  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.316306  2608 raft_consensus.cc:2889] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:16.315034  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.01581e+06us
I20260709 14:52:16.319082  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:16.289330  2759 rpcz_store.cc:276] 0709 14:52:13.591304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.604248 (+ 12944us) service_pool.cc:224] Handling call
0709 14:52:13.604613 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.604918 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:14.234393 (+629475us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.027464 (+1793071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.027542 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:16.053151 (+ 25609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.053656 (+   505us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.054595 (+   939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24521,"prepare.queue_time_us":635632,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3810,"raft.queue_time_us":229525,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"thread_start_us":2275327,"threads_started":3,"wal-append.queue_time_us":1435585}
W20260709 14:52:16.297885  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.07328e+06us
W20260709 14:52:16.294210  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:16.300710  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.035s	sys 0.028s
W20260709 14:52:16.244720  5350 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 14:52:16.293045  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 793) took 1164 ms. Trace:
I20260709 14:52:16.291859  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:16.321003  5351 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.393s	user 0.002s	sys 0.000s
I20260709 14:52:16.320464  2755 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:16.193251  5327 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
W20260709 14:52:16.371927  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52961.5us
I20260709 14:52:16.373916  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:16.309489  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2706379310000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 14:52:16.124294  5345 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 14:52:16.086601  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3208301169999004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 14:52:16.219350  5347 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 14:52:16.295133  2611 rpcz_store.cc:276] 0709 14:52:13.988883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.023958 (+ 35075us) service_pool.cc:224] Handling call
0709 14:52:14.024385 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:14.047264 (+ 22879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.047303 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:14.066625 (+ 19322us) log.cc:844] Serialized 52 byte log entry
0709 14:52:15.263058 (+1196433us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:15.298005 (+ 34947us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.319713 (+ 21708us) log.cc:844] Serialized 14 byte log entry
0709 14:52:15.345196 (+ 25483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.345297 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.122896 (+777599us) raft_consensus.cc:1685] finished
0709 14:52:16.122920 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.123113 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":197203,"prepare.queue_time_us":86610,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":8619,"raft.queue_time_us":570072,"raft.run_cpu_time_us":679,"raft.run_wall_time_us":184628,"spinlock_wait_cycles":13720960,"thread_start_us":1123475,"threads_started":1,"wal-append.queue_time_us":1906833}
I20260709 14:52:16.392565  2759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
W20260709 14:52:16.125254  5326 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 14:52:16.300668  5198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.038s	sys 0.129s
W20260709 14:52:16.323942  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.035s	sys 0.028s
W20260709 14:52:16.288432  5218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.019s	sys 0.025s
I20260709 14:52:16.289499  2680 rpcz_store.cc:276] 0709 14:52:14.303124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.464843 (+161719us) service_pool.cc:224] Handling call
0709 14:52:14.505380 (+ 40537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:14.530629 (+ 25249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.530682 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:14.709778 (+179096us) log.cc:844] Serialized 26 byte log entry
0709 14:52:15.993472 (+1283694us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:15.993553 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.993676 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.093712 (+100036us) raft_consensus.cc:1685] finished
0709 14:52:16.093732 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.093897 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49739,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":837322,"threads_started":1,"wal-append.queue_time_us":991715}
I20260709 14:52:16.087491  5334 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:16.333102  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:16.315449  2603 rpcz_store.cc:276] 0709 14:52:14.348828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.057418 (+1708590us) service_pool.cc:224] Handling call
0709 14:52:16.057833 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.314566 (+256733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.314880 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.358254  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:16.360802  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 300593us
I20260709 14:52:16.325102  3537 rpcz_store.cc:276] 0709 14:52:15.128711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.089910 (+961199us) service_pool.cc:224] Handling call
0709 14:52:16.127397 (+ 37487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.292720 (+165323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.292984 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.324851  2678 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:16.357573  2753 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:16.367285  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:16.368829  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 794) took 1239 ms. Trace:
W20260709 14:52:16.374511  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2652379310000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 14:52:16.390180  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3127301169998873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 14:52:16.321518  5170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.014s	sys 0.040s
I20260709 14:52:16.320960  5360 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.393399  2759 raft_consensus.cc:606] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:16.437300  2759 raft_consensus.cc:614] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:16.366453  5380 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:16.407519  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 984) took 2387 ms (client timeout 1726 ms). Trace:
W20260709 14:52:16.445542  2683 rpcz_store.cc:269] 0709 14:52:14.020310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.039287 (+ 18977us) service_pool.cc:224] Handling call
0709 14:52:16.407186 (+2367899us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54003435d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.407416 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4622335360}
I20260709 14:52:16.407946  2685 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:16.291162  5166 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:16.449811  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:16.450748  2684 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:16.408726  4675 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.452689  4675 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:16.409327  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 435) took 2016 ms. Trace:
I20260709 14:52:16.455060  2756 rpcz_store.cc:276] 0709 14:52:14.392576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.392752 (+   176us) service_pool.cc:224] Handling call
0709 14:52:15.516266 (+1123514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:16.002497 (+486231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.409068 (+406571us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540050e1d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.409255 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582770432}
I20260709 14:52:16.419071  5007 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:16.392973  2754 raft_consensus.cc:1217] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[4.4-5.5]   Dedup: 4.4->[5.5-5.5]
W20260709 14:52:16.457661  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58113.8us
I20260709 14:52:16.458321  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 784) took 2480 ms. Trace:
I20260709 14:52:16.457921  5007 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:52:16.412520  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 82788.7us
I20260709 14:52:16.459617  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:16.460216  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:16.413919  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 443) took 1956 ms. Trace:
I20260709 14:52:16.460965  2752 rpcz_store.cc:276] 0709 14:52:14.457668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.300250 (+842582us) service_pool.cc:224] Handling call
0709 14:52:16.413707 (+1113457us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.413842 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001283968}
I20260709 14:52:16.416925  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 428) took 1204 ms. Trace:
I20260709 14:52:16.395843  4914 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:16.420514  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53364.5us
W20260709 14:52:16.421156  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1221379309999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 14:52:16.398790  5205 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:16.413919  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 450) took 1218 ms. Trace:
I20260709 14:52:16.478979  2758 rpcz_store.cc:276] 0709 14:52:15.195248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.311001 (+1115753us) service_pool.cc:224] Handling call
0709 14:52:16.413844 (+102843us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:16.403051  3409 connection.cc:256] RPC call timeout handler was delayed by -1.1026037079999469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 14:52:16.326664  5383 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.433969  2677 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:16.367049  4786 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.412485  5309 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:16.436157  5140 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:16.448395  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 19) took 1307 ms. Trace:
I20260709 14:52:16.315716  2757 rpcz_store.cc:276] 0709 14:52:13.592270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.636949 (+ 44679us) service_pool.cc:224] Handling call
0709 14:52:13.637360 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.637727 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:14.241797 (+604070us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.294373 (+2052576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.294446 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:16.294536 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.294770 (+   234us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.294967 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":713559,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":272139,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":161,"thread_start_us":2604591,"threads_started":3,"wal-append.queue_time_us":1861443}
I20260709 14:52:16.457952  2754 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [4.4-5.5]
I20260709 14:52:16.370352  5359 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:16.435798  5339 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.379s	user 0.001s	sys 0.001s
W20260709 14:52:16.421581  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:16.529254  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2623301169999195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 14:52:16.529690  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2434301169998889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 14:52:16.530086  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2389301169998816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 14:52:16.530427  2726 connection.cc:256] RPC call timeout handler was delayed by -1.149830116999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 14:52:16.473322  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1005379309999626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 14:52:16.531033  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0762379309999233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
I20260709 14:52:16.448539  5360 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:16.414989  5218 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.467885  5380 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:16.532048  5218 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:16.534312  5360 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:16.534997  2756 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:16.535715  2726 connection.cc:256] RPC call timeout handler was delayed by -1.0094301169999653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
I20260709 14:52:16.462076  2681 rpcz_store.cc:276] 0709 14:52:15.212975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.213155 (+   180us) service_pool.cc:224] Handling call
0709 14:52:15.276567 (+ 63412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:15.523675 (+247108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:15.523734 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:15.524590 (+   856us) log.cc:844] Serialized 52 byte log entry
0709 14:52:16.024674 (+500084us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:16.024739 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.416452 (+391713us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:16.416507 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.416562 (+    55us) raft_consensus.cc:1685] finished
0709 14:52:16.416576 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.416859 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8535,"raft.queue_time_us":1962,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":33735,"spinlock_wait_cycles":566761216,"thread_start_us":499541,"threads_started":1,"wal-append.queue_time_us":872313}
I20260709 14:52:16.404691  5226 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:16.411046  5197 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.414000  5037 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.423265  5376 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 14:52:16.413365  5385 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.128432  5310 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:16.496241  2683 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 14:52:16.497737  2755 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 2. Current term is 3. Ops: []
I20260709 14:52:16.411571  5334 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:16.512945  5205 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:16.414438  5088 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:16.518369  2752 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:16.518993  5372 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:16.519840  2758 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:16.520519  5371 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:16.521579  4675 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:16.522039  5286 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:16.512512  2759 rpcz_store.cc:276] 0709 14:52:15.131405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.337770 (+1206365us) service_pool.cc:224] Handling call
0709 14:52:16.437999 (+100229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.514175  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 436) took 2121 ms. Trace:
I20260709 14:52:16.518142  4786 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:16.511128  5367 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:16.527148  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1067) took 1151 ms. Trace:
I20260709 14:52:16.528018  4989 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:16.531404  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0393379309999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
I20260709 14:52:16.458859  3546 rpcz_store.cc:276] 0709 14:52:13.977921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.002922 (+ 25001us) service_pool.cc:224] Handling call
0709 14:52:14.003393 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:14.003665 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.003690 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:14.004330 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 14:52:15.069234 (+1064904us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:15.069303 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:15.069549 (+   246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.458022 (+1388473us) raft_consensus.cc:1685] finished
0709 14:52:16.458048 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.458247 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1064003,"threads_started":1,"wal-append.queue_time_us":1887888}
I20260709 14:52:16.542570  2684 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:16.491436  5198 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:16.411217  5041 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.550938  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1047) took 2634 ms. Trace:
I20260709 14:52:16.487795  5238 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:16.505684  5007 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.534503  5218 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:16.582203  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 439) took 2188 ms. Trace:
I20260709 14:52:16.584182  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 895) took 1088 ms. Trace:
I20260709 14:52:16.420046  3540 rpcz_store.cc:276] 0709 14:52:15.129036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.310434 (+1181398us) service_pool.cc:224] Handling call
0709 14:52:16.368186 (+ 57752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.368459 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:16.368488 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.368532 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:16.368575 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.368633 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.368765 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:16.661854  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:16.900310  3536 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 14:52:17.251945  2751 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.922s	user 0.022s	sys 0.004s
I20260709 14:52:17.252146  3536 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:17.252517  2751 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:16.827179  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 325263us
W20260709 14:52:17.253245  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1064) took 1906 ms (client timeout 1676 ms). Trace:
I20260709 14:52:17.017436  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:16.611160  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 987) took 2312 ms. Trace:
I20260709 14:52:16.967968  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:16.998858  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:16.596050  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 963) took 2783 ms. Trace:
I20260709 14:52:17.258330  2609 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 14:52:16.745342  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:16.675549  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:16.675001  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:17.232069  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.009s
I20260709 14:52:16.635375  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:16.885699  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 776) took 3449 ms. Trace:
I20260709 14:52:17.258376  2605 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:16.893499  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:16.593453  3543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 650) took 2555 ms (client timeout 2248 ms). Trace:
I20260709 14:52:16.900094  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:17.204797  2678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.020s
I20260709 14:52:17.255898  2611 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:17.250989  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0321379309999656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 14:52:16.588285  2757 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:17.252801  2756 rpcz_store.cc:276] 0709 14:52:14.393963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.456614 (+2062651us) service_pool.cc:224] Handling call
0709 14:52:16.457200 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.547618 (+ 90418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.581788 (+ 34170us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:16.582089 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178266368}
I20260709 14:52:17.253662  2606 rpcz_store.cc:276] 0709 14:52:13.916856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.956046 (+ 39190us) service_pool.cc:224] Handling call
0709 14:52:13.980953 (+ 24907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.316070 (+2335117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.550538 (+234468us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b540016d5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:16.550814 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4682232064}
W20260709 14:52:16.805181  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 724244us
I20260709 14:52:17.294155  2753 rpcz_store.cc:276] 0709 14:52:15.376061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.899795 (+523734us) service_pool.cc:224] Handling call
0709 14:52:16.526902 (+627107us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.527062 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37875072}
I20260709 14:52:17.295854  2607 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:16.676270  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.291195  3547 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:17.291795  2603 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:17.295162  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.005s	sys 0.010s
I20260709 14:52:17.253664  2685 rpcz_store.cc:276] 0709 14:52:15.496023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.517475 (+ 21452us) service_pool.cc:224] Handling call
0709 14:52:16.407622 (+890147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.452328 (+ 44706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.583846 (+131518us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54003435d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.584078 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046118272}
W20260709 14:52:17.249213  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 659962us
W20260709 14:52:17.274171  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:17.299645  2680 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:17.254662  2754 rpcz_store.cc:276] 0709 14:52:14.392956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.334546 (+1941590us) service_pool.cc:224] Handling call
0709 14:52:16.392754 (+ 58208us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:16.513710 (+120956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.514087 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:17.293551  2678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.002s	sys 0.020s
W20260709 14:52:17.253876  2751 rpcz_store.cc:269] 0709 14:52:15.346730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.028152 (+681422us) service_pool.cc:224] Handling call
0709 14:52:17.253127 (+1224975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.307154  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 441) took 2855 ms. Trace:
I20260709 14:52:17.307221  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 437) took 2863 ms. Trace:
W20260709 14:52:17.291419  3543 rpcz_store.cc:269] 0709 14:52:14.003495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.018141 (+ 14646us) service_pool.cc:224] Handling call
0709 14:52:16.558750 (+2540609us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400003a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:16.558960 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5338425088}
I20260709 14:52:17.307687  2752 rpcz_store.cc:276] 0709 14:52:14.394608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.477559 (+2082951us) service_pool.cc:224] Handling call
0709 14:52:16.517962 (+ 40403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.249589 (+731627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.249843 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.308224  2755 rpcz_store.cc:276] 0709 14:52:14.393284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.443425 (+2050141us) service_pool.cc:224] Handling call
0709 14:52:16.497469 (+ 54044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.256406 (+758937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.256673 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.258661  2760 rpcz_store.cc:276] 0709 14:52:13.812894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.015710 (+202816us) service_pool.cc:224] Handling call
0709 14:52:14.016206 (+   496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:15.168290 (+1152084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:15.168335 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:15.188306 (+ 19971us) log.cc:844] Serialized 52 byte log entry
0709 14:52:16.051552 (+863246us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:16.051694 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.559735 (+508041us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000e6d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:16.559820 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.559898 (+    78us) raft_consensus.cc:1685] finished
0709 14:52:16.559919 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.595933 (+ 36014us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":274,"raft.queue_time_us":44203,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":10284,"spinlock_wait_cycles":283607296,"thread_start_us":1785158,"threads_started":2,"wal-append.queue_time_us":912130}
I20260709 14:52:17.309069  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:17.309669  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50029.9us
I20260709 14:52:17.310194  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:17.310376  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44750.6us
I20260709 14:52:17.290280  2686 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:17.311446  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 444) took 2743 ms. Trace:
I20260709 14:52:17.311480  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:17.313086  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 447) took 2447 ms. Trace:
I20260709 14:52:17.311985  2758 rpcz_store.cc:276] 0709 14:52:14.531582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.480008 (+1948426us) service_pool.cc:224] Handling call
0709 14:52:16.519482 (+ 39474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:17.274621 (+755139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.275056 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.313558  2757 rpcz_store.cc:276] 0709 14:52:14.865871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.536771 (+1670900us) service_pool.cc:224] Handling call
0709 14:52:16.587878 (+ 51107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.296127 (+708249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.313030 (+ 16903us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:16.850791  5034 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.066s	user 0.006s	sys 0.009s
I20260709 14:52:17.332458  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.543380  5380 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:17.048225  5051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.008s	sys 0.040s
W20260709 14:52:17.333971  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:16.889567  5195 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.972s	user 0.011s	sys 0.000s
I20260709 14:52:16.556288  4982 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.339738  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:17.230137  5192 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.266s	user 0.017s	sys 0.001s
W20260709 14:52:16.825686  5390 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 14:52:16.581264  5226 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:17.361313  2751 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:17.359545  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.394483  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:16.812870  5384 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
W20260709 14:52:17.365814  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54400.7us
W20260709 14:52:17.368333  2576 connection.cc:256] RPC call timeout handler was delayed by -1.242028858000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 14:52:17.381626  5051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.010s	sys 0.047s
I20260709 14:52:17.404389  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 898) took 3107 ms. Trace:
I20260709 14:52:17.370465  5310 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:16.718070  5058 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:17.255918  2682 rpcz_store.cc:276] 0709 14:52:14.298555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:14.298724 (+   169us) service_pool.cc:224] Handling call
0709 14:52:14.336070 (+ 37346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:14.336348 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:14.336379 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:14.369321 (+ 32942us) log.cc:844] Serialized 26 byte log entry
0709 14:52:15.277442 (+908121us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:16.039725 (+762283us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.567099 (+527374us) spinlock_profiling.cc:243] Waited 7.56 ms on lock 0x7b54003bbda0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c1733 00007fa16e646dbf 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c
0709 14:52:16.567295 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.567503 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:16.567588 (+    85us) raft_consensus.cc:1685] finished
0709 14:52:16.610710 (+ 43122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.611052 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":803556,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":18466,"raft.queue_time_us":125,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":10634,"spinlock_wait_cycles":17380224,"thread_start_us":1516478,"threads_started":2,"wal-append.queue_time_us":930969}
I20260709 14:52:16.957705  5267 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.579432  5332 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:17.311013  5088 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.307042  5385 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:16.589468  5362 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.899312  5378 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.291221  3541 rpcz_store.cc:276] 0709 14:52:13.435920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.436062 (+   142us) service_pool.cc:224] Handling call
0709 14:52:13.436505 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:13.436965 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:13.968661 (+531696us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.435359 (+2466698us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.858274 (+422915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:16.858384 (+   110us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:16.885209 (+ 26825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.885375 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:16.885581 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25295,"prepare.queue_time_us":1250849,"prepare.run_cpu_time_us":160,"prepare.run_wall_time_us":4053,"raft.queue_time_us":615882,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":31153,"spinlock_wait_cycles":88448,"thread_start_us":3206100,"threads_started":5,"wal-append.queue_time_us":1223618}
W20260709 14:52:17.433398  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117695us
I20260709 14:52:17.259459  5203 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.377797  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:17.443527  3543 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:17.369552  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.733973  5222 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:17.369249  5379 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:17.379052  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:17.445302  2606 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 10
I20260709 14:52:17.381255  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:16.690441  5069 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:16.675343  5194 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.415647  5058 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:17.455796  2756 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:17.377784  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:17.456712  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1063) took 2110 ms. Trace:
I20260709 14:52:17.457098  2754 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:17.457186  2756 rpcz_store.cc:276] 0709 14:52:15.346217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.455131 (+2108914us) service_pool.cc:224] Handling call
0709 14:52:17.455580 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.456367 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.456638 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.383741  2575 leader_election.cc:400] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10.
I20260709 14:52:17.460201  2575 leader_election.cc:403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:52:17.418108  5377 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:17.386248  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:16.579741  4914 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:16.555447  5074 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.393924  2604 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:17.259904  5356 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.414366  5226 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.395064  5195 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:16.579380  5197 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:17.400355  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2228761940000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 14:52:17.360981  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61523.7us
I20260709 14:52:17.400872  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:17.339849  5161 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:16.555851  5383 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:17.410887  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 112542us
I20260709 14:52:17.411652  2758 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:17.446785  5166 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:16.587327  5037 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.405098  3538 rpcz_store.cc:276] 0709 14:52:14.289484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.020541 (+1731057us) service_pool.cc:224] Handling call
0709 14:52:16.021108 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:16.021496 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:16.325764 (+304268us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.903529 (+577765us) log.cc:844] Serialized 14 byte log entry
0709 14:52:17.396564 (+493035us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:17.396626 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:17.396716 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.396811 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.396931 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495931,"prepare.queue_time_us":496214,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":13616,"raft.queue_time_us":710076,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":148,"thread_start_us":793721,"threads_started":3,"wal-append.queue_time_us":262320}
I20260709 14:52:17.409605  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 891) took 3109 ms. Trace:
I20260709 14:52:17.369835  5159 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:17.306429  5041 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:17.421700  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 786417us
I20260709 14:52:17.426592  5267 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:16.898578  5392 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 14:52:17.306849  4989 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.513790  4989 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:17.514171  5197 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:17.424391  4786 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:17.372628  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:17.379133  5375 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:17.432354  2751 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 14:52:17.525780  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 451) took 2233 ms. Trace:
I20260709 14:52:17.526214  2751 rpcz_store.cc:276] 0709 14:52:15.292080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.414676 (+2122596us) service_pool.cc:224] Handling call
0709 14:52:17.430335 (+ 15659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.525370 (+ 95035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.525694 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.432715  5398 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:17.433609  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 788) took 3272 ms. Trace:
I20260709 14:52:17.440972  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:17.436180  3536 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.447682  2752 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 14:52:17.426299  5244 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.670s	user 0.020s	sys 0.000s
I20260709 14:52:16.970713  5309 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.397874  4982 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.454419  5069 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.516783  3546 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:17.383769  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:17.454525  5203 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.533655  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 429) took 2268 ms. Trace:
I20260709 14:52:17.310675  5334 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:17.430209  5218 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:17.467643  5356 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:17.551819  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:17.552615  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:17.553354  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:17.469473  2606 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
W20260709 14:52:17.469659  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 162585us
I20260709 14:52:17.472222  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.471680  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.555244  3543 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:17.556159  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 658) took 1138 ms. Trace:
I20260709 14:52:17.459496  5379 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:17.506122  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 124572us
I20260709 14:52:17.471458  4977 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10.
I20260709 14:52:17.508967  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:17.519914  2755 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:17.527900  3545 rpcz_store.cc:276] 0709 14:52:14.167335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.201624 (+1034289us) service_pool.cc:224] Handling call
0709 14:52:15.202051 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:16.038667 (+836616us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:16.038715 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:16.055948 (+ 17233us) log.cc:844] Serialized 52 byte log entry
0709 14:52:16.365181 (+309233us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:16.365254 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.365401 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:17.433304 (+1067903us) raft_consensus.cc:1685] finished
0709 14:52:17.433328 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.433526 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":310010,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":158,"thread_start_us":1050974,"threads_started":2,"wal-append.queue_time_us":417898}
I20260709 14:52:17.529578  2754 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:17.530176  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.464030  5194 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.395915  5192 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.537705  5309 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.536396  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:17.473493  4914 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.556588  3543 rpcz_store.cc:276] 0709 14:52:16.417360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.448923 (+1031563us) service_pool.cc:224] Handling call
0709 14:52:17.556069 (+107146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.557639  5218 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:17.600467  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:17.606316  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 797) took 1078 ms. Trace:
I20260709 14:52:17.314582  5361 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:17.534230  2679 rpcz_store.cc:276] 0709 14:52:15.264902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.265051 (+   149us) service_pool.cc:224] Handling call
0709 14:52:15.299322 (+ 34271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:15.363442 (+ 64120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:15.363499 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:15.424954 (+ 61455us) log.cc:844] Serialized 26 byte log entry
0709 14:52:16.396986 (+972032us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:16.397035 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:16.397128 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:17.520922 (+1123794us) raft_consensus.cc:1685] finished
0709 14:52:17.520954 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.533566 (+ 12612us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24518,"raft.queue_time_us":63112,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":29465,"thread_start_us":866321,"threads_started":1,"wal-append.queue_time_us":970512}
I20260709 14:52:17.555152  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:17.612049  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56994.8us
I20260709 14:52:17.528875  5074 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 14:52:17.505070  3542 rpcz_store.cc:276] 0709 14:52:14.287144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.938486 (+1651342us) service_pool.cc:224] Handling call
0709 14:52:15.986215 (+ 47729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:15.986576 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:16.225614 (+239038us) log.cc:844] Serialized 14 byte log entry
0709 14:52:16.861024 (+635410us) log.cc:844] Serialized 14 byte log entry
0709 14:52:17.395850 (+534826us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18003d5340. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:17.396045 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:17.396102 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:17.396167 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.396255 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.396379 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46346,"prepare.queue_time_us":543877,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":112,"raft.queue_time_us":710638,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":26188,"spinlock_wait_cycles":30755200,"thread_start_us":834741,"threads_started":3,"wal-append.queue_time_us":443207}
W20260709 14:52:17.561957  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67487.6us
I20260709 14:52:17.568599  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.529016  5335 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.575923  5398 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:17.627331  2610 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:17.459103  5205 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.491297  5161 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.525040  5375 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:17.577644  2756 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:52:17.580276  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 969) took 1002 ms. Trace:
I20260709 14:52:17.635455  2753 rpcz_store.cc:276] 0709 14:52:16.578264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.559785 (+981521us) service_pool.cc:224] Handling call
0709 14:52:17.579538 (+ 19753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.579867 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:17.579909 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:17.579960 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:17.580002 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.580063 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.580202 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.456707  5058 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:17.580987  3538 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:17.640918  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 656) took 1224 ms. Trace:
I20260709 14:52:17.473191  5377 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:17.581529  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 72643us
I20260709 14:52:17.616983  4977 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:17.580614  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:17.530860  5359 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 14:52:17.517336  5159 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.606699  3545 rpcz_store.cc:276] 0709 14:52:16.527876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.599986 (+1072110us) service_pool.cc:224] Handling call
0709 14:52:17.606218 (+  6232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.307493  5007 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.649430  2681 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:52:17.453011  5378 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.617920  2751 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:17.620345  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.653941  2606 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 3: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:17.538380  5197 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 14:52:17.540266  4989 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 14:52:17.630546  2611 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:17.449185  5362 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:17.619179  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:17.609663  5218 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:17.675539  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56497.3us
I20260709 14:52:17.637755  5375 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:17.638726  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:17.646044  4977 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:17.641346  3538 rpcz_store.cc:276] 0709 14:52:16.416582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.504955 (+1088373us) service_pool.cc:224] Handling call
0709 14:52:17.640841 (+135886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.625342  5404 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:17.618435  5361 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.689399  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 905) took 1224 ms. Trace:
I20260709 14:52:17.692996  3543 rpcz_store.cc:276] 0709 14:52:16.464681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.621976 (+1157295us) service_pool.cc:224] Handling call
0709 14:52:17.689319 (+ 67343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.653010  5403 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:17.622812  5342 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.660446  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.636885  5335 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:17.662668  4914 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:17.663995  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.664894  5379 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.666095  4982 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:17.667627  5309 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:17.685002  2605 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:52:17.647514  5365 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:17.639858  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:17.686885  5035 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.693913  4977 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:17.700304  5041 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.700168  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.701237  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:17.646093  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.687500  5034 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:17.724051  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.691301  5226 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:17.688400  5399 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:17.741878  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:17.694520  5404 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:17.761060  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 907) took 1292 ms. Trace:
I20260709 14:52:17.726063  5403 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.761754  5379 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:17.701895  5406 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.241s	user 0.000s	sys 0.002s
I20260709 14:52:17.761873  5069 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:17.779419  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:17.726796  5342 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:17.762753  5195 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.764595  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:17.815058  2605 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:17.689623  5356 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.775588  3542 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:17.777357  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 654) took 1361 ms. Trace:
W20260709 14:52:17.785984  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 249671us
I20260709 14:52:17.825398  3543 rpcz_store.cc:276] 0709 14:52:16.415796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.693616 (+1277820us) service_pool.cc:224] Handling call
0709 14:52:17.777238 (+ 83622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.826009  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:17.641470  5398 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.762470  2603 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:17.828099  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:17.817371  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 415289us
I20260709 14:52:17.701059  5088 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.808148  3545 rpcz_store.cc:276] 0709 14:52:16.468552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.647621 (+1179069us) service_pool.cc:224] Handling call
0709 14:52:17.760950 (+113329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.809687  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 996) took 1406 ms. Trace:
I20260709 14:52:17.766571  5037 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.834795  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:17.818156  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.008s	sys 0.014s
W20260709 14:52:17.836700  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.008s	sys 0.014s
I20260709 14:52:17.818943  5423 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:17.825119  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 244617us
I20260709 14:52:17.851754  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:17.833092  2677 rpcz_store.cc:276] 0709 14:52:16.403461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.492604 (+ 89143us) service_pool.cc:224] Handling call
0709 14:52:16.493176 (+   572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:16.539518 (+ 46342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:16.539583 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:16.551501 (+ 11918us) log.cc:844] Serialized 52 byte log entry
0709 14:52:17.297597 (+746096us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:17.776634 (+479037us) log.cc:844] Serialized 14 byte log entry
0709 14:52:17.808710 (+ 32076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.808807 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:17.808855 (+    48us) raft_consensus.cc:1685] finished
0709 14:52:17.808868 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:17.809577 (+   709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1606,"prepare.queue_time_us":233228,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":10054,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":4385,"thread_start_us":1035926,"threads_started":2,"wal-append.queue_time_us":772047}
I20260709 14:52:17.830669  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 913) took 1294 ms. Trace:
I20260709 14:52:17.840235  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:17.841806  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 201999us
I20260709 14:52:17.853653  3542 rpcz_store.cc:276] 0709 14:52:16.535857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.625929 (+1090072us) service_pool.cc:224] Handling call
0709 14:52:17.830555 (+204626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.847445  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.866999  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:17.876964  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:17.880738  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.807171  5198 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:17.807742  4982 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:17.884419  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 239770us
I20260709 14:52:17.807502  5404 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:17.886776  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.814033  5383 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:17.894762  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.898222  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:17.899050  3545 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:17.891644  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.825335  5194 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:17.899605  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:17.900478  2605 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:17.773669  5035 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:17.903369  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.903302  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 800) took 1344 ms. Trace:
I20260709 14:52:17.858135  5365 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:17.910807  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:17.913893  2684 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:17.905259  3545 rpcz_store.cc:276] 0709 14:52:16.558616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.832690 (+1274074us) service_pool.cc:224] Handling call
0709 14:52:17.903207 (+ 70517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:17.906962  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.884110  5414 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:17.908689  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.920127  3540 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:17.909294  5198 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.882103  5037 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:17.897217  5411 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:17.907546  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:17.923280  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.909920  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:17.927295  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.930176  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:17.886490  5405 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:17.859128  5385 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:17.900271  5407 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 14:52:17.936908  3543 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:17.823971  5379 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:17.810695  5399 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:17.965730  3538 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:17.968016  5398 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:17.968245  5362 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:17.970345  5312 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:17.944830  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55878us
W20260709 14:52:17.923183  5419 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 14:52:17.946323  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:17.932607  5037 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:17.991987  3545 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:17.947620  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:17.950593  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:17.996073  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.997736  2682 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:17.913812  5383 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:17.936892  5192 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:17.955694  5409 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 14:52:17.956802  5035 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:17.892705  5166 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 14:52:18.008966  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.959280  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.010249  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:17.963980  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.928269  5195 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 14:52:17.926440  5365 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:18.019214  5365 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:17.975210  2606 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:17.975987  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.021935  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1065) took 1025 ms. Trace:
I20260709 14:52:18.022238  3540 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 14:52:18.022459  2606 rpcz_store.cc:276] 0709 14:52:16.996618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.940187 (+943569us) service_pool.cc:224] Handling call
0709 14:52:17.974636 (+ 34449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.021592 (+ 46956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.021862 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.023515  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.025108  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:17.982273  2684 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:18.025828  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 966) took 2760 ms. Trace:
I20260709 14:52:17.925233  5034 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:17.937932  5421 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 14:52:18.010166  5383 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.027994  5034 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:17.924751  5414 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:18.030982  5383 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.031287  5414 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:18.032881  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:17.986332  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56266.9us
I20260709 14:52:17.954141  2605 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:18.036182  2605 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:18.038000  2605 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 14:52:18.000538  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:17.951061  5405 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:17.946323  5194 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:18.013178  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:18.046257  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 906) took 1581 ms. Trace:
I20260709 14:52:18.046787  3543 rpcz_store.cc:276] 0709 14:52:16.465052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.987226 (+1522174us) service_pool.cc:224] Handling call
0709 14:52:18.046156 (+ 58930us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:18.015328  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67789.3us
I20260709 14:52:18.047466  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:18.048112  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:18.049305  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:18.016428  4675 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 14:52:18.039872  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.005596  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 799) took 1477 ms. Trace:
I20260709 14:52:18.060642  3545 rpcz_store.cc:276] 0709 14:52:16.528639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.922192 (+1393553us) service_pool.cc:224] Handling call
0709 14:52:18.005504 (+ 83312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.041116  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:18.026746  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:17.991230  5411 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:17.987773  5399 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:18.077000  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 449) took 2945 ms. Trace:
W20260709 14:52:18.050868  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.025797  3540 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:18.040762  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 432) took 2530 ms. Trace:
I20260709 14:52:18.077714  2757 rpcz_store.cc:276] 0709 14:52:15.131841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.347340 (+2215499us) service_pool.cc:224] Handling call
0709 14:52:17.395892 (+ 48552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.571242 (+175350us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:17.571299 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:17.571376 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:17.571442 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.076588 (+505146us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:18.076668 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.076903 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3047,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":22390,"spinlock_wait_cycles":401782016}
I20260709 14:52:18.006323  5398 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:18.026405  2760 rpcz_store.cc:276] 0709 14:52:15.265915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.379948 (+2114033us) service_pool.cc:224] Handling call
0709 14:52:17.380420 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:17.381050 (+   630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:17.381081 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:17.393443 (+ 12362us) log.cc:844] Serialized 52 byte log entry
0709 14:52:17.852209 (+458766us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:17.852293 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.852402 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:18.025552 (+173150us) raft_consensus.cc:1685] finished
0709 14:52:18.025579 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.025756 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52235,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"thread_start_us":397321,"threads_started":1,"wal-append.queue_time_us":404140}
W20260709 14:52:18.068445  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46889.8us
W20260709 14:52:18.091766  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.007031  5267 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:18.050787  5371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.014s	sys 0.011s
I20260709 14:52:18.099472  5356 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:18.100306  5356 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:18.056672  5412 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:18.086601  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.094527  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.064498  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:18.096585  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:18.117340  2755 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:18.067106  5383 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:18.079304  2678 rpcz_store.cc:276] 0709 14:52:15.528127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.418175 (+890048us) service_pool.cc:224] Handling call
0709 14:52:16.418568 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:16.478186 (+ 59618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:16.478223 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:16.535580 (+ 57357us) log.cc:844] Serialized 52 byte log entry
0709 14:52:17.301672 (+766092us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:17.468062 (+166390us) log.cc:844] Serialized 14 byte log entry
0709 14:52:17.468352 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.468447 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:18.040297 (+571850us) raft_consensus.cc:1685] finished
0709 14:52:18.040320 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.040683 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59948,"prepare.queue_time_us":254588,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":34797,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":20736,"thread_start_us":698197,"threads_started":2,"wal-append.queue_time_us":747684}
I20260709 14:52:18.061414  5222 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:18.076326  5194 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:18.106153  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 435) took 1548 ms (client timeout 1780 ms). Trace:
I20260709 14:52:18.107832  2607 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:18.109977  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.088742  5398 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:18.117362  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53036.2us
I20260709 14:52:18.096588  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.077446  5197 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:18.065842  5417 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 14:52:18.121548  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80523.5us
I20260709 14:52:18.144176  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:18.081635  5421 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:52:18.121660  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:18.083513  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83319.7us
I20260709 14:52:18.122991  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.117666  2682 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:18.107215  5397 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.676s	user 0.004s	sys 0.004s
W20260709 14:52:18.100369  5371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.015s	sys 0.011s
I20260709 14:52:18.160657  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.162609  2607 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 14:52:18.129283  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.157357  2757 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:18.072767  5161 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.137465  3540 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:18.100809  5356 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:18.084424  4675 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:18.183274  4675 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:18.117065  5367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.019s	sys 0.037s
W20260709 14:52:18.186862  5367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.019s	sys 0.038s
I20260709 14:52:18.164664  5371 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:18.187335  5367 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:18.188200  5371 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:18.188408  5367 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:18.140266  5383 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:18.190660  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 997) took 1152 ms. Trace:
I20260709 14:52:18.103225  5399 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:18.191475  5411 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.191931  5399 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.142671  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:18.202983  5425 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:18.203938  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61392.7us
I20260709 14:52:18.203804  5425 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.143558  5398 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:18.129755  5415 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:18.212031  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 795) took 3082 ms. Trace:
W20260709 14:52:18.135497  2681 rpcz_store.cc:269] 0709 14:52:16.558108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.589808 (+ 31700us) service_pool.cc:224] Handling call
0709 14:52:18.102648 (+1512840us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:18.106030 (+  3382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810797440}
I20260709 14:52:18.212747  3537 rpcz_store.cc:276] 0709 14:52:15.129358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.415610 (+1286252us) service_pool.cc:224] Handling call
0709 14:52:16.416026 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.461317 (+1045291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:17.461352 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:17.461408 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:17.461450 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.211744 (+750294us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400200154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:18.211802 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.211960 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403231104}
I20260709 14:52:18.173804  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:18.136013  5385 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:18.177301  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:18.224603  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.180042  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.161968  5222 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:18.139722  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.226980  3547 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:18.227274  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.165866  5421 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:18.194012  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.237947  2760 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:18.152650  5361 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:18.169370  5417 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:18.241142  5417 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.195139  5194 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.191193  2685 rpcz_store.cc:276] 0709 14:52:17.038116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.344290 (+306174us) service_pool.cc:224] Handling call
0709 14:52:17.378717 (+ 34427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:17.776041 (+397324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:17.776083 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:17.811349 (+ 35266us) log.cc:844] Serialized 52 byte log entry
0709 14:52:17.946763 (+135414us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:17.946791 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.946896 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:18.190362 (+243466us) raft_consensus.cc:1685] finished
0709 14:52:18.190386 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.190587 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15235,"raft.queue_time_us":49892,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":418939,"threads_started":2,"wal-append.queue_time_us":224400}
I20260709 14:52:18.245441  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.130517  5037 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.200520  2607 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:18.247534  5429 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:18.248421  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.205745  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:18.248682  5310 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.004s
I20260709 14:52:18.167637  5405 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.249858  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.250880  2678 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 14:52:18.251999  5431 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 14:52:18.250478  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:18.166437  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:18.207191  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:18.213392  5218 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.003s
I20260709 14:52:18.207302  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:18.138736  5412 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:18.145421  5192 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:18.211999  3545 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10.
I20260709 14:52:18.211761  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:18.254707  5378 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.214905  2757 raft_consensus.cc:2376] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9. Re-sending same reply.
I20260709 14:52:18.255398  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.229210  5385 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:18.229856  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.277065  3547 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:18.231801  5222 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.278079  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:18.278358  5385 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:18.212563  5415 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:18.263128  5429 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:18.234189  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.280539  5415 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:18.282235  5375 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.000s
I20260709 14:52:18.269336  5412 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:18.282680  5375 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:18.270555  5192 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:18.260862  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 94583.3us
W20260709 14:52:18.269177  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62024.9us
I20260709 14:52:18.260385  5161 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:18.287633  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.292622  2755 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:18.293679  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:18.271914  3543 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:52:18.281909  2677 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:18.298301  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:18.298609  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43512.9us
W20260709 14:52:18.299244  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.279865  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.302528  2609 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 14:52:18.305648  2685 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:18.305609  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.307168  2678 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:18.312510  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.313350  2678 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:18.316957  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:18.318019  2677 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:18.316957  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.319613  3547 raft_consensus.cc:2368] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7.
I20260709 14:52:18.321867  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:18.322471  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.322540  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.323235  2755 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:18.323194  3537 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6. Re-sending same reply.
I20260709 14:52:18.271195  5421 raft_consensus.cc:2764] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:18.317057  5312 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:18.327632  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.329252  2685 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:18.288465  5379 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.308804  5375 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:18.341311  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:18.312611  5378 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:18.346369  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.267594  5218 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.375583  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.327198  5312 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:18.332561  5192 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:18.388976  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.274639  5404 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.311811  5414 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.356002  5159 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.756258  2751 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:18.718509  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.749599  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 450) took 1440 ms. Trace:
I20260709 14:52:18.418183  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:18.761906  2755 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:18.539959  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:18.755046  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 456746us
I20260709 14:52:18.458566  3541 raft_consensus.cc:2368] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:18.358532  5378 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:18.763868  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.658326  5426 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:18.768436  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:18.776470  2677 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:18.768419  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.760054  2605 rpcz_store.cc:276] 0709 14:52:17.309854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.042738 (+732884us) service_pool.cc:224] Handling call
0709 14:52:18.043283 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:18.060997 (+ 17714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.061032 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:18.096085 (+ 35053us) log.cc:844] Serialized 26 byte log entry
0709 14:52:18.211509 (+115424us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:18.211556 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.211636 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:18.440176 (+228540us) raft_consensus.cc:1685] finished
0709 14:52:18.440205 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.749463 (+309258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17072,"raft.queue_time_us":38327,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":18284,"thread_start_us":67242,"threads_started":1,"wal-append.queue_time_us":235561}
W20260709 14:52:18.438181  4977 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.000s
I20260709 14:52:18.780841  3547 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:18.686015  5192 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:18.765012  5069 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:18.787874  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 657) took 2371 ms. Trace:
I20260709 14:52:18.796485  3547 rpcz_store.cc:276] 0709 14:52:16.416964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.767079 (+2350115us) service_pool.cc:224] Handling call
0709 14:52:18.780521 (+ 13442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.787396 (+  6875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.787780 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.755851  5425 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:18.416301  5041 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:18.784216  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:18.778649  5356 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.596s	user 0.014s	sys 0.000s
I20260709 14:52:18.772481  5378 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:18.808687  2755 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:18.781908  5411 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:18.805238  2605 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:18.559763  5365 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:18.804735  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:18.809888  5041 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:18.810009  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1069) took 1302 ms. Trace:
I20260709 14:52:18.817202  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:18.817796  3547 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:18.827251  2605 rpcz_store.cc:276] 0709 14:52:17.509703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.779556 (+1269853us) service_pool.cc:224] Handling call
0709 14:52:18.793243 (+ 13687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.809717 (+ 16474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.809947 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.828299  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 660) took 2410 ms. Trace:
I20260709 14:52:18.828768  2605 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:18.829423  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1070) took 1313 ms. Trace:
I20260709 14:52:18.828943  3547 rpcz_store.cc:276] 0709 14:52:16.418083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.802526 (+2384443us) service_pool.cc:224] Handling call
0709 14:52:18.817406 (+ 14880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:18.827783 (+ 10377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.828206 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.829990  2605 rpcz_store.cc:276] 0709 14:52:17.516685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.828149 (+1311464us) service_pool.cc:224] Handling call
0709 14:52:18.828455 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.829189 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.829368 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.439589  5312 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.782734  5426 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.785717  4977 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:18.811154  5218 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:18.826131  5367 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:18.803354  5192 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:18.846101  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.846566  2679 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:18.861130  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.850128  3547 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:52:18.809589  5356 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.867810  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 662) took 2449 ms. Trace:
I20260709 14:52:18.868759  3547 rpcz_store.cc:276] 0709 14:52:16.418863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.830525 (+2411662us) service_pool.cc:224] Handling call
0709 14:52:18.849629 (+ 19104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.867334 (+ 17705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.867690 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.846072  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1070) took 1284 ms. Trace:
I20260709 14:52:18.872310  2605 rpcz_store.cc:276] 0709 14:52:17.561865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.830623 (+1268758us) service_pool.cc:224] Handling call
0709 14:52:18.844968 (+ 14345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.845516 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.845568 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:18.845638 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:18.845699 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.845784 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:18.845982 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":7819}
I20260709 14:52:18.814254  5069 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:18.904871  2678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.006s	sys 0.005s
I20260709 14:52:18.825908  5222 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 14:52:18.905398  2678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.006s	sys 0.005s
W20260709 14:52:18.848651  5334 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.282s	user 0.018s	sys 0.000s
I20260709 14:52:18.912623  3547 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:18.926731  5422 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:18.927398  5422 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:18.928601  5422 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:18.930049  5422 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:18.931226  4977 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:18.931463  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 448) took 3921 ms. Trace:
I20260709 14:52:18.931864  2758 rpcz_store.cc:276] 0709 14:52:15.010217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.346498 (+2336281us) service_pool.cc:224] Handling call
0709 14:52:17.396984 (+ 50486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.488055 (+ 91071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.931224 (+1443169us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b5400060250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:18.931390 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11084032}
I20260709 14:52:18.933436  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.934646  2758 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:18.935725  5404 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:18.936406  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1067) took 4459 ms (client timeout 3180 ms). Trace:
W20260709 14:52:18.946275  2610 rpcz_store.cc:269] 0709 14:52:14.477407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.764500 (+1287093us) service_pool.cc:224] Handling call
0709 14:52:18.936144 (+3171644us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540042dbd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:18.936292 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3896815872}
W20260709 14:52:18.939134  5415 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.001s
I20260709 14:52:18.911145  5222 consensus_queue.cc:1243] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 14:52:18.942711  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:18.858932  5432 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:18.943848  3543 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.001s
I20260709 14:52:18.964934  3543 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 3.
I20260709 14:52:18.965821  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 670) took 1054 ms. Trace:
I20260709 14:52:18.943375  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 663) took 2524 ms. Trace:
I20260709 14:52:18.967056  3547 rpcz_store.cc:276] 0709 14:52:16.419256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.874917 (+2455661us) service_pool.cc:224] Handling call
0709 14:52:18.912334 (+ 37417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.919951 (+  7617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.943294 (+ 23343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.936458  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.953843  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 983) took 1052 ms. Trace:
I20260709 14:52:18.968495  2758 rpcz_store.cc:276] 0709 14:52:17.901442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.932924 (+1031482us) service_pool.cc:224] Handling call
0709 14:52:18.953739 (+ 20815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.968914  2755 raft_consensus.cc:2410] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:18.966255  3543 rpcz_store.cc:276] 0709 14:52:17.911527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.168587 (+257060us) service_pool.cc:224] Handling call
0709 14:52:18.965723 (+797136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.972146  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1079) took 1047 ms. Trace:
I20260709 14:52:18.990355  2755 rpcz_store.cc:276] 0709 14:52:17.924441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.910869 (+986428us) service_pool.cc:224] Handling call
0709 14:52:18.972070 (+ 61201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:18.976356  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.978984  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:18.988647  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:18.994258  3543 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 3.
I20260709 14:52:18.970688  5432 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:18.943570  5334 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:19.106957  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 289870us
W20260709 14:52:18.972875  5434 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 14:52:19.112648  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1006) took 1066 ms. Trace:
W20260709 14:52:18.972293  5398 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.002s
I20260709 14:52:19.113214  2678 rpcz_store.cc:276] 0709 14:52:18.046639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.325664 (+279025us) service_pool.cc:224] Handling call
0709 14:52:18.339561 (+ 13897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.339904 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:18.909334 (+569430us) log.cc:844] Serialized 14 byte log entry
0709 14:52:19.112193 (+202859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:19.112273 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:19.112342 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.112426 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.112574 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68875,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":3748,"raft.queue_time_us":131,"thread_start_us":740075,"threads_started":2}
W20260709 14:52:19.097506  5436 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.001s	sys 0.024s
I20260709 14:52:18.971719  5411 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:19.118299  2726 leader_election.cc:336] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.228s (SENT)
I20260709 14:52:18.953513  5415 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.119647  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:19.119580  5436 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.001s	sys 0.025s
I20260709 14:52:19.124401  3542 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 4: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:19.126871  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:19.116417  5398 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.128072  2755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:19.134655  2755 raft_consensus.cc:606] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:19.129536  5441 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 14:52:19.130889  3543 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:52:19.099921  5432 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:19.137481  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63661.9us
I20260709 14:52:19.141418  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.139009  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 20) took 1479 ms. Trace:
I20260709 14:52:19.146240  2755 rpcz_store.cc:276] 0709 14:52:17.656621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.116640 (+1460019us) service_pool.cc:224] Handling call
0709 14:52:19.135428 (+ 18788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.148289  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.149981  5426 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:19.147200  5411 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:19.154140  5334 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:19.154412  5356 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:19.136987  5222 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:19.157961  5415 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:19.159313  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1055) took 4813 ms (client timeout 2228 ms). Trace:
I20260709 14:52:19.159659  2606 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 14:52:19.159893  2604 rpcz_store.cc:269] 0709 14:52:14.345808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:15.439217 (+1093409us) service_pool.cc:224] Handling call
0709 14:52:19.159039 (+3719822us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.159228 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4490158336}
I20260709 14:52:19.161322  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.162267  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 661) took 2743 ms. Trace:
I20260709 14:52:19.162762  3537 rpcz_store.cc:276] 0709 14:52:16.418443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.810099 (+2391656us) service_pool.cc:224] Handling call
0709 14:52:18.837669 (+ 27570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:18.838285 (+   616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.838332 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:18.839165 (+   833us) log.cc:844] Serialized 26 byte log entry
0709 14:52:18.943698 (+104533us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:19.106308 (+162610us) log.cc:844] Serialized 14 byte log entry
0709 14:52:19.106640 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.106723 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:19.161989 (+ 55266us) raft_consensus.cc:1685] finished
0709 14:52:19.162018 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.162190 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188949,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":179,"raft.queue_time_us":27853,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":9142,"thread_start_us":241621,"threads_started":2,"wal-append.queue_time_us":268145}
I20260709 14:52:19.165261  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.166052  3537 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:19.159471  2755 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:19.167394  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1074) took 2141 ms. Trace:
I20260709 14:52:19.167760  2755 rpcz_store.cc:276] 0709 14:52:17.026194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.148730 (+2122536us) service_pool.cc:224] Handling call
0709 14:52:19.159254 (+ 10524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.167106 (+  7852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.167333 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.162279  2604 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:52:19.170192  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.170892  2604 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:19.126543  5414 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.176075  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.178730  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:19.179177  2650 connection.cc:256] RPC call timeout handler was delayed by -1.2547351320000644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 14:52:19.181732  2604 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:19.184861  5415 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:19.187970  5415 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:19.191463  5415 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.193466  2755 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:19.194548  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 975) took 1878 ms. Trace:
I20260709 14:52:19.143970  5398 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.195030  2755 rpcz_store.cc:276] 0709 14:52:17.316495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.168564 (+1852069us) service_pool.cc:224] Handling call
0709 14:52:19.193043 (+ 24479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.194141 (+  1098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.194462 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.141800  5192 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:19.196702  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 904) took 2732 ms. Trace:
I20260709 14:52:19.196542  5378 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:19.198378  2755 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:19.199294  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 976) took 1867 ms. Trace:
I20260709 14:52:19.199383  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:19.199990  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86975.1us
W20260709 14:52:19.202683  2726 connection.cc:256] RPC call timeout handler was delayed by -1.8593203039999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31800000000000006s for a response
I20260709 14:52:19.202783  2755 rpcz_store.cc:276] 0709 14:52:17.332010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.197522 (+1865512us) service_pool.cc:224] Handling call
0709 14:52:19.197955 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.198975 (+  1020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.199225 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:19.209051  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.197315  3543 rpcz_store.cc:276] 0709 14:52:16.464277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.183153 (+2718876us) service_pool.cc:224] Handling call
0709 14:52:19.183664 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.193223 (+  9559us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:19.193258 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:19.193308 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:19.196349 (+  3041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.196435 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.196626 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":968,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155}
I20260709 14:52:19.202154  3537 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:19.184525  5074 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:19.222162  5074 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:19.228315  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.229255  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.230189  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1009) took 1013 ms. Trace:
I20260709 14:52:19.230846  2604 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:19.230165  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.231863  3543 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:19.234797  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:19.230674  2685 rpcz_store.cc:276] 0709 14:52:18.216435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.761772 (+545337us) service_pool.cc:224] Handling call
0709 14:52:18.762209 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:18.762622 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.762650 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:18.763387 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 14:52:18.974942 (+211555us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:19.101567 (+126625us) log.cc:844] Serialized 14 byte log entry
0709 14:52:19.101931 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.102053 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:19.229924 (+127871us) raft_consensus.cc:1685] finished
0709 14:52:19.229944 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.230121 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127493,"raft.queue_time_us":2347,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":13341,"spinlock_wait_cycles":18113792,"thread_start_us":184234,"threads_started":1,"wal-append.queue_time_us":355520}
I20260709 14:52:19.237560  5058 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:19.247582  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:19.248078  5378 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.248399  3543 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:19.248062  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.249976  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.251008  2755 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:19.251995  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.253405  2755 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:19.255872  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.256943  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.260589  3543 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:19.261833  5432 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:19.263221  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80856.1us
I20260709 14:52:19.236653  5069 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:19.353585  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.330703  2685 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:19.308597  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1067) took 2262 ms. Trace:
W20260709 14:52:19.298204  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 798) took 2769 ms (client timeout 1524 ms). Trace:
I20260709 14:52:19.264905  3543 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 14:52:19.355890  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91343.1us
I20260709 14:52:19.299865  2753 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:19.356019  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.370324  2606 rpcz_store.cc:276] 0709 14:52:17.046798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.023606 (+976808us) service_pool.cc:224] Handling call
0709 14:52:19.159215 (+1135609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.160186 (+   971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.308200 (+148014us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003791d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.308486 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610689920}
W20260709 14:52:19.371791  3546 rpcz_store.cc:269] 0709 14:52:16.528296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.331886 (+803590us) service_pool.cc:224] Handling call
0709 14:52:19.297824 (+1965938us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.298079 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116719872}
I20260709 14:52:19.345362  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.375423  2686 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:19.373240  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.372349  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 911) took 2886 ms. Trace:
I20260709 14:52:19.324659  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.374058  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1071) took 2385 ms. Trace:
I20260709 14:52:19.322037  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:19.379633  3543 rpcz_store.cc:276] 0709 14:52:16.485910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.262107 (+2776197us) service_pool.cc:224] Handling call
0709 14:52:19.262633 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.371993 (+109360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.372281 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.381075  2680 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:52:19.380656  2753 rpcz_store.cc:276] 0709 14:52:16.988647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.636767 (+648120us) service_pool.cc:224] Handling call
0709 14:52:17.638333 (+  1566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.286912 (+1648579us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54001ed820. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:52:19.373698 (+ 86786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.373991 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30409600}
I20260709 14:52:19.382458  2604 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:19.382622  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:19.382948  2604 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:19.383347  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:19.318365  5438 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.633s	user 0.001s	sys 0.001s
W20260709 14:52:19.383291  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61330.8us
I20260709 14:52:19.383370  2606 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:19.384232  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:19.297192  5458 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 14:52:19.385248  5222 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:19.386263  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:19.387162  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.390677  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 669) took 1489 ms. Trace:
I20260709 14:52:19.413384  3545 rpcz_store.cc:276] 0709 14:52:17.901142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.061597 (+160455us) service_pool.cc:224] Handling call
0709 14:52:19.390421 (+1328824us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.390601 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218450688}
I20260709 14:52:19.390937  5460 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:19.400434  2604 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:19.406056  3547 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 14:52:19.416134  3547 pending_rounds.cc:85] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:19.416421  3547 pending_rounds.cc:107] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 14:52:19.416675  3547 raft_consensus.cc:2889] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:19.417872  2604 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:19.408586  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.409021  2686 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:19.384541  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.419373  2685 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.321054  5432 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:19.412747  2753 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6. Re-sending same reply.
I20260709 14:52:19.373800  5074 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:19.269943  5398 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:19.423094  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.607s (SENT)
I20260709 14:52:19.415639  5222 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:19.437603  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.438483  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.446651  3545 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:19.436887  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.418570  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:19.425865  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.427155  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 914) took 1852 ms. Trace:
I20260709 14:52:19.417431  5205 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:19.443395  5222 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.482656  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.493062  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.502118  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:19.503305  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:19.533818  3409 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.982s (SENT)
I20260709 14:52:19.473448  5074 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:19.474061  5205 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:19.541935  2606 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:19.544379  2606 pending_rounds.cc:85] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 4
I20260709 14:52:19.538256  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:19.545081  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 97460.3us
W20260709 14:52:19.492611  5402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.005s	sys 0.014s
I20260709 14:52:19.448679  3547 rpcz_store.cc:276] 0709 14:52:17.567555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.405167 (+1837612us) service_pool.cc:224] Handling call
0709 14:52:19.405601 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.417172 (+ 11571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.417431 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.546427  2606 pending_rounds.cc:107] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:52:19.546905  2606 raft_consensus.cc:2889] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:19.547323  3547 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 14:52:19.548920  2755 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.548688  2604 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 14:52:19.549638  2604 pending_rounds.cc:85] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 2
I20260709 14:52:19.549981  2604 pending_rounds.cc:107] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 14:52:19.550304  2604 raft_consensus.cc:2889] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:19.552659  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129609us
I20260709 14:52:19.463371  5398 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.553828  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:19.555094  2603 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:19.552466  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:19.546100  5402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.899s	user 0.005s	sys 0.015s
W20260709 14:52:19.537307  5312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.014s	sys 0.028s
I20260709 14:52:19.558784  2685 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:19.560688  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.557801  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:19.562563  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:19.564121  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 910) took 3078 ms. Trace:
I20260709 14:52:19.564297  2604 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:19.564642  3537 rpcz_store.cc:276] 0709 14:52:16.485564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.211384 (+2725820us) service_pool.cc:224] Handling call
0709 14:52:19.229617 (+ 18233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.229908 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:19.235365 (+  5457us) log.cc:844] Serialized 14 byte log entry
0709 14:52:19.389068 (+153703us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800047d40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:52:19.563755 (+174687us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:19.563816 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:19.563865 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.563935 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.564062 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17705,"prepare.queue_time_us":4336,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"spinlock_wait_cycles":6651264,"thread_start_us":156659,"threads_started":1}
I20260709 14:52:19.566382  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:19.570194  3537 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 14:52:19.570300  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 142633us
I20260709 14:52:19.571892  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:19.548615  5205 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 14:52:19.559921  5312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.014s	sys 0.028s
I20260709 14:52:19.577167  5074 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:19.577913  5074 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:19.572983  5414 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:19.564298  5422 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.007s
I20260709 14:52:19.579327  5414 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.579035  5074 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.579917  5414 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.580190  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1061) took 5231 ms. Trace:
I20260709 14:52:19.580632  2612 rpcz_store.cc:276] 0709 14:52:14.349103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.328260 (+1979157us) service_pool.cc:224] Handling call
0709 14:52:16.367929 (+ 39669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.288419 (+1920490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.288455 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:18.288501 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:18.288562 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.579884 (+1291322us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400371c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:19.579950 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.580127 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9220,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"spinlock_wait_cycles":4329500416}
I20260709 14:52:19.584874  2605 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:19.575065  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.585920  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.586359  2755 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:19.585386  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1055) took 5284 ms. Trace:
I20260709 14:52:19.588999  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.590255  2752 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:19.563833  5460 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:19.593783  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 912) took 3107 ms. Trace:
I20260709 14:52:19.606519  5460 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.607334  5460 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:19.606590  3546 rpcz_store.cc:276] 0709 14:52:16.486234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.381112 (+2894878us) service_pool.cc:224] Handling call
0709 14:52:19.391731 (+ 10619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:19.392207 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:19.392238 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:19.392800 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 14:52:19.547724 (+154924us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:19.548144 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.548872 (+   728us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:19.593492 (+ 44620us) raft_consensus.cc:1685] finished
0709 14:52:19.593518 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.593705 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":153828,"threads_started":1,"wal-append.queue_time_us":180146}
I20260709 14:52:19.609644  3546 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:19.612023  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1068) took 2114 ms. Trace:
I20260709 14:52:19.612547  2609 rpcz_store.cc:276] 0709 14:52:17.497310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.304040 (+806730us) service_pool.cc:224] Handling call
0709 14:52:18.304455 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.328765 (+ 24310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.328804 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:18.328852 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:18.328887 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.611767 (+1282880us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:19.611826 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.611955 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55333248}
I20260709 14:52:19.613876  5398 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.615864  5342 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:19.618153  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.619441  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.620807  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.570492  5069 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:19.604357  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.605703  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.626405  2753 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:19.626533  2686 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.588258  2608 rpcz_store.cc:276] 0709 14:52:14.300762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:16.029202 (+1728440us) service_pool.cc:224] Handling call
0709 14:52:16.101979 (+ 72777us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:16.903159 (+801180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:16.903191 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:16.958242 (+ 55051us) log.cc:844] Serialized 52 byte log entry
0709 14:52:17.107483 (+149241us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b480026cdd8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:52:17.390719 (+283236us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:17.390794 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:17.390977 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:18.161609 (+770632us) raft_consensus.cc:1685] finished
0709 14:52:18.161631 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.582699 (+1421068us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b54001d9050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.585305 (+  2606us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":545174,"raft.queue_time_us":611484,"raft.run_cpu_time_us":3770,"raft.run_wall_time_us":122339,"spinlock_wait_cycles":119104512,"thread_start_us":362816,"threads_started":1,"wal-append.queue_time_us":503697,"wal-append.run_cpu_time_us":4599,"wal-append.run_wall_time_us":1666609}
W20260709 14:52:19.635463  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.644472  5402 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:19.637975  5205 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:19.648873  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 796) took 3121 ms. Trace:
I20260709 14:52:19.649034  5469 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:19.649503  3543 rpcz_store.cc:276] 0709 14:52:16.527471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.382410 (+2854939us) service_pool.cc:224] Handling call
0709 14:52:19.382757 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:19.385250 (+  2493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:19.385280 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:19.385958 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 14:52:19.547984 (+162026us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:19.548365 (+   381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.549119 (+   754us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:19.648623 (+ 99504us) raft_consensus.cc:1685] finished
0709 14:52:19.648647 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.648798 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2604,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":132,"thread_start_us":159937,"threads_started":1,"wal-append.queue_time_us":238595}
I20260709 14:52:19.650729  5402 consensus_queue.cc:1243] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:19.651875  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:19.652778  2609 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:19.656227  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.658105  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.659060  2685 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:19.657196  2753 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:19.674805  2686 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:19.629863  5069 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.625653  5398 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.715195  5205 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:19.716459  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:19.770851  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.771625  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.772589  2680 raft_consensus.cc:2376] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7. Re-sending same reply.
I20260709 14:52:19.774068  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.666121  5422 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 14:52:19.727958  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:19.733944  3536 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:19.726382  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61047.5us
W20260709 14:52:19.738510  5462 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 14:52:19.740000  5222 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:19.743074  2576 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.918s (SENT)
W20260709 14:52:19.778079  2576 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.936s (SENT)
I20260709 14:52:19.723495  5069 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:19.741850  5414 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.733210  5398 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:19.761425  5442 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.066s	user 0.003s	sys 0.000s
W20260709 14:52:19.765295  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 655) took 3349 ms (client timeout 2936 ms). Trace:
W20260709 14:52:19.766316  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 664) took 2300 ms (client timeout 1706 ms). Trace:
W20260709 14:52:19.801705  3538 rpcz_store.cc:269] 0709 14:52:17.466170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.691865 (+225695us) service_pool.cc:224] Handling call
0709 14:52:19.766077 (+2074212us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001a0254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.766221 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440377600}
I20260709 14:52:19.680444  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:19.800429  3542 rpcz_store.cc:269] 0709 14:52:16.416205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.882184 (+1465979us) service_pool.cc:224] Handling call
0709 14:52:19.764968 (+1882784us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.765195 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2765721344}
I20260709 14:52:19.806320  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:37148 (request call id 779) took 6341 ms. Trace:
W20260709 14:52:19.820614  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 254277us
I20260709 14:52:19.820863  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:19.807106  3536 rpcz_store.cc:276] 0709 14:52:13.437162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.438444 (+  1282us) service_pool.cc:224] Handling call
0709 14:52:19.725616 (+6287172us) spinlock_profiling.cc:243] Waited 3.45 s on lock 0x7b54004b2dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e63da3d 00007fa17360dcd0 00007fa16cb92254 00007fa16cb921e2 00007fa16cb92111 00007fa16cb9208c 00007fa16cb91342 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.777968 (+ 52352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7945011072}
I20260709 14:52:19.825479  2753 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:19.662117  5426 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:19.823603  2680 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:19.828044  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:19.659787  5402 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:19.835489  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:19.837164  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60045.7us
W20260709 14:52:19.838755  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.014s (SENT)
I20260709 14:52:19.839490  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.770529  5460 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.840864  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:19.666581  5334 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.169s
I20260709 14:52:19.836592  5402 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:19.824452  5205 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.835829  5426 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:19.862671  2576 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.038s (SENT)
I20260709 14:52:19.863430  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.864704  5238 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 14:52:19.880057  5440 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.186s	user 0.005s	sys 0.000s
I20260709 14:52:19.880256  5414 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:19.861215  5205 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:19.855079  5378 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:19.892750  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 459) took 2397 ms. Trace:
I20260709 14:52:19.893306  2607 rpcz_store.cc:276] 0709 14:52:17.495086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.247678 (+752592us) service_pool.cc:224] Handling call
0709 14:52:18.248171 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:18.248655 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:18.248689 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:18.308890 (+ 60201us) log.cc:844] Serialized 52 byte log entry
0709 14:52:18.510722 (+201832us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:18.510799 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:18.510980 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:19.760355 (+1249375us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:19.888804 (+128449us) raft_consensus.cc:1685] finished
0709 14:52:19.888842 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.892650 (+  3808us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":822,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":62614400,"thread_start_us":183354,"threads_started":1,"wal-append.queue_time_us":383812}
W20260709 14:52:19.894786  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:19.900653  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:19.903872  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1064) took 2926 ms (client timeout 3031 ms). Trace:
I20260709 14:52:19.865612  5471 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:52:19.905190  2603 rpcz_store.cc:269] 0709 14:52:16.977579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.852871 (+875292us) service_pool.cc:224] Handling call
0709 14:52:19.903551 (+2050680us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:19.903737 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3832218496}
I20260709 14:52:19.905987  2686 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:19.903961  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61477.6us
W20260709 14:52:19.910357  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47725.4us
W20260709 14:52:19.912593  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.065s (SENT)
I20260709 14:52:19.913228  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.917404  5426 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:19.918870  5426 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:19.897573  5375 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 14:52:19.922853  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1071) took 2277 ms. Trace:
I20260709 14:52:19.923599  2605 rpcz_store.cc:276] 0709 14:52:17.645524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.874356 (+1228832us) service_pool.cc:224] Handling call
0709 14:52:18.904644 (+ 30288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.585482 (+680838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.922261 (+336779us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:19.922711 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563783936}
W20260709 14:52:19.925215  5439 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.064s	user 0.000s	sys 0.001s
I20260709 14:52:19.926405  5088 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:19.927147  5088 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:19.927469  5479 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:19.855077  5472 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 14:52:19.929349  2610 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:19.934743  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.936216  2755 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:19.939674  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.953984  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:19.954619  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:19.956667  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:19.957410  3547 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:19.958081  3536 pending_rounds.cc:85] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:19.958449  3536 pending_rounds.cc:107] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 14:52:19.958688  3536 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:19.958779  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:19.959735  2755 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.963267  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:19.964061  3538 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:19.970523  2755 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:19.971212  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.106s (SENT)
I20260709 14:52:19.967275  3547 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
I20260709 14:52:19.976277  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.977411  5378 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:19.973495  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:19.979076  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.114s (SENT)
I20260709 14:52:19.979552  5432 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:19.979671  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:19.980521  5432 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:19.980994  5378 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:19.981271  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:19.972852  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1076) took 2568 ms. Trace:
I20260709 14:52:19.982722  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:19.983166  2755 rpcz_store.cc:276] 0709 14:52:17.404694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.971816 (+2567122us) service_pool.cc:224] Handling call
0709 14:52:19.972222 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.972444 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:19.972471 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:19.972519 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:19.972565 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.972622 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:19.972742 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:19.908614  5471 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:19.987658  3538 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
W20260709 14:52:19.993278  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63662us
I20260709 14:52:19.949438  5375 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:20.006079  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:20.011142  2754 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:20.011729  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:19.900523  5475 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 14:52:19.982676  5385 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:20.022536  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:20.023983  2605 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:19.981069  5432 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:19.984891  4786 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:19.986204  5342 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:19.928638  5088 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:20.003181  5471 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:20.049317  3409 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.780s (SENT)
W20260709 14:52:20.086088  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.248s (SENT)
I20260709 14:52:20.107044  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.011826  5383 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:19.923636  5402 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:20.119753  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:20.106046  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:19.992305  5411 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 14:52:19.985852  5477 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.132s
W20260709 14:52:20.134590  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48564.5us
W20260709 14:52:20.139294  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90054.1us
I20260709 14:52:20.140219  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:20.131140  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 146935us
I20260709 14:52:20.112071  5222 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:20.180408  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:20.036860  5385 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.137956  5375 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:20.119191  5398 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.193789  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:20.171701  5088 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:20.151711  5483 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 14:52:20.209753  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:20.216605  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:20.220340  2683 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:20.225319  3538 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:20.241719  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.239264  5417 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:20.303748  2683 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:20.246762  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64368.4us
I20260709 14:52:20.317330  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:20.191545  5460 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.184651  5484 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:20.192756  5222 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 14:52:20.309666  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:20.200364  5342 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:20.252887  5417 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:20.356453  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.069s	sys 0.135s
W20260709 14:52:20.357832  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:20.359961  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.069s	sys 0.135s
W20260709 14:52:20.361243  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 520440us
I20260709 14:52:20.362459  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:20.338285  5205 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:20.385222  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 75715.2us
I20260709 14:52:20.347149  5484 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:20.477715  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 328062us
W20260709 14:52:20.518651  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.522154  2611 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:20.412058  5396 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:20.443861  5404 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 14:52:20.526239  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:20.526332  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.481441  5342 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:20.549558  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.567721  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 977) took 2693 ms. Trace:
I20260709 14:52:20.569264  2607 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:20.568691  2755 rpcz_store.cc:276] 0709 14:52:17.874455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.988772 (+2114317us) service_pool.cc:224] Handling call
0709 14:52:19.989234 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.989537 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:19.989945 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.567194 (+577249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.567266 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:20.567330 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.567421 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.567604 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3376,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"thread_start_us":575271,"threads_started":2,"wal-append.queue_time_us":423966}
I20260709 14:52:20.573661  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:20.575049  3537 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:20.578424  5487 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:20.578501  5496 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:20.579443  5487 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:20.580269  5496 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.582993  5496 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:20.583110  5487 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.583155  5205 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:20.584262  5205 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.587702  5205 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:20.531914  5378 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:20.555052  5466 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.173s	user 0.001s	sys 0.008s
W20260709 14:52:20.562788  5486 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.225s	user 0.000s	sys 0.004s
I20260709 14:52:20.595635  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1077) took 2689 ms. Trace:
I20260709 14:52:20.600585  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:20.591142  5394 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.894s	user 0.009s	sys 0.015s
I20260709 14:52:20.608071  3538 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 14:52:20.608754  3538 pending_rounds.cc:85] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
I20260709 14:52:20.609218  3538 pending_rounds.cc:107] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 14:52:20.608036  2755 rpcz_store.cc:276] 0709 14:52:17.906278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.594293 (+2688015us) service_pool.cc:224] Handling call
0709 14:52:20.594854 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.595132 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:20.595169 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.595229 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:20.595283 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.595361 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.595535 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:20.609889  3538 raft_consensus.cc:2889] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:20.616654  2611 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:20.571476  5396 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:20.595063  5417 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:20.620661  5396 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:20.603793  5493 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.248s	user 0.000s	sys 0.005s
W20260709 14:52:20.607005  5488 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.163s	user 0.004s	sys 0.000s
I20260709 14:52:20.605872  5378 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:20.637815  5426 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.002s
I20260709 14:52:20.640681  5411 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:20.641695  2752 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:20.641602  5411 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.643265  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1087) took 2351 ms. Trace:
I20260709 14:52:20.643841  2752 rpcz_store.cc:276] 0709 14:52:18.291508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.640738 (+2349230us) service_pool.cc:224] Handling call
0709 14:52:20.641351 (+   613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:20.642561 (+  1210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.643008 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:20.638177  5385 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.655802  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 982) took 2761 ms. Trace:
I20260709 14:52:20.656526  2753 rpcz_store.cc:276] 0709 14:52:17.893961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.318770 (+2424809us) service_pool.cc:224] Handling call
0709 14:52:20.353482 (+ 34712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.353867 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:20.382102 (+ 28235us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.655031 (+272929us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.655336 (+   305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.655405 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:20.655468 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.655551 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.655727 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31375,"prepare.queue_time_us":28101,"prepare.run_cpu_time_us":8776,"prepare.run_wall_time_us":102542,"thread_start_us":232989,"threads_started":1,"wal-append.queue_time_us":249194}
W20260709 14:52:20.658232  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.002s	sys 0.015s
W20260709 14:52:20.658851  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.002s	sys 0.016s
I20260709 14:52:20.660287  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 987) took 1902 ms. Trace:
I20260709 14:52:20.660972  2753 rpcz_store.cc:276] 0709 14:52:18.757271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.658943 (+1901672us) service_pool.cc:224] Handling call
0709 14:52:20.659350 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.659649 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:20.659696 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.659762 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:20.659816 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.659907 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.660088 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:20.662087  2652 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.600s (SENT)
I20260709 14:52:20.662679  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.655457  5037 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:20.665928  5399 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:20.667807  2604 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:20.668108  5203 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.669517  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:20.671800  5471 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.533s	user 0.004s	sys 0.010s
W20260709 14:52:20.672807  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.667922  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:20.683457  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.683651  2609 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:20.676914  2611 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4.
W20260709 14:52:20.684350  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.605s (SENT)
I20260709 14:52:20.692260  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.699451  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:20.700608  2611 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:20.687896  5490 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 14:52:20.706746  5492 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:20.707036  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.708590  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.716431  2609 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:20.722863  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 978) took 2848 ms. Trace:
I20260709 14:52:20.723832  2754 rpcz_store.cc:276] 0709 14:52:17.874856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.035504 (+2160648us) service_pool.cc:224] Handling call
0709 14:52:20.035980 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.036470 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:20.140367 (+103897us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.699701 (+559334us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.722299 (+ 22598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.722377 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:20.722442 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.722542 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.722734 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22103,"prepare.queue_time_us":116459,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":6941,"raft.queue_time_us":281619,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"thread_start_us":552807,"threads_started":3,"wal-append.queue_time_us":456091}
I20260709 14:52:20.733934  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:20.695616  5495 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:20.694145  5471 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.742722  5411 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:20.743086  5069 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.934s	user 0.013s	sys 0.009s
I20260709 14:52:20.745270  5069 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.746631  2680 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:20.747547  5335 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:20.749869  5069 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.751303  2678 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:20.751796  5432 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.006s
I20260709 14:52:20.752308  5432 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:20.747782  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1073) took 3723 ms. Trace:
I20260709 14:52:20.753832  2751 rpcz_store.cc:276] 0709 14:52:17.024559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.655941 (+631382us) service_pool.cc:224] Handling call
0709 14:52:17.687857 (+ 31916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.759644 (+1071787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.747438 (+1987794us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:20.747655 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456502656}
I20260709 14:52:20.728220  5492 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:20.755972  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71685us
I20260709 14:52:20.757411  5492 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.704674  5375 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:20.770176  5007 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:20.772518  2724 connection.cc:256] RPC call timeout handler was delayed by -2.851357741000129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 14:52:20.721243  5203 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:20.791242  2724 connection.cc:256] RPC call timeout handler was delayed by -2.5390577410000787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 14:52:20.791352  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.793066  2604 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
W20260709 14:52:20.800800  2724 connection.cc:256] RPC call timeout handler was delayed by -1.7226996760000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2743s for a response
W20260709 14:52:20.801529  2724 connection.cc:256] RPC call timeout handler was delayed by -1.5665959269999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 14:52:20.802013  2724 connection.cc:256] RPC call timeout handler was delayed by -1.2925577410001097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3256s for a response
I20260709 14:52:20.820366  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:20.821503  2604 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:20.830117  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.766382  5432 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.814940  5492 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:20.789842  5203 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.836716  5492 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.811061  5484 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:20.842526  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 110609us
I20260709 14:52:20.843089  3545 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:20.846462  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.846856  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 986) took 2090 ms. Trace:
I20260709 14:52:20.845820  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:20.831740  5378 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:20.840952  5377 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.850162  5503 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:20.892452  2604 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:20.893561  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.869789  5007 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 14:52:20.869119  5506 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:20.895187  5007 consensus_queue.cc:1243] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 14:52:20.879808  5396 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 14:52:20.885030  5507 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:20.897033  5507 consensus_queue.cc:1243] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 14:52:20.885648  5498 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.050s	user 0.001s	sys 0.001s
W20260709 14:52:20.898198  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 442) took 2902 ms (client timeout 1982 ms). Trace:
I20260709 14:52:20.898037  5499 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:20.861210  5504 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:20.886739  5194 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:20.887779  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 984) took 2839 ms. Trace:
I20260709 14:52:20.888378  5041 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:20.867810  5484 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 14:52:20.881835  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:20.897504  5396 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:20.899120  5499 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:20.912155  5371 raft_consensus.cc:993] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:52:20.912511  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.935233  5035 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:20.935868  5035 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:20.936564  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.936821  5035 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.869879  5365 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.914306  2755 rpcz_store.cc:276] 0709 14:52:18.048125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.610625 (+2562500us) service_pool.cc:224] Handling call
0709 14:52:20.611397 (+   772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:20.611762 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:20.611802 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:20.613365 (+  1563us) log.cc:844] Serialized 26 byte log entry
0709 14:52:20.725902 (+112537us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:20.725968 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.726096 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:20.887487 (+161391us) raft_consensus.cc:1685] finished
0709 14:52:20.887513 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.887676 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":45895,"threads_started":1,"wal-append.queue_time_us":134008}
I20260709 14:52:20.938012  5035 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:20.938184  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1012) took 1318 ms. Trace:
I20260709 14:52:20.938676  2685 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:20.939323  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.940546  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 456) took 1337 ms. Trace:
I20260709 14:52:20.940999  2685 rpcz_store.cc:276] 0709 14:52:19.603519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.660352 (+ 56833us) service_pool.cc:224] Handling call
0709 14:52:20.938468 (+1278116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.939240 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.940337 (+  1097us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001d10d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:20.940480 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2814501248}
I20260709 14:52:20.942639  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:20.935297  5499 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:20.943995  5499 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.944938  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1074) took 1806 ms. Trace:
I20260709 14:52:20.945473  2610 rpcz_store.cc:276] 0709 14:52:19.138653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.138888 (+   235us) service_pool.cc:224] Handling call
0709 14:52:20.944730 (+1805842us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:20.944876 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800127744}
I20260709 14:52:20.947815  5402 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:20.916730  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1069) took 5525 ms. Trace:
I20260709 14:52:20.948663  2756 rpcz_store.cc:276] 0709 14:52:15.390944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.458261 (+2067317us) service_pool.cc:224] Handling call
0709 14:52:17.458702 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:17.635095 (+176393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.916451 (+3281356us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:20.916659 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271883008}
I20260709 14:52:20.920853  5377 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:20.951017  2685 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:20.955490  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.956892  5035 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:20.958138  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.922044  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:20.964110  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:20.965651  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:20.966059  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:20.966533  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:20.892518  5492 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:20.923324  5403 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:20.926970  5194 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:20.967674  5378 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:20.966589  2684 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:20.968464  5378 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:52:20.970652  5378 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:20.971669  5508 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:20.972106  5508 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:20.973001  5508 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:20.974480  5508 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:20.974915  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:20.975553  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:20.976064  3412 connection.cc:256] RPC call timeout handler was delayed by -1.1353398259999494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260709 14:52:20.976505  3412 connection.cc:256] RPC call timeout handler was delayed by -1.118084607000128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 14:52:20.976909  3412 connection.cc:256] RPC call timeout handler was delayed by -1.0071327790001305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.396s for a response
W20260709 14:52:20.978061  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 615716us
I20260709 14:52:20.978768  5479 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:20.978945  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:20.918857  5396 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:20.981415  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1014) took 1360 ms. Trace:
I20260709 14:52:20.981197  5396 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:20.981849  2683 rpcz_store.cc:276] 0709 14:52:19.620520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.237505 (+616985us) service_pool.cc:224] Handling call
0709 14:52:20.241121 (+  3616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.305351 (+ 64230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.981088 (+675737us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:20.981332 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77295488}
I20260709 14:52:20.928926  5471 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:20.879457  5411 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 14:52:20.937350  2603 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:20.996773  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.998116  2603 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 14:52:20.938544  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1000) took 3047 ms (client timeout 3385 ms). Trace:
W20260709 14:52:20.999066  2679 rpcz_store.cc:269] 0709 14:52:17.891084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.891230 (+   146us) service_pool.cc:224] Handling call
0709 14:52:20.938330 (+3047100us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:20.938479 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173284608}
I20260709 14:52:20.905987  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.900329  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:20.959765  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:21.001456  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101258us
I20260709 14:52:21.001947  2610 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:20.921701  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.962359  5504 raft_consensus.cc:1081] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:52:20.993746  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:20.992915  5035 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:20.985930  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.005658  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:20.994879  2683 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:21.006603  2606 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:20.938623  2680 rpcz_store.cc:276] 0709 14:52:19.619967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.838870 (+218903us) service_pool.cc:224] Handling call
0709 14:52:19.854216 (+ 15346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.747136 (+892920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.937891 (+190755us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:20.938113 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051851648}
I20260709 14:52:20.881655  5069 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 14:52:20.898659  2682 rpcz_store.cc:269] 0709 14:52:17.995800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.156775 (+160975us) service_pool.cc:224] Handling call
0709 14:52:20.897913 (+2741138us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:20.898091 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4618727168}
I20260709 14:52:21.004123  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.003185  2612 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:20.963461  2756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:21.009541  2756 raft_consensus.cc:606] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.009856  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:20.872002  5491 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 14:52:20.945415  5499 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:21.015208  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1075) took 3698 ms (client timeout 1706 ms). Trace:
W20260709 14:52:21.088537  2759 rpcz_store.cc:269] 0709 14:52:17.316287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.369899 (+ 53612us) service_pool.cc:224] Handling call
0709 14:52:21.014814 (+3644915us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:21.015061 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4865853312}
I20260709 14:52:21.013792  2604 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:21.010886  5335 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:21.089838  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.090379  5371 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:21.005019  5492 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:21.108441  5426 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:21.011293  4786 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:21.110373  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100640us
I20260709 14:52:21.124571  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.007014  5378 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 14:52:20.990067  5396 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:21.112385  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121220us
I20260709 14:52:21.117847  5335 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:21.133723  5335 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:21.134126  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.135087  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.121876  5508 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.136487  5508 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.125990  5492 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:21.139194  4989 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:21.140143  5403 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:21.141261  5403 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:21.144976  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.140313  4989 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 14:52:21.147807  4989 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:21.148216  4989 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:21.149140  4989 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:21.150452  4989 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:21.140161  5492 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.151609  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.153939  2679 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:21.015594  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.106997  2612 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:21.126583  5469 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 14:52:21.158808  5508 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:21.120992  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113575us
I20260709 14:52:21.123299  5426 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:21.159646  5508 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.160130  5426 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.160391  5508 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:21.107446  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 128598us
I20260709 14:52:21.163188  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.165972  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.169835  5383 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:21.169981  5007 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:21.172435  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.173507  5416 raft_consensus.cc:2764] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 14:52:20.847620  2752 rpcz_store.cc:276] 0709 14:52:18.756877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.645866 (+1888989us) service_pool.cc:224] Handling call
0709 14:52:20.673002 (+ 27136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.673411 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:20.786133 (+112722us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.846190 (+ 60057us) log.cc:844] Serialized 14 byte log entry
0709 14:52:20.846408 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:20.846460 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:20.846520 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:20.846606 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:20.846756 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15833,"prepare.queue_time_us":67728,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"thread_start_us":143965,"threads_started":2}
I20260709 14:52:21.174873  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.177400  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:21.178110  3537 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.181691  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.182279  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 923) took 1981 ms. Trace:
I20260709 14:52:21.182821  3546 rpcz_store.cc:276] 0709 14:52:19.201168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.610486 (+409318us) service_pool.cc:224] Handling call
0709 14:52:19.766787 (+156301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:19.855285 (+ 88498us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 14:52:19.855340 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:19.867220 (+ 11880us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:19.867328 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:19.867450 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:21.182007 (+1314557us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54003d86d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:21.182194 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31304,"raft.queue_time_us":102563,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":4267,"spinlock_wait_cycles":358588672,"thread_start_us":79885,"threads_started":1}
I20260709 14:52:21.138937  5334 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:21.156833  5203 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:21.204012  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.135123  2685 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:21.189143  2652 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.998s (SENT) [suppressed 56 similar messages]
I20260709 14:52:21.195915  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.120517  5375 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:21.120483  5371 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:21.218139  3537 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:20.968467  5074 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:21.208345  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.013875  5035 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.228371  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.237106  3542 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:21.229382  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:21.232599  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.238951  3540 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7.
I20260709 14:52:21.233690  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.195294  5499 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 14:52:21.242139  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.247225  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.249029  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.249879  3537 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7.
I20260709 14:52:21.170759  5383 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:21.251480  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.253328  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.142513  5403 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.258939  5375 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.234541  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.260643  3547 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:21.170738  5007 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:21.181313  5508 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:21.269412  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.271032  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:21.277137  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.230360  5371 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:21.287704  3537 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:21.287420  5469 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:21.292538  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.300051  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:21.312726  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.314383  3546 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:21.305312  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:21.299575  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 305864us
I20260709 14:52:21.317658  3536 raft_consensus.cc:2376] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:21.302563  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:21.323391  3540 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6. Re-sending same reply.
I20260709 14:52:21.323793  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.296653  5402 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:21.293953  5497 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:21.259770  5375 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.333262  2576 leader_election.cc:400] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7.
I20260709 14:52:21.340050  5088 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:21.293047  5416 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:21.353250  2576 leader_election.cc:403] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:52:21.357815  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.331004  5383 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:21.361208  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 909) took 4892 ms (client timeout 1607 ms). Trace:
W20260709 14:52:21.339200  5355 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 14:52:21.361826  3541 rpcz_store.cc:269] 0709 14:52:16.469316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:17.442543 (+973227us) service_pool.cc:224] Handling call
0709 14:52:21.360926 (+3918383us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54002dc3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:21.361112 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048550656}
W20260709 14:52:21.363751  2577 connection.cc:256] RPC call timeout handler was delayed by -2.9246330619999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 14:52:21.370765  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118820us
W20260709 14:52:21.372288  2577 connection.cc:256] RPC call timeout handler was delayed by -1.8107781030000751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 14:52:21.372781  2577 connection.cc:256] RPC call timeout handler was delayed by -1.6101033740000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 14:52:21.373075  2577 connection.cc:256] RPC call timeout handler was delayed by -1.3616781030000311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23530000000000004s for a response
I20260709 14:52:21.373184  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.373685  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.343328  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.301604  5335 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.389869  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.356670  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.391906  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.351758  5497 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:21.395691  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.401510  2612 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:21.362576  5416 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:21.406352  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.406754  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 907) took 3404 ms. Trace:
I20260709 14:52:21.407226  2677 rpcz_store.cc:276] 0709 14:52:18.002648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.322860 (+320212us) service_pool.cc:224] Handling call
0709 14:52:18.666097 (+343237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:18.776978 (+110881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:21.406467 (+2629489us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:21.406677 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253265792}
I20260709 14:52:21.408090  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:21.410076  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53536.8us
I20260709 14:52:21.411847  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.409017  2679 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:21.414336  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:21.415318  2679 raft_consensus.cc:2376] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7. Re-sending same reply.
I20260709 14:52:21.416803  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.423341  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.423667  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:21.425024  2677 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:21.426748  2612 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:21.404448  5497 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:21.340883  5007 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.453522  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 935) took 1273 ms. Trace:
I20260709 14:52:21.413296  5416 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.413914  5527 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 7
I20260709 14:52:21.449434  5335 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:21.447947  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:21.407866  5417 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.447448  5035 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:21.456097  3543 rpcz_store.cc:276] 0709 14:52:20.180581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.180749 (+   168us) service_pool.cc:224] Handling call
0709 14:52:20.181096 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:20.192806 (+ 11710us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:20.977313 (+784507us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b1800377ce0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:52:21.177117 (+199804us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b1800377ce0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:52:21.177505 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 14:52:21.381352 (+203847us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:21.382161 (+   809us) log.cc:844] Serialized 26 byte log entry
0709 14:52:21.382355 (+   194us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:21.382398 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:21.382483 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:21.453298 (+ 70815us) raft_consensus.cc:1685] finished
0709 14:52:21.453318 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:21.453445 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89046,"prepare.queue_time_us":985825,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":12249,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":33693,"spinlock_wait_cycles":102344576,"thread_start_us":198237,"threads_started":1,"wal-append.queue_time_us":252933}
I20260709 14:52:21.419932  5359 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:21.429737  2612 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:21.474712  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101005us
I20260709 14:52:21.478538  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:21.420135  5088 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:21.488368  5088 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:21.488977  5007 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:21.488770  2612 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:21.490633  2612 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:21.430883  5530 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:21.498413  5530 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:21.498881  5530 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.499987  2612 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 14:52:21.503006  5417 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.497737  5426 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:21.509928  5508 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 14:52:21.484894  5525 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 14:52:21.467085  5037 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:21.443867  5375 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:21.517074  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:21.517406  5426 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:21.529800  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 156659us
I20260709 14:52:21.534164  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:21.540633  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.540401  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.543244  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:21.545665  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.489851  5088 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.546432  2603 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4. Re-sending same reply.
I20260709 14:52:21.533854  5359 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:21.547456  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:21.557271  2612 pending_rounds.cc:85] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 1
I20260709 14:52:21.557796  2612 pending_rounds.cc:107] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 14:52:21.558295  2612 raft_consensus.cc:2889] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:21.561820  5359 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:21.563005  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 458) took 1798 ms (client timeout 2223 ms). Trace:
W20260709 14:52:21.563532  2686 rpcz_store.cc:269] 0709 14:52:19.764365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.792510 (+ 28145us) service_pool.cc:224] Handling call
0709 14:52:21.562735 (+1770225us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:21.562926 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167359360}
I20260709 14:52:21.565038  2684 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:21.556020  5380 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.567634  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1015) took 1946 ms. Trace:
I20260709 14:52:21.567454  5380 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:21.571105  5051 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:21.573135  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.574314  2684 rpcz_store.cc:276] 0709 14:52:19.620809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.968952 (+1348143us) service_pool.cc:224] Handling call
0709 14:52:21.564800 (+595848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.565608 (+   808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:21.567367 (+  1759us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54002b2d50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:21.567559 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369338752}
I20260709 14:52:21.576256  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:21.550956  5222 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.583397  5222 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.586087  5222 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:21.551875  5511 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:21.587057  5511 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:21.589465  5511 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:21.590536  5402 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.591274  5402 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:21.592918  5405 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:21.595011  5426 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.595731  5426 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:21.599757  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.603379  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.548810  5037 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:21.616339  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.623939  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1013) took 2003 ms. Trace:
I20260709 14:52:21.625574  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.648715  2611 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:21.626796  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:21.650063  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.550136  5411 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:21.648566  2678 rpcz_store.cc:276] 0709 14:52:19.620272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.874310 (+254038us) service_pool.cc:224] Handling call
0709 14:52:19.874742 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:20.751983 (+877241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:21.623493 (+871510us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54001c0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:21.623851 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015212288}
I20260709 14:52:21.548347  5396 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:21.649197  2605 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 4. Re-sending same reply.
W20260709 14:52:21.659428  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1072) took 1264 ms (client timeout 1684 ms). Trace:
W20260709 14:52:21.659963  2605 rpcz_store.cc:269] 0709 14:52:20.417372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.531593 (+114221us) service_pool.cc:224] Handling call
0709 14:52:20.899372 (+367779us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:21.625233 (+725861us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521e9c 00007fa16b521a5b 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:21.659355 (+ 34122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22014080}
I20260709 14:52:21.661293  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.649681  2608 raft_consensus.cc:2376] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10. Re-sending same reply.
I20260709 14:52:21.662315  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1076) took 1142 ms. Trace:
I20260709 14:52:21.662709  2608 rpcz_store.cc:276] 0709 14:52:20.520103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.543782 (+ 23679us) service_pool.cc:224] Handling call
0709 14:52:21.626558 (+1082776us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:21.662241 (+ 35683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14579072}
I20260709 14:52:21.662089  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.665256  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:21.665902  3537 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:21.668673  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:21.673560  2611 raft_consensus.cc:2376] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9. Re-sending same reply.
I20260709 14:52:21.677093  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:21.674345  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:21.682536  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.686007  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:21.686813  2752 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:21.687189  2752 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:21.685853  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:21.688760  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:21.607896  5088 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:21.608211  5051 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:21.695653  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:21.695772  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:21.702533  3537 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:21.688001  2752 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:21.681802  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.704852  2752 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:21.705129  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:21.704059  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:21.647079  5402 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:21.654497  4786 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.718937  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.719223  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.722498  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.710896  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:21.724444  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.714560  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.725380  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:21.723107  3546 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 14:52:21.726428  3546 pending_rounds.cc:85] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:21.719954  2608 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:21.663250  5396 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:21.729209  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.681731  5491 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:21.731046  2611 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:21.726799  3546 pending_rounds.cc:107] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 14:52:21.737526  3546 raft_consensus.cc:2889] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:21.681032  5426 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:21.741199  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:21.698316  5334 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:21.745647  3542 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:21.708776  5051 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.750649  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.680630  5411 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:21.777146  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51967.4us
I20260709 14:52:21.786312  3537 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:21.789795  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:21.741071  4989 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:21.735364  5396 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:21.788949  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.794190  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.746680  5334 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:21.797838  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.794596  2608 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
W20260709 14:52:21.805140  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.733587  4786 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:21.806998  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.826452  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:21.840008  2752 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:21.792994  4989 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:21.803633  5359 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:21.884392  4989 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:21.884521  5037 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:21.886021  5007 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:21.886641  5007 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:21.887152  5007 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:21.887531  5539 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 14:52:21.891109  5371 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.004s
I20260709 14:52:21.891696  5371 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:21.894187  5371 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:21.899557  5538 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 14:52:21.900514  5411 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:21.901306  5533 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 14:52:21.901572  5411 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:21.745143  5491 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:21.904412  5491 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:21.909935  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.911381  3547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:52:21.922650  3547 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:52:21.923506  3547 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:21.800419  5334 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:21.948539  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.952049  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:21.953625  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 464) took 1382 ms. Trace:
W20260709 14:52:21.957078  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:21.958745  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.791628  5524 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
W20260709 14:52:22.005077  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.016973  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 928) took 2593 ms (client timeout 3019 ms). Trace:
W20260709 14:52:22.017702  3545 rpcz_store.cc:269] 0709 14:52:19.423072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.474980 (+ 51908us) service_pool.cc:224] Handling call
0709 14:52:22.016671 (+2541691us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:22.016869 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3011478400}
I20260709 14:52:22.025533  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:22.026207  5380 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.026942  5380 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:52:22.027319  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:22.028673  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:22.030951  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:22.031629  5524 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:22.032318  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:22.032984  3545 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:22.034030  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:22.034757  3545 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:22.035765  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:22.024327  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65699.4us
I20260709 14:52:22.043705  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:22.044494  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:21.971858  5482 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:21.977335  4786 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:21.908883  5417 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:21.977361  5051 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:21.935155  5537 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 14:52:21.918933  5426 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:21.985419  5497 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 14:52:21.986099  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:21.961542  5491 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:21.988801  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:22.101670  2611 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:22.045331  2609 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:21.954249  2607 rpcz_store.cc:276] 0709 14:52:20.571357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.896301 (+324944us) service_pool.cc:224] Handling call
0709 14:52:21.950546 (+1054245us) spinlock_profiling.cc:243] Waited 9.11 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:21.953520 (+  2974us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20956032}
W20260709 14:52:22.050096  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 515975us
I20260709 14:52:22.104825  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:21.982482  5530 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:22.081065  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:21.987157  5432 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 14:52:22.074334  5335 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:22.094856  5359 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:22.107957  5497 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.109164  2608 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:22.109632  5542 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:22.110330  5051 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:22.100126  5426 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:22.049356  5035 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:22.133930  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.144449  2752 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:22.144989  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.110118  5007 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:22.023658  4989 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:22.095459  3547 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:22.126807  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.021s
I20260709 14:52:22.165246  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:22.167889  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.000s	sys 0.022s
W20260709 14:52:22.168279  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 592215us
I20260709 14:52:22.169483  5051 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:22.170210  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:22.166226  2608 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:22.163089  2755 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:22.142480  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.016s
W20260709 14:52:22.171698  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.007s	sys 0.016s
I20260709 14:52:22.173915  3547 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:22.150802  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.020s	sys 0.027s
I20260709 14:52:22.135393  5491 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:22.221535  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.021s	sys 0.027s
I20260709 14:52:22.221683  5491 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:22.221946  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 647984us
I20260709 14:52:22.222252  5491 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:22.226310  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:22.178092  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.252569  2753 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:22.253655  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.131513  5432 consensus_queue.cc:1243] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:22.182781  3545 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:22.134167  5541 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:22.255635  5541 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:22.256517  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.257256  2753 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:22.257833  5527 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.001s
I20260709 14:52:22.186287  3540 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:22.258249  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.132910  5524 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:22.136695  5198 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 14:52:22.259610  5524 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:22.260749  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:22.145798  5426 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:22.191620  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.193535  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.137305  5335 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:22.141115  5359 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:22.214637  5545 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:52:22.214197  5051 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:22.215969  5375 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:22.216810  4989 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.216840  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.274710  2758 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:22.134276  5417 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 14:52:22.258311  5527 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7.
I20260709 14:52:22.228600  5362 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:22.170464  5530 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:22.284600  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.199783  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:22.248731  5497 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:22.256989  5541 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:22.185293  2752 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:22.261461  3540 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:22.261986  2755 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:22.098716  5482 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:22.263329  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.012s
W20260709 14:52:22.272101  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45873.4us
I20260709 14:52:22.227869  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:22.278944  2679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.020s	sys 0.032s
W20260709 14:52:22.286953  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87445.5us
I20260709 14:52:22.297554  3540 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 8. Ops: []
I20260709 14:52:22.298100  3547 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:22.298967  2679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.020s	sys 0.033s
I20260709 14:52:22.302512  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 465) took 1098 ms. Trace:
I20260709 14:52:22.290969  2605 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:22.303980  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.305631  3540 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 14:52:22.297444  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.011s	sys 0.013s
I20260709 14:52:22.308952  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.284847  5051 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:22.311954  5335 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.288372  5556 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:22.313330  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:22.314896  5416 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.001s
I20260709 14:52:22.315026  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.309613  5359 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:22.307940  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 625636us
I20260709 14:52:22.316653  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.346654  2752 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:22.349323  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:22.351413  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:22.301044  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.316790  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.315493  5416 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.289273  5362 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:22.313395  5556 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:22.303236  2679 rpcz_store.cc:276] 0709 14:52:21.204595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.428426 (+223831us) service_pool.cc:224] Handling call
0709 14:52:21.428938 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.429297 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:21.455311 (+ 26014us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.301984 (+846673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:22.302080 (+    96us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:22.302166 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.302281 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:22.302435 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4087,"prepare.queue_time_us":12395,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":67,"raft.queue_time_us":331793,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":15438,"thread_start_us":835126,"threads_started":2,"wal-append.queue_time_us":643969}
I20260709 14:52:22.316183  5396 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:22.335983  5417 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:22.339674  5007 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:22.341346  5524 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:22.366580  2755 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:22.371130  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.375160  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.299636  5375 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:22.378429  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.391808  5556 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:22.396545  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.396348  5375 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:22.397248  2754 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:22.391888  5499 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:22.398038  5499 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:22.399232  2758 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:22.400276  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.390117  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:22.421609  2758 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:22.376083  5545 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 14:52:22.434975  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.008s	sys 0.010s
W20260709 14:52:22.435497  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.008s	sys 0.010s
I20260709 14:52:22.437004  2758 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:22.434154  2753 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:22.372895  5497 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:22.444286  2752 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:22.392189  5417 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:22.311866  5051 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:22.397289  5396 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:22.451777  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.455430  2755 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:22.455900  2755 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:22.419500  2754 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:22.466329  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:22.474598  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1019) took 1531 ms. Trace:
I20260709 14:52:22.475347  2682 rpcz_store.cc:276] 0709 14:52:20.943589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.194035 (+250446us) service_pool.cc:224] Handling call
0709 14:52:21.212206 (+ 18171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.212519 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:21.212560 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:21.212616 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:21.212654 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:21.212721 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:22.474289 (+1261568us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54002b3c50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:22.474498 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37486976}
I20260709 14:52:22.476977  2758 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 14:52:22.487608  2760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.018s	sys 0.021s
W20260709 14:52:22.488261  2760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.018s	sys 0.021s
W20260709 14:52:22.492647  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.08084e+06us
I20260709 14:52:22.494412  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:22.495234  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.495945  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:22.498404  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.023s	sys 0.015s
W20260709 14:52:22.518771  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.023s	sys 0.015s
W20260709 14:52:22.519414  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 567129us
W20260709 14:52:22.540733  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.558507  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.006s	sys 0.013s
W20260709 14:52:22.559266  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.007s	sys 0.013s
W20260709 14:52:22.560098  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 898050us
W20260709 14:52:22.601078  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79928.6us
I20260709 14:52:22.602037  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:22.603278  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:22.604768  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:22.610689  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:22.451102  5417 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:22.617601  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.618319  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.528115  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:22.619475  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91487.7us
I20260709 14:52:22.620507  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:22.623685  3537 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.040s	sys 0.015s
W20260709 14:52:22.624233  3537 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.040s	sys 0.015s
I20260709 14:52:22.609488  2755 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.656297  2755 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 8
W20260709 14:52:22.669199  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.675559  2606 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.010s	sys 0.044s
W20260709 14:52:22.676225  2606 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.044s
I20260709 14:52:22.609606  5530 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:22.689376  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.690440  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:22.609921  4989 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:22.477038  5335 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.187s
I20260709 14:52:22.623207  5508 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:22.618273  5497 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:22.658357  5362 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.625885  5416 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:22.623448  5542 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:22.624949  5396 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:22.706421  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.001s	sys 0.021s
I20260709 14:52:22.615715  5035 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 14:52:22.623257  5355 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:52:22.720032  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.020s	sys 0.046s
W20260709 14:52:22.726858  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68647.2us
W20260709 14:52:22.756992  5544 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.004s	sys 0.045s
W20260709 14:52:22.792635  5544 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.004s	sys 0.045s
I20260709 14:52:22.703938  5555 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:22.753345  2604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.024s	sys 0.025s
W20260709 14:52:22.815215  2604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.024s	sys 0.025s
W20260709 14:52:22.754506  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.016s	sys 0.033s
W20260709 14:52:22.816501  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.016s	sys 0.033s
W20260709 14:52:22.817592  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 712801us
W20260709 14:52:22.788921  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.020s	sys 0.046s
I20260709 14:52:22.713349  5508 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:22.776875  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.001s	sys 0.022s
W20260709 14:52:22.835765  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.3573e+06us
W20260709 14:52:22.839377  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.840044  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.778028  2751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.022s	sys 0.028s
W20260709 14:52:22.840466  2751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.022s	sys 0.028s
I20260709 14:52:22.839826  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:22.845410  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:22.793295  5037 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.907s	user 0.016s	sys 0.004s
W20260709 14:52:22.826448  2759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.105s	user 0.014s	sys 0.014s
W20260709 14:52:22.849810  2759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.014s	sys 0.015s
I20260709 14:52:22.833837  5542 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:22.826773  3541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.010s	sys 0.007s
I20260709 14:52:22.833323  2755 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:22.846195  2754 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:22.778615  5491 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.001s
W20260709 14:52:22.847123  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.001s	sys 0.014s
W20260709 14:52:22.858384  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.001s	sys 0.014s
W20260709 14:52:22.860915  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:22.826133  2610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.015s	sys 0.002s
W20260709 14:52:22.853222  3541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.011s	sys 0.007s
I20260709 14:52:22.860435  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:22.849464  5541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.016s
I20260709 14:52:22.843274  5508 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:22.842684  5544 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:22.861074  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 821) took 1086 ms. Trace:
W20260709 14:52:22.861619  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.016s	sys 0.002s
I20260709 14:52:22.864892  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:22.858047  5359 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:22.790525  5396 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:22.793838  5416 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 14:52:22.889037  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 807) took 1300 ms. Trace:
I20260709 14:52:22.859298  5037 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:22.864562  3538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.002s	sys 0.013s
W20260709 14:52:22.866792  5334 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.005s
W20260709 14:52:22.861188  2603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.016s	sys 0.016s
I20260709 14:52:22.835815  5555 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.873631  5355 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:22.873274  5565 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:22.864373  5491 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.945849  5355 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:22.940930  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.875977  5550 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:52:22.901852  5404 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.006s	sys 0.017s
W20260709 14:52:22.939322  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.017s	sys 0.002s
W20260709 14:52:22.955848  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.018s	sys 0.002s
W20260709 14:52:22.910481  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.009s	sys 0.023s
W20260709 14:52:22.956940  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.009s	sys 0.025s
W20260709 14:52:22.939043  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.009s	sys 0.014s
I20260709 14:52:22.866086  5482 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.873363  5570 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:22.873191  5541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.000s	sys 0.017s
W20260709 14:52:22.951865  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.014s	sys 0.028s
W20260709 14:52:22.973905  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.015s	sys 0.029s
W20260709 14:52:22.949812  2603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.017s	sys 0.017s
I20260709 14:52:22.875475  5499 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:22.898628  3543 rpcz_store.cc:276] 0709 14:52:21.774482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.774685 (+   203us) service_pool.cc:224] Handling call
0709 14:52:21.796844 (+ 22159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.797267 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:21.811794 (+ 14527us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.436423 (+624629us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.563779 (+127356us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.856160 (+292381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:22.856423 (+   263us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:22.856682 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.857060 (+   378us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:22.860915 (+  3855us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126554,"prepare.queue_time_us":100809,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":5301,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":5892,"spinlock_wait_cycles":13495552,"thread_start_us":915700,"threads_started":2,"wal-append.queue_time_us":1062548}
W20260709 14:52:22.948926  3538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.004s	sys 0.013s
I20260709 14:52:22.898217  5402 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:22.910254  5198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.007s	sys 0.018s
I20260709 14:52:22.962404  3542 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:22.960115  5378 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:22.949292  3541 rpcz_store.cc:276] 0709 14:52:21.588489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.601340 (+ 12851us) service_pool.cc:224] Handling call
0709 14:52:21.601774 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.602059 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:21.602448 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.863650 (+1261202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:22.863774 (+   124us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:22.864209 (+   435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.888745 (+ 24536us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:22.888943 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3978,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":117336,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":4947,"spinlock_wait_cycles":24590592,"thread_start_us":1260691,"threads_started":2,"wal-append.queue_time_us":1093310}
W20260709 14:52:22.960213  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.009s	sys 0.014s
W20260709 14:52:22.959257  5404 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.006s	sys 0.017s
W20260709 14:52:22.982003  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.014s	sys 0.013s
W20260709 14:52:22.994257  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.015s	sys 0.014s
I20260709 14:52:22.962918  5550 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:22.907999  5544 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:22.959331  5577 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:22.851408  5542 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:22.989595  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 949565us
W20260709 14:52:22.977315  5088 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.262s	user 0.015s	sys 0.001s
I20260709 14:52:22.964107  5491 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:22.946193  5565 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:23.002897  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:22.962672  5335 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:22.976464  5570 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:23.014907  5555 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:22.947129  5355 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:23.008107  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:22.988472  5573 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:22.989039  5198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.008s	sys 0.019s
I20260709 14:52:22.979579  5482 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:22.955204  5334 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:22.990516  5541 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:23.024930  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.008s	sys 0.014s
W20260709 14:52:23.074702  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.008s	sys 0.014s
W20260709 14:52:23.043141  5560 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 14:52:23.079627  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:23.084022  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:23.018957  5571 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:23.024605  5552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.005s	sys 0.017s
I20260709 14:52:23.081557  3541 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:23.102448  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:23.104238  3543 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:23.027429  5371 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:23.109760  3536 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:23.063136  5035 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:23.102094  5552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.006s	sys 0.021s
I20260709 14:52:23.113302  5497 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:23.136873  5497 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:23.139590  5567 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 14:52:23.130231  5549 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:23.141595  5549 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:23.144505  5549 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:23.107297  5355 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:23.147774  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:23.152369  5565 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:23.154000  5570 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:23.159088  5396 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:23.070960  5573 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:23.173295  5573 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:23.175011  5544 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:23.091564  5541 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:23.179291  5334 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:23.180138  5560 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 14:52:23.122573  5550 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:23.185384  5035 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:23.107988  5571 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:23.188002  5035 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:23.188371  5571 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:23.196831  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 940) took 2617 ms. Trace:
I20260709 14:52:23.081583  4989 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:23.205149  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:23.205909  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:23.206666  2683 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:23.207412  2678 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:23.209578  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:23.127697  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.010s
W20260709 14:52:23.212220  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.004s	sys 0.011s
W20260709 14:52:23.213203  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 592757us
I20260709 14:52:23.208735  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:23.214421  2682 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:23.128289  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.005s	sys 0.009s
I20260709 14:52:23.214859  2682 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:23.216269  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 952) took 1322 ms. Trace:
I20260709 14:52:23.197439  3538 rpcz_store.cc:276] 0709 14:52:20.579538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.653611 (+ 74073us) service_pool.cc:224] Handling call
0709 14:52:20.654231 (+   620us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:20.992126 (+337895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:20.992182 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:20.993053 (+   871us) log.cc:844] Serialized 52 byte log entry
0709 14:52:22.175084 (+1182031us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b1800341320. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:22.980429 (+805345us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:22.980480 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.980581 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.196400 (+215819us) raft_consensus.cc:1685] finished
0709 14:52:23.196423 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.196749 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":436883,"raft.queue_time_us":329578,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8520,"spinlock_wait_cycles":7525632,"thread_start_us":804016,"threads_started":1,"wal-append.queue_time_us":2170560}
I20260709 14:52:23.212124  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:23.225268  3541 rpcz_store.cc:276] 0709 14:52:21.893861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.174604 (+1280743us) service_pool.cc:224] Handling call
0709 14:52:23.177057 (+  2453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:23.215773 (+ 38716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:23.215818 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:23.215879 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:23.215939 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.216015 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.216181 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.230973  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 822) took 1435 ms. Trace:
I20260709 14:52:23.214497  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:23.008324  5577 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:23.237653  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:23.231670  3537 rpcz_store.cc:276] 0709 14:52:21.795897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.796096 (+   199us) service_pool.cc:224] Handling call
0709 14:52:21.811264 (+ 15168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.811565 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:22.624772 (+813207us) log.cc:844] Serialized 14 byte log entry
0709 14:52:23.230551 (+605779us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:23.230616 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:23.230678 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.230759 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.230906 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15352,"prepare.queue_time_us":86687,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":15534,"thread_start_us":1327378,"threads_started":2}
I20260709 14:52:23.231437  2682 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:23.243741  2682 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:23.251834  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 244311us
I20260709 14:52:23.253530  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:23.252907  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:23.258523  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:23.215345  5411 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:23.262035  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:23.229378  5530 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:23.278755  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:23.260840  5549 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:23.235136  5576 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:23.315572  3541 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:23.261315  5541 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:23.251255  5590 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:23.295158  5088 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:23.321031  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 953) took 1426 ms. Trace:
I20260709 14:52:23.359754  3541 rpcz_store.cc:276] 0709 14:52:21.894133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.253672 (+1359539us) service_pool.cc:224] Handling call
0709 14:52:23.315014 (+ 61342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:23.320609 (+  5595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.320940 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29440}
I20260709 14:52:23.366205  3546 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:23.354966  5569 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:23.430995  5588 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 14:52:23.353838  5415 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:23.378721  5088 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:23.436573  5587 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 14:52:23.442402  5556 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.046s	user 0.008s	sys 0.004s
W20260709 14:52:23.445931  5426 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.169s	user 0.010s	sys 0.005s
I20260709 14:52:23.344538  4989 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:23.445755  5088 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:23.559623  5508 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.003s
W20260709 14:52:23.560012  5598 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.112s	user 0.001s	sys 0.001s
I20260709 14:52:23.542282  5556 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:23.548883  5426 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:23.596745  5577 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:23.617981  5556 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:23.573141  5508 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:23.624513  5508 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:23.748565  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.750474  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.751248  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.753538  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.754096  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.755630  2603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.010s	sys 0.002s
W20260709 14:52:23.755681  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:23.756440  2603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.011s	sys 0.003s
I20260709 14:52:23.785401  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1093) took 2228 ms. Trace:
I20260709 14:52:23.786262  2610 rpcz_store.cc:276] 0709 14:52:21.556844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.558848 (+  2004us) service_pool.cc:224] Handling call
0709 14:52:21.566522 (+  7674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.050860 (+484338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.050896 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.076539 (+ 25643us) log.cc:844] Serialized 26 byte log entry
0709 14:52:22.908666 (+832127us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:22.908723 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.908814 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.784499 (+875685us) raft_consensus.cc:1685] finished
0709 14:52:23.784586 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.785107 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":507578,"raft.queue_time_us":609185,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":11385,"spinlock_wait_cycles":9600,"thread_start_us":741451,"threads_started":1,"wal-append.queue_time_us":925466}
I20260709 14:52:23.790632  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:23.259078  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:23.800349  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1092) took 2244 ms. Trace:
W20260709 14:52:23.809587  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:23.801201  2612 rpcz_store.cc:276] 0709 14:52:21.556424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.556601 (+   177us) service_pool.cc:224] Handling call
0709 14:52:21.557019 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:22.277306 (+720287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.277353 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:22.278047 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 14:52:22.974532 (+696485us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:22.974595 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.974713 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.799552 (+824839us) raft_consensus.cc:1685] finished
0709 14:52:23.799601 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.800058 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":493054,"raft.queue_time_us":609776,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7917,"thread_start_us":870548,"threads_started":2,"wal-append.queue_time_us":945691}
W20260709 14:52:23.812668  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 515458us
I20260709 14:52:23.814363  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:23.817919  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:23.819502  2680 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:23.821125  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:23.822257  2681 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:23.825974  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:23.831326  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:23.831851  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:23.836223  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:23.840286  2685 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:23.852020  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:23.853444  2612 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:23.850728  2682 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.606s	user 0.015s	sys 0.000s
I20260709 14:52:23.866076  2682 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:23.874028  2612 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:23.877413  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 485) took 1171 ms. Trace:
W20260709 14:52:23.878252  2723 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.006s	sys 0.016s
W20260709 14:52:23.878891  2723 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.007s	sys 0.016s
I20260709 14:52:23.878087  2612 rpcz_store.cc:276] 0709 14:52:22.705599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.851421 (+1145822us) service_pool.cc:224] Handling call
0709 14:52:23.877299 (+ 25878us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:23.879688  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 932940us
I20260709 14:52:23.881219  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:23.885862  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:23.895609  2610 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:23.900755  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1105) took 1198 ms (client timeout 1574 ms). Trace:
I20260709 14:52:23.901364  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 482) took 2231 ms. Trace:
W20260709 14:52:23.901470  2610 rpcz_store.cc:269] 0709 14:52:22.702652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.789864 (+1087212us) service_pool.cc:224] Handling call
0709 14:52:23.900619 (+110755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.901996  2751 rpcz_store.cc:276] 0709 14:52:21.670461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.670980 (+   519us) service_pool.cc:224] Handling call
0709 14:52:21.671466 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:22.040346 (+368880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.040396 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:22.055635 (+ 15239us) log.cc:844] Serialized 52 byte log entry
0709 14:52:22.840949 (+785314us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:23.246434 (+405485us) log.cc:844] Serialized 14 byte log entry
0709 14:52:23.247491 (+  1057us) log.cc:844] Serialized 14 byte log entry
0709 14:52:23.285233 (+ 37742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.285366 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.900628 (+615262us) raft_consensus.cc:1685] finished
0709 14:52:23.900663 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.901072 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48236,"prepare.queue_time_us":890663,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3026,"raft.queue_time_us":260641,"raft.run_cpu_time_us":537,"raft.run_wall_time_us":34902,"thread_start_us":1461473,"threads_started":3,"wal-append.queue_time_us":1220612}
I20260709 14:52:23.911245  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:23.916254  2610 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:23.917225  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1109) took 1212 ms. Trace:
I20260709 14:52:23.917869  2610 rpcz_store.cc:276] 0709 14:52:22.705039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.902544 (+1197505us) service_pool.cc:224] Handling call
0709 14:52:23.917100 (+ 14556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.919651  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:23.921073  2610 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:23.935139  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 479) took 2270 ms. Trace:
I20260709 14:52:23.935977  2759 rpcz_store.cc:276] 0709 14:52:21.660095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.686594 (+ 26499us) service_pool.cc:224] Handling call
0709 14:52:21.720784 (+ 34190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.850358 (+1129574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.850401 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.851828 (+  1427us) log.cc:844] Serialized 26 byte log entry
0709 14:52:23.124153 (+272325us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:23.215110 (+ 90957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.215387 (+   277us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.930113 (+714726us) raft_consensus.cc:1685] finished
0709 14:52:23.930211 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.930684 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181078,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":1400187,"threads_started":2,"wal-append.queue_time_us":497881}
W20260709 14:52:23.259539  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.000s	sys 0.015s
W20260709 14:52:23.943282  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.000s	sys 0.015s
W20260709 14:52:23.945695  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.25528e+06us
I20260709 14:52:23.947388  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:23.950358  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:23.953624  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:23.964221  2610 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:23.967346  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1104) took 1091 ms. Trace:
I20260709 14:52:23.969475  2610 rpcz_store.cc:276] 0709 14:52:22.876307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.919018 (+1042711us) service_pool.cc:224] Handling call
0709 14:52:23.967223 (+ 48205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.973961  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:23.977284  2610 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:23.980800  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 487) took 1274 ms. Trace:
I20260709 14:52:23.982697  2610 rpcz_store.cc:276] 0709 14:52:22.706112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.973319 (+1267207us) service_pool.cc:224] Handling call
0709 14:52:23.980670 (+  7351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.988300  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:23.991585  2610 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:23.993724  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1109) took 1052 ms. Trace:
I20260709 14:52:23.995227  2610 rpcz_store.cc:276] 0709 14:52:22.941161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.987582 (+1046421us) service_pool.cc:224] Handling call
0709 14:52:23.993624 (+  6042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.997143  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 825) took 2106 ms. Trace:
I20260709 14:52:23.998144  3541 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:24.000677  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 954) took 2106 ms. Trace:
I20260709 14:52:24.001456  3541 rpcz_store.cc:276] 0709 14:52:21.894431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.361123 (+1466692us) service_pool.cc:224] Handling call
0709 14:52:23.997308 (+636185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.000089 (+  2781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.000539 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:23.459765  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 944) took 1568 ms. Trace:
I20260709 14:52:23.999864  3545 rpcz_store.cc:276] 0709 14:52:21.890132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.262159 (+372027us) service_pool.cc:224] Handling call
0709 14:52:22.262662 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.622505 (+359843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.622556 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.719095 (+ 96539us) log.cc:844] Serialized 26 byte log entry
0709 14:52:23.242274 (+523179us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:23.242337 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.242446 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.996223 (+753777us) raft_consensus.cc:1685] finished
0709 14:52:23.996275 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.996746 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49456,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":752787,"threads_started":2,"wal-append.queue_time_us":665817}
I20260709 14:52:24.003350  3547 rpcz_store.cc:276] 0709 14:52:21.891668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.304179 (+412511us) service_pool.cc:224] Handling call
0709 14:52:22.304690 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.326725 (+ 22035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.326763 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.331932 (+  5169us) log.cc:844] Serialized 26 byte log entry
0709 14:52:22.959061 (+627129us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:22.959110 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.959194 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:23.459365 (+500171us) raft_consensus.cc:1685] finished
0709 14:52:23.459402 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:23.459658 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7732,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"thread_start_us":626637,"threads_started":1,"wal-append.queue_time_us":895403}
I20260709 14:52:23.309564  3543 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:24.008133  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 960) took 1144 ms. Trace:
I20260709 14:52:24.008817  5416 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.008981  3543 rpcz_store.cc:276] 0709 14:52:22.863739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.074348 (+210609us) service_pool.cc:224] Handling call
0709 14:52:24.007779 (+933431us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.010444  5416 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.013244  5222 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:24.016085  5416 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.018738  5508 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.019878  5508 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.021628  5426 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:24.023449  5508 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.024312  5508 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.025213  5508 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.029045  5523 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.032538  5530 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.014555  5222 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.006610  5035 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.034034  5530 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.030265  5523 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:24.131843  5590 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:24.163767  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.081141  5416 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:24.127485  5492 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.008s	sys 0.015s
W20260709 14:52:24.176275  2577 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.338s	user 0.014s	sys 0.015s
W20260709 14:52:24.193753  2577 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.356s	user 0.014s	sys 0.016s
I20260709 14:52:24.193948  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.162622  5590 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:24.205767  5492 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.009s	sys 0.015s
I20260709 14:52:24.163415  5426 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:24.222302  5596 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 14:52:24.222517  5497 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 14:52:24.224193  5416 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:24.226153  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 394618us
I20260709 14:52:24.226816  5530 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.229105  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 813) took 2574 ms. Trace:
I20260709 14:52:24.229276  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:24.229920  3536 rpcz_store.cc:276] 0709 14:52:21.654327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.654625 (+   298us) service_pool.cc:224] Handling call
0709 14:52:21.705046 (+ 50421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:23.112206 (+1407160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.228339 (+1116133us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54004b23d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.228805 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3229769984}
I20260709 14:52:24.231629  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1108) took 1322 ms. Trace:
I20260709 14:52:24.233631  2610 rpcz_store.cc:276] 0709 14:52:22.898974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.134791 (+1235817us) service_pool.cc:224] Handling call
0709 14:52:24.220344 (+ 85553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.226738  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 942) took 2335 ms. Trace:
W20260709 14:52:24.244295  2606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.154s	user 0.015s	sys 0.042s
W20260709 14:52:24.244625  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.40695e+06us
W20260709 14:52:24.244983  2606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.202s	user 0.015s	sys 0.043s
I20260709 14:52:24.245812  5404 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:24.222586  5568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.002s	sys 0.032s
I20260709 14:52:24.247062  5563 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.248354  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1089) took 3490 ms. Trace:
I20260709 14:52:24.248267  5563 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.249759  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:24.250231  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.250617  2685 raft_consensus.cc:2376] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7. Re-sending same reply.
I20260709 14:52:24.252015  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.252729  2681 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:24.254472  5523 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:24.255360  5414 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.256266  5414 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.257026  5508 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.263264  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1096) took 2705 ms. Trace:
I20260709 14:52:24.264045  2611 rpcz_store.cc:276] 0709 14:52:21.557664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.128286 (+570622us) service_pool.cc:224] Handling call
0709 14:52:22.128732 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.142753 (+ 14021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.142791 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.177722 (+ 34931us) log.cc:844] Serialized 26 byte log entry
0709 14:52:22.956449 (+778727us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:22.956512 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:22.956634 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.262557 (+1305923us) raft_consensus.cc:1685] finished
0709 14:52:24.262615 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.262979 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87236,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":718580,"threads_started":1,"wal-append.queue_time_us":1987269}
I20260709 14:52:24.272710  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 477) took 2613 ms. Trace:
W20260709 14:52:24.272877  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43888.8us
I20260709 14:52:24.273501  5414 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:24.274712  5527 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:24.274941  5550 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.271369  5405 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:24.275871  5550 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.276332  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1091) took 2723 ms. Trace:
I20260709 14:52:24.277432  5222 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:24.247244  5568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.003s	sys 0.032s
I20260709 14:52:24.246717  5526 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:24.274410  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.266819  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:24.281791  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.200273  5035 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.273413  2760 rpcz_store.cc:276] 0709 14:52:21.659320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.686699 (+ 27379us) service_pool.cc:224] Handling call
0709 14:52:21.687072 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.489601 (+802529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.489648 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.603825 (+114177us) log.cc:844] Serialized 26 byte log entry
0709 14:52:23.027122 (+423297us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:23.027186 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.027321 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.272085 (+1244764us) raft_consensus.cc:1685] finished
0709 14:52:24.272132 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.272403 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":213838,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":1122439,"threads_started":2,"wal-append.queue_time_us":1593922}
W20260709 14:52:24.289484  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 336210us
I20260709 14:52:24.293298  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:24.243060  3540 rpcz_store.cc:276] 0709 14:52:21.891068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.300166 (+409098us) service_pool.cc:224] Handling call
0709 14:52:22.305183 (+  5017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:22.306436 (+  1253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.226279 (+1919843us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b54002017d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:24.226530 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10606336}
I20260709 14:52:24.289724  5590 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.302846  3541 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:24.303308  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.303689  5623 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:24.305094  5614 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:24.304802  5623 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:24.306346  5508 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.302533  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.307219  5404 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:24.307367  5508 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.308475  5404 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:24.278083  5380 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:24.309717  5380 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:24.309732  5007 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:24.311446  5380 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.311908  5405 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:24.312741  5405 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:24.312999  5380 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:24.314409  5417 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.649s	user 0.017s	sys 0.006s
I20260709 14:52:24.303730  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 829) took 2205 ms. Trace:
I20260709 14:52:24.318429  3541 rpcz_store.cc:276] 0709 14:52:22.098109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.238255 (+2140146us) service_pool.cc:224] Handling call
0709 14:52:24.302262 (+ 64007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.303372 (+  1110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.303627 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.315407  5620 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:24.285790  5549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.024s	sys 0.028s
I20260709 14:52:24.295703  5035 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.316111  5508 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.315056  5417 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.314275  5405 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:24.330340  5549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.024s	sys 0.029s
I20260709 14:52:24.304544  5590 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.342690  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:24.277347  2603 rpcz_store.cc:276] 0709 14:52:21.553713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.553912 (+   199us) service_pool.cc:224] Handling call
0709 14:52:21.554400 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.554745 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:21.568766 (+ 14021us) log.cc:844] Serialized 14 byte log entry
0709 14:52:23.757266 (+2188500us) log.cc:844] Serialized 14 byte log entry
0709 14:52:24.212768 (+455502us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:24.221860 (+  9092us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:24.242128 (+ 20268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.275964 (+ 33836us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.276251 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":377296,"prepare.queue_time_us":391499,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":104,"raft.queue_time_us":456481,"raft.run_cpu_time_us":585,"raft.run_wall_time_us":34582,"thread_start_us":2257106,"threads_started":4,"wal-append.queue_time_us":965231,"wal-append.run_cpu_time_us":1430,"wal-append.run_wall_time_us":1012656}
I20260709 14:52:24.330276  5568 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:24.343436  2611 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:24.357702  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.359344  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1105) took 1482 ms. Trace:
I20260709 14:52:24.360003  2611 rpcz_store.cc:276] 0709 14:52:22.876679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.266321 (+1389642us) service_pool.cc:224] Handling call
0709 14:52:24.359231 (+ 92910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.361867  2610 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:24.363189  5568 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.364701  5623 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:24.367305  5620 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.368057  5563 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:24.368559  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.368745  3541 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:24.369757  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 958) took 1506 ms. Trace:
I20260709 14:52:24.370134  3541 rpcz_store.cc:276] 0709 14:52:22.863167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.334036 (+1470869us) service_pool.cc:224] Handling call
0709 14:52:24.366422 (+ 32386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.369377 (+  2955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.369637 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.371410  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.372519  2680 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 14:52:24.373293  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.374984  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.375439  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.375861  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.379196  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.380302  2681 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:24.382517  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.383147  5417 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.385478  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1099) took 2826 ms. Trace:
I20260709 14:52:24.385867  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.387089  2680 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:24.386018  2608 rpcz_store.cc:276] 0709 14:52:21.558649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.171655 (+613006us) service_pool.cc:224] Handling call
0709 14:52:22.172073 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.176568 (+  4495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.176621 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.179950 (+  3329us) log.cc:844] Serialized 26 byte log entry
0709 14:52:23.075268 (+895318us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:23.075320 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:23.075407 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.384973 (+1309566us) raft_consensus.cc:1685] finished
0709 14:52:24.385019 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.385227 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3952,"raft.queue_time_us":44138,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":44,"thread_start_us":837798,"threads_started":1,"wal-append.queue_time_us":2107173}
I20260709 14:52:24.346544  5508 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.390071  5508 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.391908  2680 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
W20260709 14:52:24.395457  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.398869  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.399338  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:24.385317  5417 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:24.401464  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.402073  3545 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 14:52:24.402092  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 66495.5us
W20260709 14:52:24.405387  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86638.8us
I20260709 14:52:24.415252  2679 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:24.415429  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 299740us
I20260709 14:52:24.418493  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 831) took 2308 ms. Trace:
I20260709 14:52:24.419253  3545 rpcz_store.cc:276] 0709 14:52:22.117548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.401115 (+2283567us) service_pool.cc:224] Handling call
0709 14:52:24.401633 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.402658 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.407216 (+  4558us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.421060  2610 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:24.424561  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1108) took 1719 ms. Trace:
I20260709 14:52:24.425369  2610 rpcz_store.cc:276] 0709 14:52:22.704741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.316327 (+1611586us) service_pool.cc:224] Handling call
0709 14:52:24.423779 (+107452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.379312  5550 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:24.439226  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.440095  3536 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
W20260709 14:52:24.444221  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.379410  5578 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.450222  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.452044  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 956) took 1583 ms. Trace:
I20260709 14:52:24.453001  3536 rpcz_store.cc:276] 0709 14:52:22.862615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.283628 (+1421013us) service_pool.cc:224] Handling call
0709 14:52:24.352051 (+ 68423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.445911 (+ 93860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.446316 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.425037  5530 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:24.455041  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:24.460340  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57222.4us
I20260709 14:52:24.460441  2610 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:24.461228  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.455291  2685 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:24.462795  2685 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:24.463835  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.464293  2685 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.464906  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:24.465477  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.466004  2685 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:24.466518  5417 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:24.467016  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.467612  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.468055  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1102) took 1592 ms. Trace:
I20260709 14:52:24.468679  2686 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:24.469296  5474 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:24.469837  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:24.468083  2677 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:24.468506  2611 rpcz_store.cc:276] 0709 14:52:22.875751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.436072 (+1560321us) service_pool.cc:224] Handling call
0709 14:52:24.467987 (+ 31915us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.471071  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.472338  2679 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:24.473526  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.477934  2679 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:24.488062  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.480195  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:24.491101  5355 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.346s	user 0.006s	sys 0.019s
I20260709 14:52:24.491292  5238 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 14:52:24.492254  2612 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:24.492865  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.493690  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.494457  2611 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
I20260709 14:52:24.495329  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.496129  2686 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:24.497484  5589 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:24.498454  5589 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:24.501516  5589 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:24.502305  5551 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.719s	user 0.004s	sys 0.017s
W20260709 14:52:24.504068  5088 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.942s	user 0.011s	sys 0.005s
I20260709 14:52:24.504945  5414 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:24.505641  5334 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.506022  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.506214  5414 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:24.498951  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 21) took 3535 ms. Trace:
I20260709 14:52:24.483142  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:24.507685  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:24.511118  5583 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.095s	user 0.001s	sys 0.001s
W20260709 14:52:24.513402  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.508188  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 486) took 1802 ms. Trace:
I20260709 14:52:24.515372  2611 rpcz_store.cc:276] 0709 14:52:22.705855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.476605 (+1770750us) service_pool.cc:224] Handling call
0709 14:52:24.508128 (+ 31523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.507339  2756 rpcz_store.cc:276] 0709 14:52:20.963097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:20.963308 (+   211us) service_pool.cc:224] Handling call
0709 14:52:22.472249 (+1508941us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b54001ef120. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c3520 00007fa16e640652 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.498374 (+2026125us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b4c001d4740. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16c912a20 00007fa16c923a29 00007fa16c8e9644 00007fa16c910f2d 00007fa16c910b93 00007fa16e640c92 00007fa16e64084a 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691
0709 14:52:24.498627 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320638080}
I20260709 14:52:24.477788  5474 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:24.524838  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63993.6us
I20260709 14:52:24.506532  5334 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.528461  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:24.530615  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 959) took 1667 ms. Trace:
I20260709 14:52:24.525858  3545 pending_rounds.cc:85] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 1
I20260709 14:52:24.487324  5590 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 14:52:24.536737  2759 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:24.478961  5578 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.517997  5426 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:24.520471  5614 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:24.540616  5563 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:24.520805  5596 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.541188  5474 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.542704  5578 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:24.532279  3541 rpcz_store.cc:276] 0709 14:52:22.863441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.387304 (+1523863us) service_pool.cc:224] Handling call
0709 14:52:24.388027 (+   723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.388450 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:24.388885 (+   435us) log.cc:844] Serialized 14 byte log entry
0709 14:52:24.530165 (+141280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:24.530259 (+    94us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:24.530316 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.530389 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.530543 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":12090,"thread_start_us":130752,"threads_started":1}
I20260709 14:52:24.562024  5596 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.563338  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:24.564193  3541 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:24.565238  5596 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.566249  5620 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.545496  5334 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.546200  5222 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:24.537073  2760 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:24.539510  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.529713  5355 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:24.532408  3545 pending_rounds.cc:107] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 14:52:24.547209  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.555483  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.560926  5238 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:24.553635  5614 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:24.595062  3545 raft_consensus.cc:2889] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:24.570456  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.571668  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.533836  5414 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:24.573269  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.510529  5088 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.606190  5088 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:24.607311  4786 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:24.544400  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:24.591542  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44504.9us
I20260709 14:52:24.576625  5238 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.599849  2677 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:24.570238  5530 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 14:52:24.590613  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1094) took 3033 ms. Trace:
I20260709 14:52:24.578593  5426 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.602370  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:24.580539  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 206710us
I20260709 14:52:24.612083  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.608640  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.613566  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.622036  2679 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:24.623039  5620 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:24.623605  5620 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.624022  5620 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:24.616544  2607 rpcz_store.cc:276] 0709 14:52:21.557128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.106729 (+549601us) service_pool.cc:224] Handling call
0709 14:52:22.136230 (+ 29501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:22.220227 (+ 83997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:22.220272 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:22.220865 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 14:52:22.995008 (+774143us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:22.995081 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.579491 (+1584410us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.579591 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.579666 (+    75us) raft_consensus.cc:1685] finished
0709 14:52:24.579686 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.590304 (+ 10618us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16856,"raft.queue_time_us":29070,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":106,"spinlock_wait_cycles":152765184,"thread_start_us":703584,"threads_started":1,"wal-append.queue_time_us":2025070}
I20260709 14:52:24.630239  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:24.578004  5222 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:24.631193  5506 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:24.632035  2610 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:52:24.632537  5614 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.637784  2685 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:24.575711  5355 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:24.641090  2683 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:24.641336  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:24.657795  2610 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:24.658149  5355 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:24.642493  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.635933  5222 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:24.615128  5596 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.664162  5596 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.664716  5222 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.666606  5222 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:24.667454  5596 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.671721  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.644395  5506 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:24.676743  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:24.780793  2611 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:24.786513  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1103) took 1910 ms (client timeout 1798 ms). Trace:
W20260709 14:52:24.689934  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 467) took 3113 ms (client timeout 1710 ms). Trace:
W20260709 14:52:24.788374  2605 rpcz_store.cc:269] 0709 14:52:21.576864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.660935 (+ 84071us) service_pool.cc:224] Handling call
0709 14:52:24.689465 (+3028530us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:24.689691 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446381952}
I20260709 14:52:24.791179  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.683456  3541 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:24.792408  3538 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:24.793733  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:24.787144  2612 rpcz_store.cc:269] 0709 14:52:22.876036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:23.880495 (+1004459us) service_pool.cc:224] Handling call
0709 14:52:24.786213 (+905718us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540016dfd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:24.786392 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402387712}
I20260709 14:52:24.684101  2686 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 14:52:24.748489  2610 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:24.249020  2606 rpcz_store.cc:276] 0709 14:52:20.758175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.008639 (+250464us) service_pool.cc:224] Handling call
0709 14:52:21.008927 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:21.009144 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:21.352481 (+343337us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.000417 (+647936us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800380f20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:23.042539 (+1042122us) log.cc:844] Serialized 14 byte log entry
0709 14:52:24.245849 (+1203310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:24.245988 (+   139us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:24.246164 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.247947 (+  1783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.248283 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16499,"prepare.queue_time_us":997473,"prepare.run_cpu_time_us":1710,"prepare.run_wall_time_us":17044,"raft.queue_time_us":683566,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":45182,"spinlock_wait_cycles":23864704,"thread_start_us":2556233,"threads_started":5,"wal-append.queue_time_us":1370544,"wal-append.run_cpu_time_us":1574,"wal-append.run_wall_time_us":1091436}
I20260709 14:52:24.684736  5371 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 14:52:24.677939  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:24.691159  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.799746  2677 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:24.729045  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1111) took 1626 ms. Trace:
I20260709 14:52:24.800988  2607 rpcz_store.cc:276] 0709 14:52:23.102698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.629637 (+1526939us) service_pool.cc:224] Handling call
0709 14:52:24.728950 (+ 99313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.733772  2757 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 14:52:24.795279  3541 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:24.803130  5426 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.802974  2612 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 14:52:24.804592  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1102) took 3244 ms. Trace:
I20260709 14:52:24.804972  2612 rpcz_store.cc:276] 0709 14:52:21.559782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.795563 (+3235781us) service_pool.cc:224] Handling call
0709 14:52:24.802470 (+  6907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.804236 (+  1766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.804487 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.806244  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:24.806715  2612 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 8. Ops: []
I20260709 14:52:24.690989  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 947) took 2798 ms. Trace:
I20260709 14:52:24.688802  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:24.687278  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 946) took 2794 ms. Trace:
I20260709 14:52:24.809659  3547 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:24.808976  2611 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:24.675185  5530 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:24.810770  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 470) took 3217 ms. Trace:
I20260709 14:52:24.809306  2605 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:52:24.814369  2612 rpcz_store.cc:276] 0709 14:52:21.590403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.805823 (+3215420us) service_pool.cc:224] Handling call
0709 14:52:24.806301 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.807508 (+  1207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.807800 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:24.813149  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81811.1us
W20260709 14:52:24.811374  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 267165us
I20260709 14:52:24.808139  3546 rpcz_store.cc:276] 0709 14:52:21.892552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.824956 (+932404us) service_pool.cc:224] Handling call
0709 14:52:22.858602 (+ 33646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:23.424145 (+565543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.690417 (+1266272us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.690752 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166558080}
I20260709 14:52:24.817111  5526 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:24.817834  5526 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:24.818274  5526 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:24.818498  5578 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:24.811430  5355 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:24.821964  5355 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:24.824353  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 962) took 1960 ms. Trace:
I20260709 14:52:24.824968  3545 rpcz_store.cc:276] 0709 14:52:22.903260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.431662 (+1528402us) service_pool.cc:224] Handling call
0709 14:52:24.473764 (+ 42102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.607755 (+133991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:24.607815 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:24.608453 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 14:52:24.659532 (+ 51079us) raft_consensus.cc:1645] Marking committed up to 1
0709 14:52:24.659581 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.659678 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.824049 (+164371us) raft_consensus.cc:1685] finished
0709 14:52:24.824076 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.824248 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":37412,"threads_started":1,"wal-append.queue_time_us":212208}
I20260709 14:52:24.827607  2607 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:24.827850  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.684381  5378 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:24.828586  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 469) took 3238 ms. Trace:
I20260709 14:52:24.829380  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.829788  2607 rpcz_store.cc:276] 0709 14:52:21.590012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.801757 (+3211745us) service_pool.cc:224] Handling call
0709 14:52:24.827347 (+ 25590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.828260 (+   913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.828497 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.830449  3541 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:24.800571  5614 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:24.827665  2612 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:24.809927  3542 rpcz_store.cc:276] 0709 14:52:21.892298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.402447 (+510149us) service_pool.cc:224] Handling call
0709 14:52:22.444178 (+ 41731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:22.992020 (+547842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.686554 (+1694534us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b5400200dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.686997 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191195520}
I20260709 14:52:24.830794  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1101) took 3255 ms. Trace:
I20260709 14:52:24.836212  2605 rpcz_store.cc:276] 0709 14:52:21.559397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.790093 (+3230696us) service_pool.cc:224] Handling call
0709 14:52:24.791067 (+   974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.814842 (+ 23775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.815064 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.643834  5527 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 14:52:24.829331  5378 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:24.840310  2610 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:24.840483  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 169042us
I20260709 14:52:24.839394  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 476) took 3246 ms. Trace:
I20260709 14:52:24.837556  3543 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:24.850965  2612 rpcz_store.cc:276] 0709 14:52:21.592926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.826928 (+3234002us) service_pool.cc:224] Handling call
0709 14:52:24.827428 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.836341 (+  8913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.839293 (+  2952us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.851044  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:24.857595  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:24.855926  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1103) took 3291 ms. Trace:
I20260709 14:52:24.863575  2610 rpcz_store.cc:276] 0709 14:52:21.560202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.809060 (+3248858us) service_pool.cc:224] Handling call
0709 14:52:24.835800 (+ 26740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.850670 (+ 14870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.851124 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.862124  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.871616  2611 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [5.3-8.4]
I20260709 14:52:24.843132  5530 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.873188  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 475) took 3280 ms. Trace:
I20260709 14:52:24.874277  2611 rpcz_store.cc:276] 0709 14:52:21.592575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.811779 (+3219204us) service_pool.cc:224] Handling call
0709 14:52:24.851862 (+ 40083us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:24.872823 (+ 20961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.873116 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.881743  5355 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:24.886061  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:24.887590  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.887827  5506 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.889068  2603 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:24.889160  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:24.889521  5506 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:24.889973  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1100) took 2014 ms. Trace:
I20260709 14:52:24.892198  5526 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:24.888535  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.890359  2603 rpcz_store.cc:276] 0709 14:52:22.875140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.888156 (+2013016us) service_pool.cc:224] Handling call
0709 14:52:24.888589 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.889654 (+  1065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.889897 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.890662  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1094) took 4058 ms. Trace:
I20260709 14:52:24.896092  3538 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:24.902865  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:24.901124  5542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.000s	sys 0.035s
W20260709 14:52:24.895743  5547 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 2.737s	user 0.006s	sys 0.025s
I20260709 14:52:24.899926  3541 pending_rounds.cc:85] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 3
I20260709 14:52:24.908191  3541 pending_rounds.cc:107] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 14:52:24.908883  3541 raft_consensus.cc:2889] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:24.905288  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.909483  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.911269  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:24.906212  5542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.300s	user 0.001s	sys 0.035s
I20260709 14:52:24.898398  2604 rpcz_store.cc:276] 0709 14:52:20.832366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.115431 (+283065us) service_pool.cc:224] Handling call
0709 14:52:21.180821 (+ 65390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:21.298318 (+117497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:21.298370 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:21.303946 (+  5576us) log.cc:844] Serialized 26 byte log entry
0709 14:52:21.733762 (+429816us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:21.791176 (+ 57414us) log.cc:844] Serialized 14 byte log entry
0709 14:52:22.054396 (+263220us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b180008da60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:22.816041 (+761645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.890079 (+2074038us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400371c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:24.890151 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:24.890219 (+    68us) raft_consensus.cc:1685] finished
0709 14:52:24.890235 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:24.890443 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320486,"prepare.queue_time_us":59597,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":5498,"raft.queue_time_us":405131,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":364100096,"thread_start_us":1099369,"threads_started":2,"wal-append.queue_time_us":593648,"wal-append.run_cpu_time_us":5249,"wal-append.run_wall_time_us":1561989}
I20260709 14:52:24.913350  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.913650  2609 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:24.914410  2604 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:24.915251  2606 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:24.916471  3547 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:24.919003  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:24.919806  3547 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:24.920939  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:24.921501  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.921756  3547 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:24.922049  5550 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:24.922664  5550 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:24.923897  5550 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:24.925611  2724 connection.cc:256] RPC call timeout handler was delayed by -2.3546608519999381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 14:52:24.926338  5380 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:24.926888  5380 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:24.928017  5380 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:24.929594  5380 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:24.929811  5405 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:24.932317  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1104) took 2213 ms. Trace:
I20260709 14:52:24.934777  2606 rpcz_store.cc:276] 0709 14:52:22.702262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.846931 (+2144669us) service_pool.cc:224] Handling call
0709 14:52:24.904811 (+ 57880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.915684 (+ 10873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.915885 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.940037  5599 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:24.940696  5599 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:24.941812  5599 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:24.944028  5599 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:24.948064  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:24.931689  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1107) took 2038 ms. Trace:
I20260709 14:52:24.949944  2604 rpcz_store.cc:276] 0709 14:52:22.877219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.913451 (+2036232us) service_pool.cc:224] Handling call
0709 14:52:24.913958 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.915459 (+  1501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.915704 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:24.957219  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53689.1us
W20260709 14:52:24.959645  2726 connection.cc:256] RPC call timeout handler was delayed by -1.678848619000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 14:52:24.961139  3546 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:24.925598  5550 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:24.971429  3537 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:24.981617  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:24.984805  2603 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:24.986131  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1101) took 2110 ms. Trace:
I20260709 14:52:24.986654  2603 rpcz_store.cc:276] 0709 14:52:22.875469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.897122 (+2021653us) service_pool.cc:224] Handling call
0709 14:52:24.917526 (+ 20404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.985621 (+ 68095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:24.986038 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:24.994061  3546 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:25.000826  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1802670140000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 14:52:24.973879  5404 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:25.019380  5527 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:25.019363  5238 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:25.023124  2603 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:25.025565  2611 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 14:52:25.026361  2611 pending_rounds.cc:85] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 3
I20260709 14:52:25.026693  2611 pending_rounds.cc:107] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 14:52:25.027071  2611 raft_consensus.cc:2889] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:25.003460  5222 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.030452  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:25.031602  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 159429us
I20260709 14:52:25.021088  5492 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:25.032780  2724 connection.cc:256] RPC call timeout handler was delayed by -1.1845678959999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 14:52:25.033718  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52299.2us
I20260709 14:52:25.014439  5371 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.035416  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1110) took 2093 ms. Trace:
I20260709 14:52:25.035943  2604 rpcz_store.cc:276] 0709 14:52:22.941638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.951560 (+2009922us) service_pool.cc:224] Handling call
0709 14:52:24.994077 (+ 42517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.034959 (+ 40882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.034999 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.035080 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:25.035127 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.035192 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.035341 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:25.038388  5511 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.038959  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:25.039316  5511 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.038432  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1110) took 1993 ms. Trace:
I20260709 14:52:25.040613  2606 rpcz_store.cc:276] 0709 14:52:23.044642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.982077 (+1937435us) service_pool.cc:224] Handling call
0709 14:52:25.027480 (+ 45403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.037983 (+ 10503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.038015 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.038092 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:25.038138 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.038209 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.038352 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:25.039542  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:25.041957  5511 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:25.043393  2760 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:25.049430  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.050066  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:25.050416  2608 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:25.045162  5411 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 14:52:25.052517  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:25.055902  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.058138  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:25.058686  5378 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:25.063500  5378 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.064009  5411 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:25.049612  4786 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:25.065292  4786 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:25.065804  4786 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:25.066726  5568 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.067408  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 957) took 2204 ms. Trace:
W20260709 14:52:25.066718  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.069308  5088 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:25.071069  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.071943  5596 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:25.054435  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:25.058629  2760 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:25.070010  5088 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:25.073930  5623 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.093494  5623 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:25.096817  5497 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.097712  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.099324  4989 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.100203  4989 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:25.105567  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.100232  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:25.108212  5416 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.108834  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:25.109745  5426 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:25.110437  5426 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:25.110700  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:25.109108  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:25.109835  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 220852us
I20260709 14:52:25.042596  5563 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:25.097862  5497 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.122944  3545 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:25.131462  5378 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:25.132249  5378 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.132745  5378 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:25.135390  5527 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:25.135102  5568 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:25.135953  5527 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:25.136257  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.137389  2754 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:25.131502  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77184.2us
I20260709 14:52:25.138085  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:25.139117  3538 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:25.131798  5411 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:25.136483  5527 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:25.143458  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52615.3us
I20260709 14:52:25.149688  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:25.156643  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 484) took 2441 ms. Trace:
I20260709 14:52:25.157470  2611 rpcz_store.cc:276] 0709 14:52:22.705347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.875072 (+2169725us) service_pool.cc:224] Handling call
0709 14:52:24.961314 (+ 86242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.131058 (+169744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.146274 (+ 15216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:25.121737  5563 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:25.115793  5623 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.067871  3540 rpcz_store.cc:276] 0709 14:52:22.862910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.316951 (+1454041us) service_pool.cc:224] Handling call
0709 14:52:24.356537 (+ 39586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.461052 (+104515us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:24.483808 (+ 22756us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.066980 (+583172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.067050 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:25.067113 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.067197 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.067341 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":427049,"prepare.queue_time_us":12537,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":428992,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":134,"thread_start_us":144038,"threads_started":1,"wal-append.queue_time_us":321254}
W20260709 14:52:25.171633  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 74204.9us
I20260709 14:52:25.047164  5492 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.089679  5222 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:25.144341  5411 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:25.183987  5596 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:25.187680  5596 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:25.188388  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.190516  5596 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:25.200402  3538 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:25.196053  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 479) took 3602 ms. Trace:
I20260709 14:52:25.165834  5511 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:25.203423  2605 rpcz_store.cc:276] 0709 14:52:21.593912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.850141 (+3256229us) service_pool.cc:224] Handling call
0709 14:52:24.866935 (+ 16794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.903639 (+ 36704us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:24.903672 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:24.915983 (+ 12311us) log.cc:844] Serialized 26 byte log entry
0709 14:52:25.113612 (+197629us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:25.113653 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.113732 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:25.195708 (+ 81976us) raft_consensus.cc:1685] finished
0709 14:52:25.195742 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.195967 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":133517,"threads_started":1,"wal-append.queue_time_us":222120}
I20260709 14:52:25.165850  5563 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:25.207332  5599 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.212476  5402 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:25.215070  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.142741  5620 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.195956  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:25.140808  5371 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.231506  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:25.235086  5599 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.235771  4989 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.185597  5411 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:25.237872  5599 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.239470  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.240304  2754 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 3.
I20260709 14:52:25.240712  5222 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:25.242123  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53948.1us
I20260709 14:52:25.243276  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.244901  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.248562  5576 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:25.255271  5504 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:25.257371  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.258490  2752 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:25.259384  2758 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:25.260355  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.268559  5088 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:25.271365  5492 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.271579  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.273032  2754 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:25.274919  5530 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.276258  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:25.277531  2677 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:25.278075  5497 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.280035  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:25.281885  5504 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:25.282442  5504 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:25.283134  5504 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:25.285414  4786 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.287565  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.246474  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.292057  5402 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:25.298054  3543 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:25.298344  5402 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.299626  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.300262  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 478) took 3640 ms. Trace:
I20260709 14:52:25.292178  5623 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:25.309989  5623 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:25.313756  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67374.1us
I20260709 14:52:25.300693  2757 rpcz_store.cc:276] 0709 14:52:21.659767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:21.659980 (+   213us) service_pool.cc:224] Handling call
0709 14:52:21.662062 (+  2082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.811090 (+3149028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.299839 (+488749us) spinlock_profiling.cc:243] Waited 3.04 s on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:25.300194 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6994686720}
I20260709 14:52:25.145768  5426 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:25.282285  5378 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:25.319782  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.373476  2604 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: []
W20260709 14:52:25.377022  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58999.1us
I20260709 14:52:25.388204  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1106) took 2678 ms. Trace:
I20260709 14:52:25.395502  2758 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:25.390616  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:25.408097  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:25.408660  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87983.8us
I20260709 14:52:25.403439  2755 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:25.409907  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:25.414007  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.415230  2754 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:25.404515  5506 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.514s	user 0.017s	sys 0.000s
W20260709 14:52:25.438723  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 244760us
I20260709 14:52:25.434341  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:25.442690  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:25.451036  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70113.6us
I20260709 14:52:25.424510  5404 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:25.423923  5623 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.435146  5474 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:25.437583  5380 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.459880  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.456005  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:25.465483  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 87148.5us
I20260709 14:52:25.471582  2756 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:25.482980  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.448483  5599 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:25.532852  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63729.6us
W20260709 14:52:25.539067  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 88220.5us
I20260709 14:52:25.543838  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:25.448159  5492 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:25.457896  5404 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:25.466542  5596 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:25.551757  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.566430  5371 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:25.574508  2758 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:25.469218  5411 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:25.577922  5411 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:25.581790  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:25.582748  2684 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:25.584558  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:25.587220  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1107) took 2883 ms. Trace:
I20260709 14:52:25.587529  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:25.588660  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47582.8us
I20260709 14:52:25.589501  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.591127  5506 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:25.591358  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.591845  5599 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.592332  2756 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:25.593786  2756 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:25.465147  5474 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:25.587838  5642 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 14:52:25.594282  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:25.547214  5614 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.009s
I20260709 14:52:25.565119  2753 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:25.557202  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 477) took 3964 ms. Trace:
I20260709 14:52:25.599834  5474 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:25.565267  5637 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 14:52:25.540834  5380 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.603979  2681 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
W20260709 14:52:25.609155  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151589us
W20260709 14:52:25.554721  5550 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.001s
I20260709 14:52:25.587899  2610 rpcz_store.cc:276] 0709 14:52:22.704249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.865908 (+2161659us) service_pool.cc:224] Handling call
0709 14:52:24.903469 (+ 37561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.932237 (+ 28768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:24.932278 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:24.959580 (+ 27302us) log.cc:844] Serialized 26 byte log entry
0709 14:52:25.432568 (+472988us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:25.432631 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.432745 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:25.586872 (+154127us) raft_consensus.cc:1685] finished
0709 14:52:25.586895 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.587113 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16737,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":105,"thread_start_us":413318,"threads_started":1,"wal-append.queue_time_us":595002}
I20260709 14:52:25.591428  5492 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:25.608163  5596 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:25.551661  4989 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:25.565840  5359 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:25.563308  5404 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:25.630872  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.623991  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 973) took 1115 ms. Trace:
I20260709 14:52:25.613655  2758 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:25.605790  2607 rpcz_store.cc:276] 0709 14:52:21.593256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.831037 (+3237781us) service_pool.cc:224] Handling call
0709 14:52:24.843580 (+ 12543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.895779 (+ 52199us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:25.029474 (+133695us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.556340 (+526866us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.556607 (+   267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.556679 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:25.556766 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.556877 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.557097 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":212822,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":11207,"raft.queue_time_us":15190,"raft.run_wall_time_us":17865,"spinlock_wait_cycles":101274112,"thread_start_us":485864,"threads_started":2}
W20260709 14:52:25.614639  5644 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 14:52:25.660969  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 286707us
I20260709 14:52:25.673198  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 974) took 1037 ms. Trace:
I20260709 14:52:25.630424  5405 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.631510  5380 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.623912  5620 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.616452  5599 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:25.614099  4786 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.632233  5222 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:25.707914  5222 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:25.708779  5222 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:25.650559  3546 rpcz_store.cc:276] 0709 14:52:24.508921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.036220 (+527299us) service_pool.cc:224] Handling call
0709 14:52:25.063339 (+ 27119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:25.126027 (+ 62688us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:25.138810 (+ 12783us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.440651 (+301841us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:25.466435 (+ 25784us) log.cc:844] Serialized 26 byte log entry
0709 14:52:25.466702 (+   267us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:25.466773 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.466915 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:25.623728 (+156813us) raft_consensus.cc:1685] finished
0709 14:52:25.623753 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.623911 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65866,"prepare.queue_time_us":1530,"raft.queue_time_us":68571,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":27491,"spinlock_wait_cycles":24962944,"thread_start_us":299051,"threads_started":1,"wal-append.queue_time_us":435984}
I20260709 14:52:25.711406  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.712842  2758 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:25.715081  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:25.716483  3542 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:25.716614  5508 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:25.683647  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 92865.1us
I20260709 14:52:25.720093  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.723410  4786 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:25.724156  4786 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:25.725332  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1027) took 1335 ms. Trace:
I20260709 14:52:25.621430  5563 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.725850  2681 rpcz_store.cc:276] 0709 14:52:24.390169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.398948 (+  8779us) service_pool.cc:224] Handling call
0709 14:52:25.725071 (+1326123us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:25.725228 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378914048}
W20260709 14:52:25.729509  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.729753  5563 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:25.730203  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:25.623302  5198 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 14:52:25.631978  5651 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:25.689543  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:25.692574  2756 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:25.673981  3545 rpcz_store.cc:276] 0709 14:52:24.636742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.130096 (+493354us) service_pool.cc:224] Handling call
0709 14:52:25.154179 (+ 24083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.203967 (+ 49788us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:25.411992 (+208025us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.672562 (+260570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.672669 (+   107us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:25.672752 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.672870 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.673091 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":360783,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":188,"thread_start_us":347215,"threads_started":2}
I20260709 14:52:25.677101  5576 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:25.627202  5614 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 14:52:25.656752  4989 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:25.704252  5527 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.004s
I20260709 14:52:25.621855  5496 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 14:52:25.722203  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:25.739607  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45702.2us
I20260709 14:52:25.742142  5198 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.744936  5550 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.761152  5414 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:25.684770  5405 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.623044  5623 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 14:52:25.630121  5504 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.678715  5371 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:25.704308  5647 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 14:52:25.725828  4786 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.737284  5651 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:25.842891  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.848062  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:25.857915  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 135853us
I20260709 14:52:25.851317  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:25.851599  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:25.859201  2604 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:25.854297  2608 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:25.849804  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.858732  2607 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:25.864133  2605 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:25.723964  5596 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.870416  2751 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:25.838975  5411 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:25.874430  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.875088  5411 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.875494  2757 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:25.875665  5411 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:25.879544  5577 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:25.882072  5504 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.882783  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:25.867048  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.883685  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:25.883265  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:25.889420  3540 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:25.809917  5527 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:25.900547  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 140425us
I20260709 14:52:25.883267  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.864588  5651 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:25.910077  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:25.939245  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:25.914332  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.858341  5380 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 14:52:25.920128  5662 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:52:25.920733  5415 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:25.921932  5405 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:25.922482  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:25.919029  5504 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 14:52:25.923467  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 972) took 1426 ms. Trace:
I20260709 14:52:25.959853  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.820380  5550 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:25.866125  5198 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:25.962245  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:25.962613  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:25.966616  5415 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:25.969673  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:25.969805  5654 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:52:25.971947  5359 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:25.974119  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:25.974965  5371 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:25.976522  5526 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:25.983876  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.985997  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:25.989563  2681 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:26.060140  2605 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:25.946897  3546 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:25.999955  2684 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:26.007477  2677 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:25.949155  5620 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:25.960124  3537 rpcz_store.cc:276] 0709 14:52:24.499186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.973101 (+473915us) service_pool.cc:224] Handling call
0709 14:52:24.973731 (+   630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:25.131527 (+157796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.131575 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:25.132476 (+   901us) log.cc:844] Serialized 26 byte log entry
0709 14:52:25.317709 (+185233us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:25.318085 (+   376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.318211 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:25.923181 (+604970us) raft_consensus.cc:1685] finished
0709 14:52:25.923209 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.923386 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135049,"raft.queue_time_us":140375,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8465,"thread_start_us":184753,"threads_started":1,"wal-append.queue_time_us":402123}
W20260709 14:52:26.066180  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156258us
I20260709 14:52:25.974256  5378 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:26.046979  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:25.999853  5497 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:26.049118  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:26.013259  5359 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:26.049453  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:26.044234  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:26.090142  2756 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:26.090224  2752 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:26.091257  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 152020us
I20260709 14:52:26.042271  2753 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 14:52:26.058795  2604 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:26.062181  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:26.043416  2757 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:25.987496  4989 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:26.002583  5415 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:26.121441  5672 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:26.124446  2652 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:26.132252  2607 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:26.135448  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.136888  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:26.137799  2608 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:26.148422  5550 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:26.149323  5550 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:26.149761  5550 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:26.149458  5411 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:26.048967  5492 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:26.155426  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1097) took 4597 ms. Trace:
I20260709 14:52:26.016628  5371 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:26.135926  5527 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:26.155663  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:26.168253  2608 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:26.150722  5550 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:26.169234  5550 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:26.169764  2608 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: []
W20260709 14:52:26.174559  5474 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
I20260709 14:52:26.175258  5474 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:26.175699  2608 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:26.177402  2608 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:26.177699  5474 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:26.087042  5620 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:26.164245  2609 rpcz_store.cc:276] 0709 14:52:21.557922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:22.128740 (+570818us) service_pool.cc:224] Handling call
0709 14:52:24.691128 (+2562388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.914087 (+222959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:26.103346 (+1189259us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:26.105094 (+  1748us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b54003c7650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:26.155352 (+ 50258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6360672768}
I20260709 14:52:26.125514  2652 leader_election.cc:403] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 14:52:26.070276  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:26.087567  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71736.2us
I20260709 14:52:26.161512  2677 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:26.162048  2752 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:26.189356  2753 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:26.163300  2758 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:26.187168  2605 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:26.200728  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:26.201854  2752 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:26.202081  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77944.1us
I20260709 14:52:26.118434  2681 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:26.156160  5415 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:26.208294  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.557s (SENT) [suppressed 20 similar messages]
W20260709 14:52:26.199307  5404 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.004s
I20260709 14:52:26.218186  2605 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:26.167196  5371 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:26.222575  2684 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:26.214152  5474 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:26.254875  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:26.255734  2752 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:26.261379  2756 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:26.263063  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 420029us
I20260709 14:52:26.223287  5669 raft_consensus.cc:2764] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 14:52:26.267985  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:26.199155  5222 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:26.273576  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 361704us
W20260709 14:52:26.318321  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 440048us
I20260709 14:52:26.335282  2752 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:26.344277  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:26.346589  2752 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:26.347417  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 209053us
I20260709 14:52:26.228660  5404 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:26.374795  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:26.376353  2756 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:26.398206  2760 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:26.408764  5596 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:26.417933  5677 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:26.418192  5596 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:26.420892  5506 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:26.421644  5506 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:26.423488  5596 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:26.423825  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.411751  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.426955  5527 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:26.340893  5492 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
W20260709 14:52:26.446272  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78610.7us
I20260709 14:52:26.441259  2758 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:52:26.455160  2757 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 14:52:26.457127  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.453127  5596 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:26.446760  5411 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:26.482937  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.468245  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.487746  5526 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:26.488487  5526 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:26.489080  5526 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:26.489389  5639 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.339s	user 0.003s	sys 0.005s
I20260709 14:52:26.420969  5677 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 14:52:26.498817  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 224302us
I20260709 14:52:26.441004  5380 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 14:52:26.504153  2684 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 14:52:26.504746  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:26.509387  2758 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:26.499071  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 159606us
I20260709 14:52:26.468848  5378 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:26.507290  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.524022  2677 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:26.515990  2756 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:26.538440  2681 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 14:52:26.482059  5523 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:26.508972  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:26.542990  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.544353  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:26.458947  5527 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 14:52:26.547892  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:26.548427  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.550640  2680 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:26.557940  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46800.5us
I20260709 14:52:26.562306  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.561005  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.544664  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.580467  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:26.561419  5671 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.002s
I20260709 14:52:26.471698  5506 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:26.587129  2683 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:26.585724  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:26.503772  5596 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:26.555096  5088 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:26.617385  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:26.618232  2684 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:26.620496  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 134955us
I20260709 14:52:26.584928  5620 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:26.626158  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81595.4us
I20260709 14:52:26.634579  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.636336  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.638878  2677 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:26.643518  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:26.641716  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 174460us
I20260709 14:52:26.574393  5669 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 14:52:26.658787  5506 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:26.664583  5682 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 14:52:26.666235  2756 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:26.633313  5222 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:26.655202  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.667634  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 963) took 3764 ms. Trace:
I20260709 14:52:26.647349  5620 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:26.638924  4786 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:26.665791  5675 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 14:52:26.651144  5651 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.003s
I20260709 14:52:26.642541  5492 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:26.617059  5378 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 14:52:26.650625  5404 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:26.666124  5238 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:26.666321  5411 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:26.690060  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.624125  5088 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:26.675572  5359 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.005s
I20260709 14:52:26.675031  3536 rpcz_store.cc:276] 0709 14:52:22.903716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.458974 (+1555258us) service_pool.cc:224] Handling call
0709 14:52:24.479682 (+ 20708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:24.525683 (+ 46001us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:24.545379 (+ 19696us) log.cc:844] Serialized 14 byte log entry
0709 14:52:24.604628 (+ 59249us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b180037a200. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:25.064177 (+459549us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b180037a200. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:52:25.720561 (+656384us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b180037a200. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a
0709 14:52:25.720838 (+   277us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:25.721519 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 14:52:25.995820 (+274301us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:25.995896 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.996028 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:26.667318 (+671290us) raft_consensus.cc:1685] finished
0709 14:52:26.667343 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:26.667549 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9643,"prepare.queue_time_us":15693,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":73,"raft.queue_time_us":16337,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":24176,"spinlock_wait_cycles":52112768,"thread_start_us":289032,"threads_started":2,"wal-append.queue_time_us":643866,"wal-append.run_cpu_time_us":1623,"wal-append.run_wall_time_us":1002586}
W20260709 14:52:26.631863  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 205319us
I20260709 14:52:26.682008  5563 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:26.715862  2611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.011s	sys 0.033s
W20260709 14:52:26.716660  2611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.011s	sys 0.033s
W20260709 14:52:26.717156  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 158247us
W20260709 14:52:26.722407  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 104857us
W20260709 14:52:26.730799  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 75709us
I20260709 14:52:26.732235  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:26.733557  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74886.6us
I20260709 14:52:26.738409  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.687124  5651 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:26.713093  5563 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:26.746150  5694 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:26.693063  5492 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:26.755631  2752 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:26.763157  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:26.707326  5238 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:26.734567  5527 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:26.711227  5359 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:26.798996  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 503) took 1542 ms. Trace:
I20260709 14:52:26.799499  2755 rpcz_store.cc:276] 0709 14:52:25.257071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.259505 (+  2434us) service_pool.cc:224] Handling call
0709 14:52:26.798718 (+1539213us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:26.798889 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326674304}
I20260709 14:52:26.799853  5568 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:26.800093  5359 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:26.754071  5563 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:26.800827  5568 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:26.801954  5568 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:26.795157  5544 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:26.802359  5568 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:26.803098  5544 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:26.803620  5544 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:26.805145  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:26.805965  5544 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:26.795871  5378 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:26.807252  5596 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:26.808102  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56701.8us
I20260709 14:52:26.806648  5544 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:26.815217  5544 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:26.808199  5404 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:26.789301  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 164265us
I20260709 14:52:26.790773  5694 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:26.808203  2755 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:26.815794  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.817087  2760 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:26.821857  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:26.823813  2752 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:26.815310  5526 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:26.771423  5620 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:26.834427  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.817319  5527 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:26.817736  5568 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:26.842351  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:26.821034  4786 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:26.844201  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:26.861284  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:26.840586  5620 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:26.859171  5550 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.689s	user 0.016s	sys 0.001s
I20260709 14:52:26.844950  5527 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:26.828972  5359 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 14:52:26.906199  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45045.5us
W20260709 14:52:26.915256  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55193.5us
I20260709 14:52:26.830715  5404 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:26.864876  5474 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.006s
I20260709 14:52:26.933785  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:26.934734  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:26.933878  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:26.935079  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1118) took 1071 ms. Trace:
I20260709 14:52:26.935724  2611 rpcz_store.cc:276] 0709 14:52:25.852316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.852632 (+   316us) service_pool.cc:224] Handling call
0709 14:52:25.852962 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.853213 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:26.717095 (+863882us) log.cc:844] Serialized 14 byte log entry
0709 14:52:26.920305 (+203210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:26.920391 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:26.920479 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:26.923011 (+  2532us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:26.923260 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193424,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"thread_start_us":1039658,"threads_started":2}
I20260709 14:52:26.935511  3543 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:26.933435  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:26.940824  3547 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:26.940382  2760 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:26.941720  3536 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:52:26.942819  3546 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 14:52:26.905959  5378 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:26.912361  5620 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:25.388996  2612 rpcz_store.cc:276] 0709 14:52:22.702999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.858420 (+2155421us) service_pool.cc:224] Handling call
0709 14:52:24.876427 (+ 18007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:24.916082 (+ 39655us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:25.029495 (+113413us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.317013 (+287518us) log.cc:844] Serialized 14 byte log entry
0709 14:52:25.321514 (+  4501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.321595 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:25.381070 (+ 59475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.381198 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:25.381388 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45965,"prepare.queue_time_us":256203,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":135,"raft.queue_time_us":239,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":84714,"spinlock_wait_cycles":18480896,"thread_start_us":248067,"threads_started":2}
I20260709 14:52:26.971393  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:26.961933  2682 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:26.972045  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 129782us
I20260709 14:52:26.965955  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:26.992705  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:27.003840  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.012956  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:26.995337  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.014007  2677 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:26.864691  4786 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:26.964170  5371 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.006s
W20260709 14:52:26.988243  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53181.5us
I20260709 14:52:27.027303  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 835) took 1954 ms. Trace:
I20260709 14:52:26.986088  5683 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 14:52:26.985471  5687 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 14:52:26.999943  3547 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:26.941823  5474 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:27.031548  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:27.031875  5690 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 14:52:27.032325  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.052111  2683 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:26.958096  5088 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:27.032845  5492 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:26.965091  5526 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:27.035018  5696 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:27.035535  5482 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.196s
W20260709 14:52:27.036667  5691 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 14:52:26.971655  2755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
W20260709 14:52:27.037134  5700 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 14:52:26.987708  5707 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:26.933571  5544 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.038038  5371 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.015565  5702 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:26.985402  5590 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:27.022945  5669 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:27.028187  3538 rpcz_store.cc:276] 0709 14:52:25.072605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.206963 (+134358us) service_pool.cc:224] Handling call
0709 14:52:25.302489 (+ 95526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.328766 (+ 26277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.328822 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:25.328912 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:25.383513 (+ 54601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.026496 (+1642983us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5400318154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:27.026583 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.027207 (+   624us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26119552}
I20260709 14:52:26.963003  5238 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:27.008512  5688 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:52:26.986162  5695 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.047228  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.113142  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:26.986218  5681 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.243s	user 0.002s	sys 0.000s
I20260709 14:52:26.969878  5404 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:26.986110  5704 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:26.986264  5703 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:27.116710  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.116706  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.060250  3543 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:27.042335  5415 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.779s	user 0.013s	sys 0.003s
I20260709 14:52:27.099999  5482 consensus_queue.cc:1243] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:26.973235  5596 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 14:52:27.051725  5378 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:27.071252  5651 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.073341  5526 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:27.044008  5683 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:26.985831  5697 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.310s
I20260709 14:52:27.014947  5563 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:27.107931  5645 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.638s	user 0.004s	sys 0.002s
I20260709 14:52:27.144160  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.108978  2683 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:27.126148  5703 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:27.114403  2758 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:52:27.085374  5707 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:27.117776  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 169518us
W20260709 14:52:27.117973  2605 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.005s	sys 0.023s
W20260709 14:52:27.149443  2605 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.006s	sys 0.024s
I20260709 14:52:27.150436  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 503) took 1262 ms. Trace:
I20260709 14:52:27.118185  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:27.131405  5415 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.151974  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:27.123992  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.153220  5415 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.154268  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.104171  5590 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.090912  5492 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:27.095274  5685 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.064694  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.108575  5699 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:26.960057  5694 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.104370  4786 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.110906  5677 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.088505  5706 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 14:52:27.107923  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.146529  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.157259  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:27.176250  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1113) took 1224 ms. Trace:
I20260709 14:52:27.159523  2678 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:27.183959  3538 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:27.161944  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.186116  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:27.167012  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142511us
I20260709 14:52:27.169003  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:27.170785  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 142736us
W20260709 14:52:27.171260  5414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.014s	sys 0.004s
I20260709 14:52:27.187264  3547 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:27.169027  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.153314  2605 rpcz_store.cc:276] 0709 14:52:25.887840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.221474 (+333634us) service_pool.cc:224] Handling call
0709 14:52:26.221825 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:26.222049 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:26.265649 (+ 43600us) log.cc:844] Serialized 14 byte log entry
0709 14:52:26.549884 (+284235us) log.cc:844] Serialized 14 byte log entry
0709 14:52:27.149958 (+600074us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:27.150034 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:27.150097 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.150185 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.150367 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39556,"prepare.queue_time_us":39627,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":3941,"raft.queue_time_us":170189,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":7954,"spinlock_wait_cycles":8767488,"thread_start_us":824572,"threads_started":2,"wal-append.queue_time_us":505041}
I20260709 14:52:27.188292  2683 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:27.109966  5702 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.156674  5703 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:27.155999  5707 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:27.192412  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:27.255822  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63476.7us
I20260709 14:52:27.256765  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:27.257680  5378 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:27.257799  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:27.258332  5378 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.259047  5697 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:27.169078  5492 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.195327  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.195581  5705 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:27.195910  5718 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:52:27.197347  5712 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 14:52:27.203940  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:27.207473  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 832) took 4303 ms. Trace:
I20260709 14:52:27.207985  5714 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:27.195634  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.296695  2684 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:27.212445  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.212899  5590 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:27.169064  5699 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.213604  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:27.191745  5414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.321s	user 0.015s	sys 0.005s
I20260709 14:52:27.218361  2686 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:27.225317  2680 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:27.225695  5527 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.227690  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.228399  2685 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 14:52:27.185897  5677 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.310464  5699 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:27.312189  5699 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:27.312958  5677 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.229941  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:27.169697  5685 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.315239  5371 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:27.315577  5685 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:27.121011  5695 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.316740  2679 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:27.317483  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1019) took 1186 ms. Trace:
I20260709 14:52:27.318372  5695 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.240777  5563 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:27.244474  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:27.251544  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.183965  2759 rpcz_store.cc:276] 0709 14:52:25.945541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.945764 (+   223us) service_pool.cc:224] Handling call
0709 14:52:25.959015 (+ 13251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:25.962933 (+  3918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.962971 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:25.963658 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 14:52:26.280616 (+316958us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:26.402030 (+121414us) log.cc:844] Serialized 14 byte log entry
0709 14:52:26.402366 (+   336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:26.402483 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:27.040490 (+638007us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b18002318e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:27.165505 (+125015us) raft_consensus.cc:1685] finished
0709 14:52:27.169195 (+  3690us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.169368 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5958,"prepare.queue_time_us":123872,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":91575,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":266,"spinlock_wait_cycles":36864896,"thread_start_us":316344,"threads_started":1,"wal-append.queue_time_us":512809}
I20260709 14:52:27.193356  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.320367  2752 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:27.320575  2760 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:27.271281  3542 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:27.126729  5704 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.285593  3543 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:27.285524  3541 rpcz_store.cc:276] 0709 14:52:22.904065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:24.853619 (+1949554us) service_pool.cc:224] Handling call
0709 14:52:24.874555 (+ 20936us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:25.109564 (+235009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:25.109596 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:25.110339 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 14:52:25.143768 (+ 33429us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:25.143818 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:25.143896 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:27.106975 (+1963079us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:27.206951 (+ 99976us) raft_consensus.cc:1685] finished
0709 14:52:27.206988 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.207360 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":193502,"raft.queue_time_us":196467,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":27896,"spinlock_wait_cycles":24036096,"thread_start_us":32973,"threads_started":1,"wal-append.queue_time_us":277601}
I20260709 14:52:27.259404  5378 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.293473  5620 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:27.303392  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60534us
I20260709 14:52:27.303993  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:27.305501  2681 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:52:27.310184  5722 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:52:27.321754  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62987.4us
I20260709 14:52:27.318038  2756 rpcz_store.cc:276] 0709 14:52:26.130815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.684398 (+553583us) service_pool.cc:224] Handling call
0709 14:52:26.684923 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:26.704014 (+ 19091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:26.704072 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:26.716820 (+ 12748us) log.cc:844] Serialized 26 byte log entry
0709 14:52:26.878362 (+161542us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:26.878434 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:26.878563 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:27.317240 (+438677us) raft_consensus.cc:1685] finished
0709 14:52:27.317263 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.317415 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41746,"raft.queue_time_us":19600,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":6538,"thread_start_us":96553,"threads_started":1,"wal-append.queue_time_us":305455}
I20260709 14:52:27.309185  2685 pending_rounds.cc:85] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 4
I20260709 14:52:27.331080  2685 pending_rounds.cc:107] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 14:52:27.331390  2685 raft_consensus.cc:2889] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:27.328758  3547 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:27.295949  5714 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.334232  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.339715  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:27.341308  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.342298  2684 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:52:27.343098  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:27.304651  5697 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:27.345022  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100655us
I20260709 14:52:27.347455  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.347968  3543 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:27.349841  3541 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:27.350005  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.350265  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.324157  5718 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:27.351006  3543 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:27.351351  2753 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:52:27.354036  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.350263  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:27.354761  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 142443us
I20260709 14:52:27.359970  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:27.329857  5590 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:27.362389  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 174774us
I20260709 14:52:27.363474  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.364318  2760 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:27.367733  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.368341  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:27.372030  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.373271  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:27.373927  2760 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:27.217356  5683 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:27.374923  3543 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:27.375818  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:27.377219  5723 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:27.379438  2752 raft_consensus.cc:2376] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:27.367239  4989 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:27.383160  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:27.371415  5371 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.398823  3547 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:27.379139  5589 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:27.401371  3547 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:52:27.375123  5590 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.410025  5371 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.401684  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:27.401886  3538 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:27.402819  2430 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 96) took 37674 ms (client timeout 15000 ms). Trace:
I20260709 14:52:27.403975  5527 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:27.404163  5702 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:27.412986  2651 connection.cc:256] RPC call timeout handler was delayed by -1.3869130919999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 14:52:27.414965  3547 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
W20260709 14:52:27.428534  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.408574  5523 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:27.429013  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.408915  5704 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.429141  5523 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:27.383455  5714 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.385582  4989 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:27.429656  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.429967  4989 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:27.430444  3547 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:27.424080  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:27.425865  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:27.431550  5380 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:27.415232  5589 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:27.415478  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.443441  3538 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:27.415742  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:27.406968  5689 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.629s	user 0.004s	sys 0.000s
I20260709 14:52:27.438794  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:27.450335  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:27.410063  5590 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:27.442142  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:27.402227  5696 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:27.427551  5506 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.750s	user 0.014s	sys 0.002s
I20260709 14:52:27.454939  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:27.436734  5380 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:27.450665  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.413828  5620 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:27.412714  5371 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.453792  5718 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.464695  5730 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:27.478536  5730 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.478116  5718 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.479545  5730 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:27.480926  5730 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:27.465798  5677 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:27.486002  5677 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:27.487052  5677 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.488404  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1109) took 1800 ms. Trace:
I20260709 14:52:27.489024  2751 rpcz_store.cc:276] 0709 14:52:25.688515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.706189 (+ 17674us) service_pool.cc:224] Handling call
0709 14:52:25.706771 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.884450 (+177679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.488064 (+1603614us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54000d0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:27.488320 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375713536}
I20260709 14:52:27.485229  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.454358  5704 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:27.464979  5669 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:27.466809  5694 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:27.513873  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.466123  5589 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:27.519541  2752 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:27.410665  5703 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.553098  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.517804  5492 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.566267  5380 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:27.574404  4786 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 14:52:27.581470  5589 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:27.583964  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 145347us
I20260709 14:52:27.584074  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.526134  5704 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:27.585543  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:27.532876  5669 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:27.587656  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:27.588945  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 205872us
I20260709 14:52:27.589380  2753 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:27.591315  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.593549  2755 raft_consensus.cc:606] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.593699  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.595386  2755 raft_consensus.cc:614] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:27.596370  2759 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:27.559392  5732 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 14:52:27.597383  5600 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.010s	sys 0.006s
I20260709 14:52:27.605472  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.607336  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:27.414588  2430 rpcz_store.cc:269] 0709 14:51:49.728565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:51:49.738939 (+ 10374us) service_pool.cc:224] Handling call
0709 14:52:27.397712 (+37658773us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:52:17.420698 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:52:17.424831 (+  4133us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:52:17.424897 (+    66us) write_op.cc:435] Acquired schema lock
0709 14:52:17.424921 (+    24us) tablet.cc:662] Decoding operations
0709 14:52:17.647232 (+222311us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:52:17.678142 (+ 30910us) write_op.cc:641] Partition lock acquired for write op
0709 14:52:17.679591 (+  1449us) tablet.cc:685] Acquiring locks for 8 operations
0709 14:52:17.963101 (+283510us) tablet.cc:701] Row locks acquired
0709 14:52:17.963165 (+    64us) write_op.cc:260] PREPARE: finished
0709 14:52:17.963446 (+   281us) write_op.cc:270] Start()
0709 14:52:17.963687 (+   241us) write_op.cc:276] Timestamp: P: 1783608737963351 usec, L: 0
0709 14:52:17.963720 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 14:52:17.964807 (+  1087us) log.cc:844] Serialized 3253 byte log entry
0709 14:52:18.439056 (+474249us) op_driver.cc:464] REPLICATION: finished
0709 14:52:19.191116 (+752060us) write_op.cc:301] APPLY: starting
0709 14:52:19.191274 (+   158us) tablet.cc:1367] starting BulkCheckPresence
0709 14:52:19.191536 (+   262us) tablet.cc:1370] finished BulkCheckPresence
0709 14:52:19.191690 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:52:19.196879 (+  5189us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:52:19.197265 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 14:52:19.197317 (+    52us) write_op.cc:312] APPLY: finished
0709 14:52:19.197761 (+   444us) log.cc:844] Serialized 64 byte log entry
0709 14:52:19.198004 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:19.198169 (+   165us) write_op.cc:454] Released schema lock
0709 14:52:19.198253 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 14:52:19.228503 (+ 30250us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:19.228548 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4102,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":746341,"apply.run_cpu_time_us":5220,"apply.run_wall_time_us":37350,"num_ops":8,"prepare.queue_time_us":556785,"prepare.run_cpu_time_us":23074,"prepare.run_wall_time_us":671163,"raft.queue_time_us":253377,"raft.run_cpu_time_us":27425,"raft.run_wall_time_us":694338,"replication_time_us":475037,"thread_start_us":1371196,"threads_started":4,"wal-append.queue_time_us":162685,"wal-append.run_cpu_time_us":20357,"wal-append.run_wall_time_us":2081765}]]}
I20260709 14:52:27.615487  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.615594  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.452000  5697 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.582791  5198 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:27.617056  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.613792  5371 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:27.613075  5600 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.011s	sys 0.006s
I20260709 14:52:27.601181  5731 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:27.641026  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:27.612782  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:27.599786  5492 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:27.643424  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 511) took 2200 ms. Trace:
W20260709 14:52:27.648293  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 261795us
I20260709 14:52:27.620649  5222 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:27.614982  5596 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.619684  5482 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:27.620657  5506 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:52:27.685077  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44169.3us
I20260709 14:52:27.684088  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:27.632907  5378 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:27.739998  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57190.6us
W20260709 14:52:27.740415  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 127963us
I20260709 14:52:27.742321  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:27.753528  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 136587us
I20260709 14:52:27.744339  2604 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:52:27.648351  2754 rpcz_store.cc:276] 0709 14:52:25.445551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.445750 (+   199us) service_pool.cc:224] Handling call
0709 14:52:25.469883 (+ 24133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:25.685766 (+215883us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:25.813220 (+127454us) log.cc:844] Serialized 14 byte log entry
0709 14:52:26.265144 (+451924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:26.265221 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:26.265294 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.643038 (+1377744us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:27.643088 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:27.643317 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21304,"prepare.queue_time_us":72463,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":77991,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":26125,"spinlock_wait_cycles":423296384,"thread_start_us":423447,"threads_started":1,"wal-append.queue_time_us":655680}
I20260709 14:52:27.754370  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.646160  5088 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.745906  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:27.754096  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.645834  5496 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.645556  5702 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.647599  5733 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:27.687386  5696 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:27.810169  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.646126  5371 consensus_queue.cc:1243] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:27.763615  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:27.815220  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.801491  5088 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.802646  5496 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.746130  5620 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.645022  5678 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.681650  5222 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:27.686473  5492 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:27.852409  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:27.815755  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.821636  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:27.685714  5732 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:27.819828  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:27.891041  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 75351.6us
I20260709 14:52:27.855502  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.901022  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:27.811151  5723 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.816144  3537 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:27.912235  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.755877  5198 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:27.875377  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.856185  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.880105  5620 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.894344  5677 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:27.894512  5678 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.799202  5415 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.115s
I20260709 14:52:27.762629  5725 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:27.882352  2610 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:27.825603  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73060.1us
W20260709 14:52:27.911011  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 152546us
I20260709 14:52:27.816870  5696 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:27.797385  5414 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:27.896238  5222 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:27.931643  2611 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:27.930377  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55155us
I20260709 14:52:27.893491  5745 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:27.938429  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141222us
I20260709 14:52:27.958033  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:27.895884  5706 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.895625  5695 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 14:52:27.931202  5707 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.676s	user 0.021s	sys 0.001s
I20260709 14:52:27.892413  5744 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.891199  5088 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:27.895480  5731 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 14:52:27.891361  5590 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:27.740710  5596 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:27.809544  5378 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:27.913177  5671 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 14:52:27.962170  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:27.964368  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.893545  5496 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:27.826591  5704 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.816658  5702 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.937176  5672 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:27.763020  5527 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:27.936106  5677 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:27.899278  5550 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 14:52:27.974442  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:27.933329  5723 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:27.962090  5699 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.649s	user 0.016s	sys 0.009s
I20260709 14:52:27.954346  5696 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:27.964368  5568 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.100s	user 0.008s	sys 0.021s
W20260709 14:52:28.051347  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.023s	sys 0.048s
I20260709 14:52:27.823130  5733 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.985955  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:28.052680  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.023s	sys 0.048s
I20260709 14:52:27.984707  2604 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:27.983664  5734 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 14:52:27.983732  5701 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:28.012933  2754 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:27.942396  5198 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:27.997169  5506 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:28.004755  5492 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:28.044018  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.064592  2603 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:27.953935  5414 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.004842  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:27.936203  5669 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:27.977813  5744 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:27.972287  5707 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.003746  5672 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:28.073836  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 319549us
W20260709 14:52:28.063244  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105383us
I20260709 14:52:28.065662  3540 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:28.013099  5677 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:28.014271  5496 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.075909  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:27.986665  5725 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 14:52:28.091604  2611 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:27.967967  5706 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.087793  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:28.087338  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.002s	sys 0.003s
I20260709 14:52:28.043339  5747 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:28.043006  5749 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:28.093171  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.003s	sys 0.003s
W20260709 14:52:28.088527  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.101867  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.068964  5414 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.013739  5527 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.092908  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:27.970458  5745 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.109477  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.095903  3547 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:28.055536  5731 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:28.078547  5672 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.113919  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:28.070367  5753 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.137575  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:28.073101  5756 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:28.071911  5701 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:28.062393  5568 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:28.104971  5705 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.806s	user 0.015s	sys 0.000s
I20260709 14:52:28.141742  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:28.055450  5752 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 14:52:28.134192  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.148203  5753 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.153235  5707 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.153430  5705 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:28.154223  5753 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.154517  2608 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:28.067932  5732 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:28.155789  5742 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 14:52:28.157666  5740 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.182s	user 0.000s	sys 0.002s
I20260709 14:52:28.159094  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 979) took 1233 ms. Trace:
I20260709 14:52:28.159999  5527 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.160532  5733 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.161824  5378 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.162791  5763 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:28.163403  5763 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.163837  5763 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:28.134002  5719 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:28.165035  5744 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.166134  5744 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:28.117126  5088 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 14:52:28.150594  2611 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:28.115931  5731 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:28.183454  2753 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.584s	user 0.005s	sys 0.009s
I20260709 14:52:28.185195  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:28.186972  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:28.159538  3542 rpcz_store.cc:276] 0709 14:52:26.925568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.325016 (+399448us) service_pool.cc:224] Handling call
0709 14:52:27.325425 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:27.338225 (+ 12800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:27.338281 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:27.343611 (+  5330us) log.cc:844] Serialized 26 byte log entry
0709 14:52:27.609893 (+266282us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:27.609969 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.610098 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:28.158810 (+548712us) raft_consensus.cc:1685] finished
0709 14:52:28.158858 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:28.159025 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12371,"raft.queue_time_us":13029,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"thread_start_us":265700,"threads_started":1,"wal-append.queue_time_us":561909}
W20260709 14:52:28.188165  5738 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 14:52:28.146632  5590 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:28.148953  5756 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.134804  5496 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.203155  5756 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.204047  5544 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:28.205334  5496 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.167284  5759 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:28.166606  5744 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:28.209306  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 264702us
W20260709 14:52:28.194468  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.165026  5719 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:28.198714  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.219679  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 503) took 1009 ms. Trace:
I20260709 14:52:28.201979  5702 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:28.153264  5568 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.200441  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.222276  2611 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:28.221799  2685 rpcz_store.cc:276] 0709 14:52:27.210690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.215335 (+  4645us) service_pool.cc:224] Handling call
0709 14:52:27.219740 (+  4405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:27.344585 (+124845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.207951 (+863366us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b5400267d50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:28.219577 (+ 11626us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8817536}
I20260709 14:52:28.219938  5725 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 14:52:28.224965  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.220086  5744 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.220126  5405 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 14:52:28.212697  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.201149  5590 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:28.109580  5749 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.243028  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.244266  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:28.249075  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107486us
I20260709 14:52:28.251349  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:28.244930  5751 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.253263  5751 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.254421  5088 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:28.151575  5750 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.227s
I20260709 14:52:28.252245  5725 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:28.258688  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:28.221900  5620 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:28.262619  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73322.2us
W20260709 14:52:28.262691  2726 leader_election.cc:336] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.550s (SENT)
I20260709 14:52:28.237874  5747 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.260445  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:28.247509  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:28.276449  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:28.246210  5590 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:28.192704  5506 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.329573  2685 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:28.235391  5496 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 14:52:28.337735  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.333828  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.207367  5758 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 14:52:28.245532  5596 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.336279  5750 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:28.339835  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.264976  5725 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:28.336318  3409 connection.cc:256] RPC call timeout handler was delayed by -1.2994312719999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
I20260709 14:52:28.253252  5772 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:28.348846  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81438.9us
I20260709 14:52:28.349129  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.350215  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.352114  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 520) took 1230 ms. Trace:
I20260709 14:52:28.353538  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.365051  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:28.365353  2679 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:28.331638  5378 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:28.369469  2607 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:28.259819  5702 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 14:52:28.228609  5198 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.364591  2753 rpcz_store.cc:276] 0709 14:52:27.121624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.327605 (+1205981us) service_pool.cc:224] Handling call
0709 14:52:28.352045 (+ 24440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.327741  5739 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.372428  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:28.376590  3409 connection.cc:256] RPC call timeout handler was delayed by -1.069196431000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 14:52:28.439069  5768 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.278s	user 0.002s	sys 0.000s
W20260709 14:52:28.441241  5754 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 14:52:28.445230  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80310.3us
I20260709 14:52:28.378665  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:28.449118  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.237145  5731 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.369904  5378 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:28.450850  5765 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 14:52:28.356073  5750 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.368409  5772 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.256919  5701 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.481457  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.480284  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.486891  2607 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:28.451084  5723 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.494527  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1009) took 2389 ms. Trace:
I20260709 14:52:28.495285  2757 rpcz_store.cc:276] 0709 14:52:26.105094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.108012 (+  2918us) service_pool.cc:224] Handling call
0709 14:52:26.454719 (+346707us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:26.458707 (+  3988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.494183 (+2035476us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540040e2d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:28.494425 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720386304}
I20260709 14:52:28.500255  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.500666  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:28.501694  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.502468  3540 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:28.502569  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:28.503480  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.476099  5378 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.504868  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:28.485257  5088 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:28.482937  5750 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:28.512213  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.518363  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 850) took 1522 ms. Trace:
I20260709 14:52:28.518985  3543 rpcz_store.cc:276] 0709 14:52:26.995711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.380662 (+384951us) service_pool.cc:224] Handling call
0709 14:52:27.398436 (+ 17774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:27.398983 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:27.437336 (+ 38353us) log.cc:844] Serialized 14 byte log entry
0709 14:52:28.105738 (+668402us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:28.118466 (+ 12728us) log.cc:844] Serialized 26 byte log entry
0709 14:52:28.118695 (+   229us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:28.118748 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.118868 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:28.518072 (+399204us) raft_consensus.cc:1685] finished
0709 14:52:28.518099 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:28.518284 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5917,"prepare.queue_time_us":44432,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":8340,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":16688,"thread_start_us":588227,"threads_started":1,"wal-append.queue_time_us":713937}
I20260709 14:52:28.339386  5747 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.492327  5706 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.540161  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.541071  2607 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:28.486137  5772 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.538165  2611 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:28.545632  3540 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:28.547654  5751 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.517468  5620 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:28.510987  5745 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.521580  5771 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:28.558887  5771 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:28.559374  5771 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:28.569243  5762 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.408s	user 0.003s	sys 0.000s
I20260709 14:52:28.570067  5776 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:28.570493  5755 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:28.570956  5767 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 14:52:28.571497  2757 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:28.446266  5739 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.572551  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1018) took 2442 ms. Trace:
W20260709 14:52:28.573936  5523 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.144s	user 0.027s	sys 0.001s
W20260709 14:52:28.576884  5222 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.606s	user 0.005s	sys 0.009s
I20260709 14:52:28.577399  3543 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:28.581382  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56734 (request call id 22) took 1650 ms. Trace:
I20260709 14:52:28.581955  5414 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.587611  2605 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:28.592919  5719 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.595153  5544 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:28.596906  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 327619us
W20260709 14:52:28.598802  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 947) took 2095 ms (client timeout 2159 ms). Trace:
I20260709 14:52:28.600049  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.604166  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 494) took 1580 ms. Trace:
I20260709 14:52:28.604941  5614 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:28.607949  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.610303  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.490509  5696 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.614622  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.667872  3538 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 5: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:28.615366  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.615801  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.668177  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.669158  2607 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:28.669346  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.618868  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:28.670667  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51963us
W20260709 14:52:28.665656  2678 rpcz_store.cc:269] 0709 14:52:26.503729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.505091 (+  1362us) service_pool.cc:224] Handling call
0709 14:52:28.598510 (+2093419us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:28.598698 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206757120}
I20260709 14:52:28.665910  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:28.673274  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:28.644728  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:28.598536  5672 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.646867  5590 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.647334  5550 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:28.650939  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 528) took 1429 ms. Trace:
W20260709 14:52:28.695117  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.695029  2757 rpcz_store.cc:276] 0709 14:52:27.233482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.516385 (+1282903us) service_pool.cc:224] Handling call
0709 14:52:28.571294 (+ 54909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:28.650537 (+ 79243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.650868 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.649623  2758 rpcz_store.cc:276] 0709 14:52:26.130406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.510453 (+380047us) service_pool.cc:224] Handling call
0709 14:52:26.547651 (+ 37198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:27.148956 (+601305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.572106 (+1423150us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400120054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:28.572463 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302684416}
I20260709 14:52:28.504245  5723 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:28.654874  2760 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 14:52:28.717634  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 530) took 1483 ms. Trace:
I20260709 14:52:28.718211  2760 rpcz_store.cc:276] 0709 14:52:27.234199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.630020 (+1395821us) service_pool.cc:224] Handling call
0709 14:52:28.654417 (+ 24397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:28.717187 (+ 62770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.717545 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.720185  2760 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:28.657572  2755 rpcz_store.cc:276] 0709 14:52:26.930996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.931164 (+   168us) service_pool.cc:224] Handling call
0709 14:52:28.581006 (+1649842us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:28.581221 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190777216}
I20260709 14:52:28.663839  5719 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.666448  2677 rpcz_store.cc:276] 0709 14:52:27.023699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.023868 (+   169us) service_pool.cc:224] Handling call
0709 14:52:27.024374 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:27.071186 (+ 46812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:27.071223 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:27.071875 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 14:52:27.175756 (+103881us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:27.175899 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:27.176440 (+   541us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:28.603147 (+1426707us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b540007d3a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bbdbe 00007fa16e5c6930 00007fa16e5c68d8 00007fa16e5c6858 00007fa16e5c6813 00007fa16e5c5a9c 00007fa1744b1626 00007fa1744abdd8 00007fa16e6027cb 00007fa16e6052c0
0709 14:52:28.603774 (+   627us) raft_consensus.cc:1685] finished
0709 14:52:28.603797 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:28.604065 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16597,"raft.queue_time_us":28223,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":16977280,"thread_start_us":95548,"threads_started":1,"wal-append.queue_time_us":99482}
I20260709 14:52:28.671613  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:28.750310  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78810.2us
I20260709 14:52:28.745365  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.740335  2757 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 14:52:28.752990  2679 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.597868  5744 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.631305  5755 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.642199  3545 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:28.757555  3545 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:28.664803  5677 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.004s
I20260709 14:52:28.678617  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.680907  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.681282  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:28.766018  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84827.7us
I20260709 14:52:28.679997  5756 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.682973  5696 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.682924  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.781525  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.782509  2682 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:28.683772  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.688407  5590 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.685647  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.784171  3540 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.686228  5088 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:28.690860  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:28.794795  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104083us
I20260709 14:52:28.694787  5550 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:28.702541  5747 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.716173  2754 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:28.745419  2683 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 14:52:28.752534  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:28.802246  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49800.1us
I20260709 14:52:28.727362  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.802502  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.740085  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.747967  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.740362  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 533) took 1355 ms. Trace:
W20260709 14:52:28.759258  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 380703us
I20260709 14:52:28.804668  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:28.805960  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:28.761130  5492 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:28.760159  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 531) took 1414 ms. Trace:
I20260709 14:52:28.761178  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.688337  5380 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.673326  5614 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:28.715343  5680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.622s	user 0.032s	sys 0.076s
I20260709 14:52:28.688481  5672 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.640663  5651 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.727025  5405 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:28.785743  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.781945  2607 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:28.659703  5523 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.827150  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 980) took 1901 ms. Trace:
I20260709 14:52:28.697597  5496 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:28.791239  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.700752  5749 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.778754  3545 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:28.741456  5730 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.246s	user 0.011s	sys 0.007s
I20260709 14:52:28.797413  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:28.762914  5762 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 14:52:28.799353  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:28.666577  5222 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.761175  5755 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.807549  2757 rpcz_store.cc:276] 0709 14:52:27.346218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.696387 (+1350169us) service_pool.cc:224] Handling call
0709 14:52:28.740003 (+ 43616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:28.753328 (+ 13325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.760045 (+  6717us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15744}
I20260709 14:52:28.813484  5696 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:28.689565  5544 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.803279  2683 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:28.782079  5706 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.781695  5723 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:28.815547  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1119) took 1070 ms. Trace:
I20260709 14:52:28.814472  5492 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:28.859004  2603 rpcz_store.cc:276] 0709 14:52:27.738008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.747063 (+  9055us) service_pool.cc:224] Handling call
0709 14:52:28.805763 (+1058700us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:28.805928 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582386560}
I20260709 14:52:28.820803  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.860312  2684 raft_consensus.cc:2376] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9. Re-sending same reply.
I20260709 14:52:28.860525  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.824870  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:28.698112  5751 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:28.794225  5088 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:28.835210  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.875533  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:28.876507  3546 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.842963  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.840471  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.877648  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.878014  2685 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:28.850746  2682 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 5.
I20260709 14:52:28.834282  3541 rpcz_store.cc:276] 0709 14:52:26.925935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.350640 (+424705us) service_pool.cc:224] Handling call
0709 14:52:27.367871 (+ 17231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:27.368064 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:27.386125 (+ 18061us) log.cc:844] Serialized 14 byte log entry
0709 14:52:28.602497 (+1216372us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b180044f6e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:52:28.826492 (+223995us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:28.826556 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:28.826612 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.826959 (+   347us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:28.827089 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17888,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":109,"spinlock_wait_cycles":13107456,"thread_start_us":223643,"threads_started":1}
W20260709 14:52:28.820011  5680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.748s	user 0.034s	sys 0.076s
I20260709 14:52:28.854995  3536 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 14:52:28.804155  2760 rpcz_store.cc:276] 0709 14:52:27.365786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.719402 (+1353616us) service_pool.cc:224] Handling call
0709 14:52:28.719912 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:28.720784 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.721057 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.783912  5677 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.858400  3545 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.826721  5620 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:28.861302  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.821620  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:28.823660  5371 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:28.878763  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1137) took 1011 ms. Trace:
I20260709 14:52:28.893337  2607 rpcz_store.cc:276] 0709 14:52:27.867507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.827255 (+959748us) service_pool.cc:224] Handling call
0709 14:52:28.878667 (+ 51412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.823329  5776 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.907229  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:28.907879  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.908694  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:28.908851  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:28.909845  2760 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.806520  5778 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:28.882162  2757 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:28.884689  2755 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:28.886667  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65121.4us
I20260709 14:52:28.886602  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.837743  5496 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.888391  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.901132  2604 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:28.835268  5378 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.906064  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.858004  5405 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:28.862993  5749 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:28.917506  5620 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:28.933807  5780 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:52:28.918100  5778 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.938035  5778 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.940981  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 534) took 1455 ms. Trace:
I20260709 14:52:28.941419  5544 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.941232  5405 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.941879  2757 rpcz_store.cc:276] 0709 14:52:27.485791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.929638 (+1443847us) service_pool.cc:224] Handling call
0709 14:52:28.940333 (+ 10695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:28.940582 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:28.940613 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:28.940668 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:28.940708 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.940767 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:28.940911 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:28.942518   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:52:28.944319  5378 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.946170   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:52:28.923748  2677 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:28.950623  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.951555  2603 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:28.952072  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.952510  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.953143  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:52:28.883888  5763 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.004s
I20260709 14:52:28.882573  5759 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.954753  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:28.955610  2607 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:28.954267  5763 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:28.956647  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.957417  2607 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:28.957127  5614 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:28.960095  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.949779  5590 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:28.962054  2609 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:28.827759  5745 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 14:52:28.953365  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.962323  5590 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.886072  5706 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.965849  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.969033  5523 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:28.969501  5706 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.971393  5756 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.881345  5088 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:28.971338  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.975471  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.976217  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:28.976954  2677 raft_consensus.cc:2376] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11. Re-sending same reply.
I20260709 14:52:28.881309  5550 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.978026  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.979276  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:28.980237  2677 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:28.906343  5371 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:28.980466  5677 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:28.980861  5371 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:28.878945  5492 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:28.907481  5766 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:28.981909  5744 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:28.992950  5737 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.103s	user 0.003s	sys 0.000s
I20260709 14:52:28.946666  2685 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:28.991698  5778 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:28.957942  5763 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:28.997463  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:28.997586  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1120) took 1255 ms. Trace:
I20260709 14:52:28.998270  3547 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.955166  5759 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:28.988893  5766 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:29.002789  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:28.958536  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:28.981226  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:29.008862  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1025) took 1589 ms. Trace:
I20260709 14:52:28.998469  2608 rpcz_store.cc:276] 0709 14:52:27.742072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.742254 (+   182us) service_pool.cc:224] Handling call
0709 14:52:27.742651 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:28.155014 (+412363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.997155 (+842141us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54002ff3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:28.997484 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946639744}
I20260709 14:52:29.016755  3540 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:28.966320  5590 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.019778  2679 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:29.019277  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:28.849771  5651 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.022298  2607 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:29.027171  2754 rpcz_store.cc:276] 0709 14:52:27.419456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.090901 (+671445us) service_pool.cc:224] Handling call
0709 14:52:29.008633 (+917732us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54000d0f54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.008793 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347842432}
I20260709 14:52:29.031002  3541 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 14:52:29.031872  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.032071  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:29.038390  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.015508  3547 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:29.038542  3540 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 14:52:29.041303  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.043289  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.043789  2760 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:29.047075  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:28.978588  5550 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:28.959275  5614 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:28.928470  5496 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:29.021679  4989 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.591s	user 0.015s	sys 0.012s
I20260709 14:52:29.076828  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.093286  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.007506  5596 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:29.009390  5766 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:28.956658  5222 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:29.004541  5482 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:29.048745  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:29.107345  2685 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:29.007211  5778 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:29.092821  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.111012  2754 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:29.116241  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63711.3us
W20260709 14:52:29.113052  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.050376  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.059695  3540 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:29.092200  5544 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:29.048204  5782 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:29.121205  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:29.122057  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.077384  5749 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:29.131670  3545 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:29.095191  5596 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.133881  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.111796  5482 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.033133  5600 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 14:52:29.049924  5702 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 14:52:29.119984  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69696.9us
I20260709 14:52:29.104230  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.129519  5718 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:52:29.051806  5378 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:52:29.123880  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89979.1us
I20260709 14:52:29.145071  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:29.145582  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 346317us
W20260709 14:52:29.125119  5772 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.000s
I20260709 14:52:29.119038  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:29.077126  5590 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 14:52:29.136307  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.150972  2757 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.103505  4989 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:29.139282  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.111896  5759 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.142977  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:29.159315  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:29.136018  5599 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 14:52:29.165997  5576 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:29.167222  5576 raft_consensus.cc:2764] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 14:52:29.167080  5563 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:29.168180  5576 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:29.168479  5563 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:29.168588  5576 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.169034  5563 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:29.169725  5576 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.169958  5563 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.172446  5576 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.159070  5723 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:29.176054  5723 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.179956  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:29.159551  5718 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.182442  3537 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:29.182886  5718 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.183861  5651 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:29.188627  2760 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 14:52:29.190951  5563 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:29.132977  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 841) took 2225 ms. Trace:
I20260709 14:52:29.146564  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.148725  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:29.160084  2607 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:29.145701  5414 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:29.196960  5779 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 14:52:29.151383  5544 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:29.199360  5786 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 14:52:29.032979  5763 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 14:52:29.140719  5702 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:29.203488  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56991.8us
I20260709 14:52:29.206179  5763 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:29.207820  5766 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.208068  5088 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.164840  2682 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 14:52:29.192430  3545 rpcz_store.cc:276] 0709 14:52:26.907377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:26.907575 (+   198us) service_pool.cc:224] Handling call
0709 14:52:26.918465 (+ 10890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:28.642016 (+1723551us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54002fb5a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:52:29.132268 (+490252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.132883 (+   615us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486716160}
I20260709 14:52:29.190577  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.212018  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.213316  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.214025  2760 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:52:29.214118  5771 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.654s	user 0.014s	sys 0.000s
I20260709 14:52:29.137086  5745 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 14:52:29.227075  5750 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.005s
I20260709 14:52:29.208596  5766 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.227730  5750 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.229132  2680 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:29.209141  5526 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:29.230460  5766 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:29.232319  5757 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.070s	user 0.005s	sys 0.002s
I20260709 14:52:29.234468  5088 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.234494  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1128) took 1382 ms. Trace:
I20260709 14:52:29.236606  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 529) took 2002 ms. Trace:
I20260709 14:52:29.235359  2606 rpcz_store.cc:276] 0709 14:52:27.852621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.856494 (+  3873us) service_pool.cc:224] Handling call
0709 14:52:27.857001 (+   507us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:27.880936 (+ 23935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:27.880972 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:27.934729 (+ 53757us) log.cc:844] Serialized 52 byte log entry
0709 14:52:28.069362 (+134633us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:28.092481 (+ 23119us) log.cc:844] Serialized 14 byte log entry
0709 14:52:28.108333 (+ 15852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:28.108455 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:29.230954 (+1122499us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:29.234180 (+  3226us) raft_consensus.cc:1685] finished
0709 14:52:29.234292 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:29.234431 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38866,"prepare.queue_time_us":22898,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":196105,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":7649,"spinlock_wait_cycles":25792256,"thread_start_us":95784,"threads_started":1,"wal-append.queue_time_us":209277}
I20260709 14:52:29.238395  2754 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:52:29.238713  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85909.8us
I20260709 14:52:29.215327  2757 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:29.240525  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.219473  3536 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.224783  5773 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:29.220023  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.205601  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.160389  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.237089  2752 rpcz_store.cc:276] 0709 14:52:27.233901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.564210 (+1330309us) service_pool.cc:224] Handling call
0709 14:52:28.564648 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:28.565100 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:28.565132 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:28.565736 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 14:52:29.049063 (+483327us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:29.049107 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.049259 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:29.236327 (+187068us) raft_consensus.cc:1685] finished
0709 14:52:29.236350 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:29.236530 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2594,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":482834,"threads_started":1,"wal-append.queue_time_us":576726}
I20260709 14:52:29.244946  2760 raft_consensus.cc:2368] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:52:29.247295  5526 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:29.248018  5526 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:29.249248  5526 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.250059  5749 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:29.251997  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.140332  5782 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 14:52:29.255683  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:29.257835  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 97543.1us
I20260709 14:52:29.242764  3546 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:29.253103  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.259923  2606 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 6: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:29.259487  2607 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:29.258626  3411 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.221s (SENT)
I20260709 14:52:29.259114  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.246474  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.263545  2754 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:29.259871  5763 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 14:52:29.276894  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.261034  5777 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:29.284583  2606 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:29.244261  5773 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.299541  5773 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.301216  5773 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.287899  5787 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:29.307739  5787 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:29.313151  5698 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:29.314345  5698 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:29.314888  5698 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.315269  5725 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.315923  5698 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.157003  5772 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.317467  5698 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.307578  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.318354  2606 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:29.321810  5714 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.323854  5776 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:29.325522  5702 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:29.321765  3411 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.774s (SENT)
I20260709 14:52:29.327343  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1130) took 1473 ms. Trace:
I20260709 14:52:29.328032  2604 rpcz_store.cc:276] 0709 14:52:27.853576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.092826 (+239250us) service_pool.cc:224] Handling call
0709 14:52:29.327090 (+1234264us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540050ffd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.327268 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714363264}
I20260709 14:52:29.329452  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.327517  5702 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:29.329558  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1115) took 1646 ms (client timeout 1550 ms). Trace:
I20260709 14:52:29.333174  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:29.332998  2605 rpcz_store.cc:269] 0709 14:52:27.683454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.683675 (+   221us) service_pool.cc:224] Handling call
0709 14:52:29.329340 (+1645665us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.329486 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952531712}
I20260709 14:52:29.316529  5725 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:29.341557  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 222571us
I20260709 14:52:29.344317  2651 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9.
I20260709 14:52:29.230515  5750 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.344483  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.345130  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:29.347755  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.348348  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:29.348980  2677 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:29.349413  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.351519  2754 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:29.351622  3411 leader_election.cc:400] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:52:29.353510  3411 leader_election.cc:403] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:52:29.353528  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:29.354743  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.354553  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.355983  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.356098  2605 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.358937  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.359735  2604 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:29.359907  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.214740  5771 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.362191  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:29.244771  5482 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.363430  5782 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:29.363806  5776 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:29.365226  5576 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6.
I20260709 14:52:29.372787  2678 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:29.366432  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:29.391016  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.408761  2607 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:29.410046  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.362138  5750 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:29.330742  5492 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.414561  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:29.414594  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48316.1us
I20260709 14:52:29.417078  2607 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:29.366084  5550 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:29.370702  4989 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.373688  5694 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:29.372516  5771 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.427459  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.428584  2604 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:52:29.411441  5776 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:29.419239  5614 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:29.437703  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.447296  2607 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:29.439021  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.449409  2604 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:29.421464  5772 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:29.463500  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.464236  5492 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:29.464533  2754 raft_consensus.cc:2368] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9.
I20260709 14:52:29.465065  2612 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:52:29.465562  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:29.438802  5732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.280s	user 0.007s	sys 0.018s
W20260709 14:52:29.466806  5732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.308s	user 0.008s	sys 0.018s
I20260709 14:52:29.471923  2605 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:29.473069  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.473706  2605 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:29.436612  5694 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.451212  5614 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:29.475317  5614 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:29.475855  5694 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.477993  5482 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.482044  5371 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.421988  5550 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:29.466357  2754 raft_consensus.cc:2368] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
W20260709 14:52:29.466212  5755 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.589s	user 0.004s	sys 0.010s
I20260709 14:52:29.438414  5776 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.421103  5750 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.489295  2757 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:29.490037  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.491813  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.494349  2754 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.495558  2607 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:29.490936  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 232200us
I20260709 14:52:29.499225  2606 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:29.500468  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.501513  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 505) took 1381 ms. Trace:
I20260709 14:52:29.501902  2606 rpcz_store.cc:276] 0709 14:52:28.120533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.500151 (+1379618us) service_pool.cc:224] Handling call
0709 14:52:29.501437 (+  1286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.466303  5496 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.504910  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:29.491078  5550 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:29.507845  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:29.528261  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.517427  2604 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:29.532832  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:29.503005  5695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.507s	user 0.001s	sys 0.046s
I20260709 14:52:29.550289  2604 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:29.544287  2606 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 14:52:29.566257  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1129) took 1713 ms. Trace:
I20260709 14:52:29.568487  2606 rpcz_store.cc:276] 0709 14:52:27.853126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.502658 (+1649532us) service_pool.cc:224] Handling call
0709 14:52:29.544043 (+ 41385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:29.565926 (+ 21883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.566181 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.505173  5378 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.569521  5378 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.571914  5378 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.567917  2754 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:29.547730  5695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.563s	user 0.001s	sys 0.047s
I20260709 14:52:29.574704  5695 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.576751  5695 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.573532  2606 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:29.582778  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.582428  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1132) took 1728 ms. Trace:
I20260709 14:52:29.503470  5714 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 14:52:29.587146  2606 rpcz_store.cc:276] 0709 14:52:27.854419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.572705 (+1718286us) service_pool.cc:224] Handling call
0709 14:52:29.573131 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.581404 (+  8273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.582259 (+   855us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.580596  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.588610  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:29.589941  2754 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:29.590015  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.583882  2685 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:29.558554  2683 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:29.593272  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:29.592895  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.554137  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1131) took 1700 ms. Trace:
I20260709 14:52:29.594393  2604 rpcz_store.cc:276] 0709 14:52:27.854047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.541430 (+1687383us) service_pool.cc:224] Handling call
0709 14:52:29.541949 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.552472 (+ 10523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.554040 (+  1568us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.557801  2605 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:29.596083  2610 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:29.597805  2681 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
W20260709 14:52:29.553592  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.494053  5755 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.543335  5492 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:29.603395  5725 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:29.548095  5380 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:29.631841  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:29.641108  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:29.603960  5685 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:29.604738  2754 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:29.586587  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.649292  2680 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:29.611141  5763 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.611755  5698 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:29.612095  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:29.652050  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.652398  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.653048  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.653716  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:29.619279  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:29.654733  2682 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:29.624384  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.624591  3546 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:29.625141  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:29.625453  3540 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:29.628942  5714 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:29.630004  5731 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.558789  5750 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.630745  5793 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:29.631269  3547 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:29.649499  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 535) took 2161 ms. Trace:
W20260709 14:52:29.633733  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1124) took 1819 ms (client timeout 2033 ms). Trace:
I20260709 14:52:29.645300  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.643461  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1129) took 1435 ms. Trace:
I20260709 14:52:29.665253  2754 rpcz_store.cc:276] 0709 14:52:27.487946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.603922 (+2115976us) service_pool.cc:224] Handling call
0709 14:52:29.604472 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.649104 (+ 44632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.649423 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:29.665493  2612 rpcz_store.cc:269] 0709 14:52:27.814892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.827319 (+ 12427us) service_pool.cc:224] Handling call
0709 14:52:29.633477 (+1806158us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.633663 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3568727936}
I20260709 14:52:29.651309  3542 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:29.640515  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.668776  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.670418  2608 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:29.666021  2603 raft_consensus.cc:2376] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9. Re-sending same reply.
I20260709 14:52:29.671049  2682 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 14:52:29.671286  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1139) took 1150 ms. Trace:
I20260709 14:52:29.671298  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:29.601840  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:29.505126  5732 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.122s
I20260709 14:52:29.666131  2760 rpcz_store.cc:276] 0709 14:52:28.208350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.642182 (+1433832us) service_pool.cc:224] Handling call
0709 14:52:29.642640 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.642960 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:29.643000 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:29.643069 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:29.643122 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.643202 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:29.643377 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.585747  5371 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.655973  2685 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:29.673297  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:29.602095  5563 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:29.671734  2603 rpcz_store.cc:276] 0709 14:52:28.521289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.953845 (+432556us) service_pool.cc:224] Handling call
0709 14:52:29.645198 (+691353us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b5c00141c8c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa171fb0161 00007fa172151a79 00007fa17360b9b6 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.671206 (+ 26008us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155175808}
I20260709 14:52:29.659672  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:29.673206  5789 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:29.683399  5789 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:29.658555  5714 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:29.674746  4989 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:29.693593  4989 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.694628  2755 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:29.694782  4989 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.683921  5789 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.697216  5788 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:29.697700  5788 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.698662  5788 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.700026  5788 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:29.699654  5596 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:29.701798  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 518) took 2580 ms (client timeout 2774 ms). Trace:
I20260709 14:52:29.675060  5772 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:29.702642  5772 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:29.702461  2759 rpcz_store.cc:269] 0709 14:52:27.121055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.415419 (+294364us) service_pool.cc:224] Handling call
0709 14:52:29.701427 (+2286008us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.701664 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347894144}
I20260709 14:52:29.703312  5772 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.705060  2759 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:29.675935  5703 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.660197  5763 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.663957  5793 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:29.720274  5793 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:29.648049  5492 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:29.726143  5492 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:29.727687  5088 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:29.728250  5088 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.729331  5088 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.730423  5088 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:29.730967  5755 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.735134  5563 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.736124  4786 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:29.736784  4786 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:29.664944  5725 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.737936  4786 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.740309  5725 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.741671  2752 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:29.637110  5776 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.743216  2603 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:29.681389  2754 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:29.743906  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 506) took 1623 ms. Trace:
I20260709 14:52:29.744202  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1132) took 1253 ms. Trace:
I20260709 14:52:29.744266  2603 rpcz_store.cc:276] 0709 14:52:28.120842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.742262 (+1621420us) service_pool.cc:224] Handling call
0709 14:52:29.742854 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.743628 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.743844 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.744696  2754 rpcz_store.cc:276] 0709 14:52:28.491350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.666889 (+1175539us) service_pool.cc:224] Handling call
0709 14:52:29.681010 (+ 14121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.743946 (+ 62936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.744147 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.745682  5718 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:29.746047  2603 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 14:52:29.682384  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80648.8us
I20260709 14:52:29.748391  2752 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:29.685497  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:29.750103  5776 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:29.750851  5776 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:29.752117  5776 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:29.757469  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:29.757963  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1128) took 1633 ms (client timeout 2057 ms). Trace:
I20260709 14:52:29.686555  2760 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:29.688750  5222 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:29.689486  5790 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:29.690526  5771 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:29.681483  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:29.660760  5589 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:29.740602  4786 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:29.659405  5698 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:29.765002  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1133) took 1176 ms. Trace:
I20260709 14:52:29.777101  2760 rpcz_store.cc:276] 0709 14:52:28.589035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.680801 (+1091766us) service_pool.cc:224] Handling call
0709 14:52:29.686217 (+  5416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.764543 (+ 78326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.764910 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.678210  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 504) took 1558 ms. Trace:
I20260709 14:52:29.778446  2612 rpcz_store.cc:276] 0709 14:52:28.120147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.667251 (+1547104us) service_pool.cc:224] Handling call
0709 14:52:29.677626 (+ 10375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.677843 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:29.677870 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:29.677913 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:29.677951 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.678005 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:29.678143 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:29.758488  2609 rpcz_store.cc:269] 0709 14:52:28.124149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.609906 (+485757us) service_pool.cc:224] Handling call
0709 14:52:29.757734 (+1147828us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b540016dfd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.757892 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614339968}
W20260709 14:52:29.767503  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.779539  2685 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
W20260709 14:52:29.782475  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:29.768471  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83082.4us
I20260709 14:52:29.759780  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.794622  3411 leader_election.cc:400] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9.
I20260709 14:52:29.795125  3411 leader_election.cc:403] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 14:52:29.796250  3411 leader_election.cc:400] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:29.796654  3411 leader_election.cc:403] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:52:29.771632  2752 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:29.746743  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 509) took 1625 ms. Trace:
I20260709 14:52:29.799000  2603 rpcz_store.cc:276] 0709 14:52:28.121785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.745341 (+1623556us) service_pool.cc:224] Handling call
0709 14:52:29.745692 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.746460 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.746689 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:29.787982  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106652us
I20260709 14:52:29.783799  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:29.782353  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 124381us
I20260709 14:52:29.768052  2755 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:29.821972  2752 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:29.772333  5680 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:29.801599  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:29.775032  5746 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:29.806844  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.785094  5698 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:29.838903  5596 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:29.823833  5790 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:29.848582  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1053) took 1105 ms. Trace:
W20260709 14:52:29.852267  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.852854  5496 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:29.890584  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 229467us
I20260709 14:52:29.868359  5702 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:29.856237  5614 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.832194  5222 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.858263  2605 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:29.860423  5763 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.864900  5789 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.800276  5576 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:29.801815  5589 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9.
I20260709 14:52:29.800310  5599 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 14:52:29.798085  5771 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:29.858554  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1035) took 1005 ms. Trace:
I20260709 14:52:29.964447  2755 rpcz_store.cc:276] 0709 14:52:28.853714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.709441 (+855727us) service_pool.cc:224] Handling call
0709 14:52:29.710099 (+   658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:29.835060 (+124961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.858444 (+ 23384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:29.853180  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:29.872284  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:29.965775  2759 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:29.874862  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 507) took 1753 ms (client timeout 2092 ms). Trace:
W20260709 14:52:29.965952  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112912us
W20260709 14:52:29.966194  2611 rpcz_store.cc:269] 0709 14:52:28.121170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.666020 (+544850us) service_pool.cc:224] Handling call
0709 14:52:29.874602 (+1208582us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54000ac154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:29.874754 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242429824}
I20260709 14:52:29.875289  2606 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:29.875793  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.882640  5380 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:29.883021  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:29.811465  5563 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:29.885059  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.866802  5492 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.894243  5788 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:29.866909  5723 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:29.858294  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:29.889387  2678 rpcz_store.cc:276] 0709 14:52:28.743702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.743869 (+   167us) service_pool.cc:224] Handling call
0709 14:52:28.744359 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.389053 (+644694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.848214 (+459161us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:29.848493 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444606208}
I20260709 14:52:29.995872  5614 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:29.945458  5697 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:29.997498  5697 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.999181  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:29.999962  2611 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:29.994247  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 136130us
W20260709 14:52:29.928239  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:30.003206  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:29.854568  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70836.4us
W20260709 14:52:29.946363  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:29.948573  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1134) took 2082 ms. Trace:
I20260709 14:52:30.004666  2605 rpcz_store.cc:276] 0709 14:52:27.866185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.597436 (+1731251us) service_pool.cc:224] Handling call
0709 14:52:29.659305 (+ 61869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:29.857986 (+198681us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b5400124420. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:52:29.948216 (+ 90230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:29.948499 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150930688}
I20260709 14:52:30.006016  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:29.870779  5620 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.976533  2752 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:29.968109  2612 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:29.885067  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83564.5us
I20260709 14:52:29.967989  5222 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:29.979583  5789 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:29.979980  5771 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.025233  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:29.981532  5380 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:29.977236  5766 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:29.984638  5198 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.961625  5778 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 14:52:29.878485  2680 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:29.964013  5800 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:29.890882  5746 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:29.974802  5702 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:30.020736  2753 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:30.042294  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:30.047734  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59780.1us
I20260709 14:52:30.051043  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:30.053320  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.054611  3537 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:30.022923  2605 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:29.962007  5694 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 14:52:29.892618  5680 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:30.035624  2760 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:30.036263  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.065135  2611 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.038121  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.008853  5371 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:30.055521  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 512) took 1932 ms. Trace:
I20260709 14:52:30.008688  5788 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:30.075312  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 182760us
I20260709 14:52:30.068168  2751 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 14:52:30.069563  2605 rpcz_store.cc:276] 0709 14:52:28.122680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.018549 (+1895869us) service_pool.cc:224] Handling call
0709 14:52:30.022515 (+  3966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.055258 (+ 32743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.055469 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.084345  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.034518  5405 raft_consensus.cc:2764] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 14:52:30.091557  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.096419  3545 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:30.028565  5766 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:30.096014  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1136) took 2229 ms. Trace:
I20260709 14:52:30.100051  3547 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 14:52:30.099758  2606 rpcz_store.cc:276] 0709 14:52:27.867086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.656805 (+1789719us) service_pool.cc:224] Handling call
0709 14:52:29.875096 (+218291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.967554 (+ 92458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.095759 (+128205us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400346050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.095942 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470446976}
I20260709 14:52:30.101049  3547 pending_rounds.cc:85] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:52:30.101507  3547 pending_rounds.cc:107] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 14:52:30.101838  3547 raft_consensus.cc:2889] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:30.094134  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:30.040395  5763 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:30.039371  5799 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:30.114244  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58127us
I20260709 14:52:30.118767  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:30.068661  5734 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:30.034914  5599 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 14:52:30.120271  2607 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.065294  5718 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:30.121266  2606 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
W20260709 14:52:30.108650  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 105602us
I20260709 14:52:30.150810  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1126) took 2027 ms. Trace:
W20260709 14:52:30.121601  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 79468.4us
I20260709 14:52:30.151283  2606 rpcz_store.cc:276] 0709 14:52:28.123583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.101700 (+1978117us) service_pool.cc:224] Handling call
0709 14:52:30.121079 (+ 19379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.150478 (+ 29399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.150733 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.075920  5809 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:30.105014  5766 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:30.215190  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:30.228037  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76511.8us
I20260709 14:52:30.047405  5800 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.229070  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:30.229815  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77706.2us
I20260709 14:52:30.083428  5620 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 14:52:30.116236  5378 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.111191  5732 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.232481  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.039976  5198 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.112507  5767 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.092255  5755 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 14:52:30.240753  2611 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:30.109094  5763 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:30.259173  2607 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:30.228214  5809 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:30.231719  5088 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:30.257822   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:52:30.251822  5732 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.275974  5222 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:30.277211  5222 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:30.277823  5222 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.116946  5714 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.268353  5788 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.281483  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 961) took 1462 ms. Trace:
I20260709 14:52:30.281968  2684 rpcz_store.cc:276] 0709 14:52:28.819310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.886292 (+ 66982us) service_pool.cc:224] Handling call
0709 14:52:30.281206 (+1394914us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.281407 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684768000}
I20260709 14:52:30.262679  5697 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:30.280006  5799 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 14:52:30.283855  5492 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:30.284559  5198 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:30.284689  5492 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:30.285924  5492 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:30.287222  5763 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.256932  5680 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.263026  5767 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.292306  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:30.292685  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.122433  5772 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.237419  5718 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:30.301919  5816 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:30.119185  5506 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.133s
I20260709 14:52:30.227214  5702 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.304978  5222 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.305130  5718 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:30.277872  5798 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 14:52:30.301043  5550 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.002s
I20260709 14:52:30.320983  5550 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.279950  5683 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.324116  5550 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:30.325052  5746 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:30.254123  5378 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.328449  2606 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:30.312641  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.330396  3547 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:30.303742  5799 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:30.303997  5697 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:30.327400  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:30.320091  4786 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.004s
I20260709 14:52:30.290660  5088 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:30.336304  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 93976us
I20260709 14:52:30.339473  2611 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:52:30.296376  5809 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:30.274755  5732 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:30.360415  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:30.325045  5811 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:30.306800  5755 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 14:52:30.348270  5697 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:30.404958  5813 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.404459  5088 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.399575  4786 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 14:52:30.416155  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 297664us
I20260709 14:52:30.417726  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:30.395548  5620 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:30.396044  5811 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.396278  5799 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.423102  2611 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 4.
W20260709 14:52:30.425043  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 196081us
I20260709 14:52:30.422225  2606 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:30.415468  5697 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:30.429560  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1037) took 1552 ms. Trace:
I20260709 14:52:30.444871  3543 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:30.421758  4786 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.458495  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.459600  2611 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:30.453440  3543 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:30.438591  2754 rpcz_store.cc:276] 0709 14:52:28.877540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.745538 (+867998us) service_pool.cc:224] Handling call
0709 14:52:29.745866 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.746100 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:30.029587 (+283487us) log.cc:844] Serialized 14 byte log entry
0709 14:52:30.429130 (+399543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:30.429204 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:30.429261 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.429346 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:30.429496 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12529,"prepare.queue_time_us":347758,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":186692,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"thread_start_us":681956,"threads_started":3,"wal-append.queue_time_us":51983}
I20260709 14:52:30.428812  5620 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:30.466264  3545 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:30.463090  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:30.467195  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 373088us
W20260709 14:52:30.463644  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:30.468640  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:30.434680  5380 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:30.470281  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:30.470892  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:30.420475  5813 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.431351  5695 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 14:52:30.461838  5800 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:30.472795  5793 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.749s	user 0.017s	sys 0.000s
I20260709 14:52:30.470566  5620 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:30.486637  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 542) took 1144 ms. Trace:
I20260709 14:52:30.490233  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 522) took 1109 ms. Trace:
W20260709 14:52:30.478166  5698 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.004s
I20260709 14:52:30.494639  2757 rpcz_store.cc:276] 0709 14:52:29.342697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.344294 (+  1597us) service_pool.cc:224] Handling call
0709 14:52:30.486395 (+1142101us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001ff9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.486557 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263652224}
I20260709 14:52:30.494818  2607 rpcz_store.cc:276] 0709 14:52:29.380828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.499092 (+118264us) service_pool.cc:224] Handling call
0709 14:52:30.489959 (+990867us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54002fe9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.490133 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329771264}
I20260709 14:52:30.493119  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1133) took 2638 ms. Trace:
I20260709 14:52:30.496613  2610 rpcz_store.cc:276] 0709 14:52:27.854810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:29.579855 (+1725045us) service_pool.cc:224] Handling call
0709 14:52:29.595902 (+ 16047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.596446 (+   544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.492840 (+896394us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54002fe4d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.493038 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36015360}
I20260709 14:52:30.508152  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.474287  5818 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:30.476166  5816 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.509338  2607 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:30.482873  5811 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.519914  2611 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:30.524068  5805 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 14:52:30.476847  5380 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:30.528064  5596 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.003s
I20260709 14:52:30.511253  4786 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.530243  5800 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:52:30.531422  5821 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:30.531796  5807 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 14:52:30.534453  5748 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:30.545523  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:30.543380  3536 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:30.551023  5734 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.553126  5813 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:30.536123  5825 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.535796  5799 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.555487  5772 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:30.596311  5811 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:30.556139  5818 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.494881  5793 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.559893  5739 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:30.568912  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 100456us
W20260709 14:52:30.582720  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104984us
I20260709 14:52:30.545779  5576 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:30.608435  2607 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:30.509773  5698 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:30.614187  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 963) took 1794 ms (client timeout 2176 ms). Trace:
I20260709 14:52:30.555701  5378 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:30.591444  5777 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.566063  5816 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.609176  4786 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:30.691370  2683 rpcz_store.cc:269] 0709 14:52:28.819852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.888086 (+ 68234us) service_pool.cc:224] Handling call
0709 14:52:30.613919 (+1725833us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.614104 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453458816}
I20260709 14:52:30.569738  5810 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 14:52:30.598943  5821 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:30.621127  2604 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
W20260709 14:52:30.699620  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61439.2us
I20260709 14:52:30.700004  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1131) took 2575 ms. Trace:
I20260709 14:52:30.700656  2604 rpcz_store.cc:276] 0709 14:52:28.125053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.620472 (+2495419us) service_pool.cc:224] Handling call
0709 14:52:30.620902 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.699565 (+ 78663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.699914 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.625744  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1130) took 2501 ms. Trace:
W20260709 14:52:30.625931  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:30.630406  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:30.600610  5831 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:30.632616  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:30.691566  2607 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:30.636242  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:30.711669  2756 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:30.713544  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1132) took 2588 ms. Trace:
I20260709 14:52:30.702598  2610 rpcz_store.cc:276] 0709 14:52:28.124762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.609739 (+2484977us) service_pool.cc:224] Handling call
0709 14:52:30.622457 (+ 12718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.622696 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:30.622725 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:30.622772 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:30.622806 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.625519 (+  2713us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:30.625674 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.692317  5732 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.618896  5718 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:30.720571  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:30.721789  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:30.706736  5816 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:30.707516  5821 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.620409  5380 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:30.705343  2604 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:30.714107  2607 rpcz_store.cc:276] 0709 14:52:28.125575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.690625 (+2565050us) service_pool.cc:224] Handling call
0709 14:52:30.691302 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.713163 (+ 21861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.713463 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.717039  2608 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:30.705281  5828 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 14:52:30.735658  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1140) took 2172 ms. Trace:
I20260709 14:52:30.736171  2604 rpcz_store.cc:276] 0709 14:52:28.563094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.704553 (+2141459us) service_pool.cc:224] Handling call
0709 14:52:30.705065 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:30.735262 (+ 30197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.735589 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:30.717636  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81502.6us
I20260709 14:52:30.636420  5685 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:30.722376  2683 raft_consensus.cc:1217] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[7.5-11.7]   Dedup: 7.5->[9.6-11.7]
I20260709 14:52:30.759382  2683 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
I20260709 14:52:30.723811  2684 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:52:30.727941  2610 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:30.728189  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:30.734334  2685 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:30.745007  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1142) took 1587 ms. Trace:
I20260709 14:52:30.714241  5760 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.625002  5766 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.632937  5822 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.180s
W20260709 14:52:30.806027  5830 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 14:52:30.755926  3536 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:30.762751  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1142) took 2137 ms. Trace:
I20260709 14:52:30.869338  2610 rpcz_store.cc:276] 0709 14:52:28.624804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.714618 (+2089814us) service_pool.cc:224] Handling call
0709 14:52:30.715089 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.762293 (+ 47204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.762581 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.749629  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:30.767223  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:30.738579  5596 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.734809  5831 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.874542  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:30.708369  5778 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:30.806159  5822 consensus_queue.cc:1243] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:30.882846  5778 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:30.883304  5750 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:30.807368  5739 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.883847  5750 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:30.883886  5778 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.753676  5746 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.885258  5778 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:30.884989  5750 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.885172  5746 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.616449  5825 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.886879  5750 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:30.806440  5755 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.887703  5755 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.888852  5825 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:30.611137  5734 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.752482  2686 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:30.763703  2608 rpcz_store.cc:276] 0709 14:52:29.157269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.716409 (+1559140us) service_pool.cc:224] Handling call
0709 14:52:30.716766 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.744628 (+ 27862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.744928 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.892041  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 524) took 3721 ms. Trace:
I20260709 14:52:30.892732  2753 rpcz_store.cc:276] 0709 14:52:27.170586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.467938 (+1297352us) service_pool.cc:224] Handling call
0709 14:52:29.706453 (+1238515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.044102 (+337649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.566350 (+522248us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:30.891806 (+325456us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002bf7d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:30.891969 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3569532032}
I20260709 14:52:30.831207  5760 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.832387  5793 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:30.896152  2686 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 14:52:30.832828  2604 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [5.3-8.4]
I20260709 14:52:30.897811  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 518) took 1715 ms. Trace:
I20260709 14:52:30.898175  2604 rpcz_store.cc:276] 0709 14:52:29.182487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.737221 (+1554734us) service_pool.cc:224] Handling call
0709 14:52:30.805968 (+ 68747us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:30.897467 (+ 91499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.897720 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.899547  2604 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:30.900177  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1148) took 1626 ms. Trace:
I20260709 14:52:30.900476  2604 rpcz_store.cc:276] 0709 14:52:29.274246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.899001 (+1624755us) service_pool.cc:224] Handling call
0709 14:52:30.899355 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.899930 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.900119 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:30.811920  5380 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:30.699422  5777 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.699447  5576 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.841308  5818 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:30.759001  5685 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:30.845172  5809 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.904222  5685 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.845773  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:30.906595  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.907533  3538 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:30.846968  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:30.849905  5697 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:30.753445  5794 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:52:30.731930  5813 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:30.853271  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94599us
I20260709 14:52:30.713335  5772 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.872678  2678 raft_consensus.cc:2368] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9.
I20260709 14:52:30.877173  5596 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.880777  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1002) took 2656 ms. Trace:
I20260709 14:52:30.884310  5739 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:30.735303  5718 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:30.892223  2608 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:30.745677  5788 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:30.622112  5763 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.878335  5831 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:30.739496  5732 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:30.908146  2686 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:30.943817  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.024662  5732 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:31.019114  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:31.025971  5832 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:52:30.920071  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 254477us
I20260709 14:52:31.026645  5832 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:31.027074  2607 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:30.983263  5620 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:31.027803  5832 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:31.028650  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:30.869046  5819 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.160s
I20260709 14:52:31.029706  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1147) took 1783 ms. Trace:
I20260709 14:52:31.038246  2608 rpcz_store.cc:276] 0709 14:52:29.246232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.891565 (+1645333us) service_pool.cc:224] Handling call
0709 14:52:30.891953 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:31.029407 (+137454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.029642 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.040750  2680 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:30.909305  3538 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 14:52:30.915171  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:31.042770  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:31.043293  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:31.019833  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.043764  2608 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:31.031265  2604 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:31.030624  3542 rpcz_store.cc:276] 0709 14:52:28.223965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.230520 (+  6555us) service_pool.cc:224] Handling call
0709 14:52:28.231669 (+  1149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:29.667359 (+1435690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:30.880455 (+1213096us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400203354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:30.880696 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264177536}
I20260709 14:52:30.912453  2609 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:31.033546  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:31.043881  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.016999  5685 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:31.057816  3536 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:31.045712  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 525) took 1253 ms. Trace:
I20260709 14:52:31.064270  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1151) took 1684 ms. Trace:
I20260709 14:52:31.064472  2610 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:31.064561  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.067852  2608 rpcz_store.cc:276] 0709 14:52:29.792367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.040706 (+1248339us) service_pool.cc:224] Handling call
0709 14:52:31.041073 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:31.045400 (+  4327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.045642 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.068650  3541 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:31.069437  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.070042  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:31.070775  2678 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:31.071326  2608 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:31.045712  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1149) took 1539 ms. Trace:
I20260709 14:52:31.072028  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 527) took 1083 ms. Trace:
I20260709 14:52:31.071872  2604 rpcz_store.cc:276] 0709 14:52:29.506539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.030569 (+1524030us) service_pool.cc:224] Handling call
0709 14:52:31.030927 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.045468 (+ 14541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.045642 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.072544  2608 rpcz_store.cc:276] 0709 14:52:29.989085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.068829 (+1079744us) service_pool.cc:224] Handling call
0709 14:52:31.070972 (+  2143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:31.071738 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.071965 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.073582  2604 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 14:52:31.073957  2604 pending_rounds.cc:85] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 6
I20260709 14:52:31.074188  2604 pending_rounds.cc:107] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 14:52:31.074405  2604 raft_consensus.cc:2889] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.068514  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:31.070163  3538 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:31.022392  5746 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.076315  2608 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 14:52:31.065344  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:31.075932  4786 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:31.037086  5809 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.037834  5767 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:31.077529  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.040216  5576 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.079211  2609 rpcz_store.cc:276] 0709 14:52:29.379813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.902912 (+1523099us) service_pool.cc:224] Handling call
0709 14:52:30.903347 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.063873 (+160526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.064172 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.044994  5496 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.088317  5496 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:31.083660  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.085034  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:31.088727  2753 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 14:52:31.036378  5788 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.079051  2608 pending_rounds.cc:85] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:52:31.038014  5822 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 14:52:31.253330  5651 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:31.258446  2608 pending_rounds.cc:107] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 14:52:31.260356  2608 raft_consensus.cc:2889] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.086462  5380 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 14:52:31.256656  5651 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.089546  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.262706  2679 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:31.215884  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:31.070600  5683 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:31.184430  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.264271  2755 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:31.167802  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:31.088747  5596 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:31.285714  5842 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:52:31.167780  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.285465  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:31.195529  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1054) took 2424 ms (client timeout 1815 ms). Trace:
W20260709 14:52:31.288059  2681 rpcz_store.cc:269] 0709 14:52:28.744083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.746952 (+  2869us) service_pool.cc:224] Handling call
0709 14:52:31.168390 (+2421438us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:31.168597 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568519936}
I20260709 14:52:31.288296  2678 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:31.292511  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1151) took 1193 ms. Trace:
I20260709 14:52:31.293377  2608 rpcz_store.cc:276] 0709 14:52:30.102390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.073357 (+970967us) service_pool.cc:224] Handling call
0709 14:52:31.073647 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.292088 (+218441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.292420 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:31.168597  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.153928  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:31.328311  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 246123us
I20260709 14:52:31.037261  5772 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:31.338634  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:31.339236  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.339756  2755 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:31.339993  3541 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:31.079646  5835 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 14:52:31.281265  5767 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.078428  5823 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.083119  5697 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:31.354805  5811 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.000s
I20260709 14:52:31.367738  4786 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:31.154309  5755 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.087333  5698 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.186895  5809 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 14:52:31.401547  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.367338  2678 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:31.256420  5821 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:31.373232  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:31.407891  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1127) took 3284 ms. Trace:
I20260709 14:52:31.407594  3541 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:31.409318  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:31.408391  2611 rpcz_store.cc:276] 0709 14:52:28.123867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.570463 (+2446596us) service_pool.cc:224] Handling call
0709 14:52:30.571087 (+   624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:30.571694 (+   607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:30.571738 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:30.586287 (+ 14549us) log.cc:844] Serialized 26 byte log entry
0709 14:52:30.874877 (+288590us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:30.905907 (+ 31030us) log.cc:844] Serialized 14 byte log entry
0709 14:52:31.024024 (+118117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.024104 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:31.407603 (+383499us) raft_consensus.cc:1685] finished
0709 14:52:31.407623 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:31.407817 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29116,"prepare.queue_time_us":30055,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":5885,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":32551,"thread_start_us":244889,"threads_started":1,"wal-append.queue_time_us":467019}
I20260709 14:52:31.413656  5772 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.414933  5823 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.375201  3542 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:31.375962  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.420017  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.421340  2757 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:31.371322  5767 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:31.422593  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.423987  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.424800  2757 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:31.425010  5767 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.380565  4786 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:31.384496  5788 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.384920  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:31.299917  5748 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 14:52:31.431170  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.434619  3546 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:31.438243  3542 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:31.444589  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:31.437556  5823 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.448973  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.475070  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:31.454116  5750 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.567s	user 0.014s	sys 0.000s
I20260709 14:52:31.433908  5763 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:31.439332  5620 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.449270  5739 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.452912  5496 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.429446  5714 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:31.502808  3541 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:31.483507  5811 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.531836  5811 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:31.532934  5734 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.534255  5088 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:31.534749  5760 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.513291  3542 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 14:52:31.520062  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1125) took 3397 ms. Trace:
W20260709 14:52:31.522696  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.523619  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:31.540371  2605 rpcz_store.cc:276] 0709 14:52:28.123271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.086634 (+1963363us) service_pool.cc:224] Handling call
0709 14:52:30.115260 (+ 28626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:30.119249 (+  3989us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:30.589055 (+469806us) log.cc:844] Serialized 14 byte log entry
0709 14:52:30.873850 (+284795us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:31.025689 (+151839us) log.cc:844] Serialized 26 byte log entry
0709 14:52:31.025870 (+   181us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:31.026107 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 14:52:31.031411 (+  5304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.031493 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:31.519674 (+488181us) raft_consensus.cc:1685] finished
0709 14:52:31.519712 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:31.519959 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5592,"prepare.queue_time_us":501317,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":128,"raft.queue_time_us":15281,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":58044,"spinlock_wait_cycles":890240,"thread_start_us":595060,"threads_started":2,"wal-append.queue_time_us":728072}
I20260709 14:52:31.543843  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:31.554200  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:31.558306  2755 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:31.562948  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:31.563212  2759 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: [3.2-6.4]
I20260709 14:52:31.563835  2605 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:31.527755  5763 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:31.589433  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:31.593636  5813 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:31.594271  5813 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:31.595266  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:31.596179  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:31.597916  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.841s (SENT) [suppressed 25 similar messages]
I20260709 14:52:31.598205  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1153) took 1495 ms. Trace:
I20260709 14:52:31.564276  5788 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.599833  2604 rpcz_store.cc:276] 0709 14:52:30.103266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.082512 (+979246us) service_pool.cc:224] Handling call
0709 14:52:31.082874 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.083129 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:31.083408 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 14:52:31.379804 (+296396us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b18003ad560. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:52:31.597647 (+217843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:31.597793 (+   146us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:31.597872 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.597971 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:31.598128 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":20043,"raft.queue_time_us":111,"spinlock_wait_cycles":9096960,"thread_start_us":217533,"threads_started":1}
I20260709 14:52:31.566552  5739 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:31.607704  3541 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:31.587210  5620 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:31.566743  5734 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:31.607081  5739 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:31.604060  5845 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 14:52:31.625276  2605 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:31.598405  5822 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.623533  5739 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:31.586794  5767 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 14:52:31.638702  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1122) took 4305 ms. Trace:
I20260709 14:52:31.652110  2751 rpcz_store.cc:276] 0709 14:52:27.333219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.684625 (+1351406us) service_pool.cc:224] Handling call
0709 14:52:28.685115 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:30.085835 (+1400720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.638278 (+1552443us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.638611 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3179637120}
W20260709 14:52:31.637135  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1154) took 1533 ms (client timeout 1841 ms). Trace:
W20260709 14:52:31.654239  2606 rpcz_store.cc:269] 0709 14:52:30.103617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.152218 (+ 48601us) service_pool.cc:224] Handling call
0709 14:52:31.636798 (+1484580us) spinlock_profiling.cc:243] Waited 83.7 ms on lock 0x7b54003c7b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:31.637008 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192621824}
W20260709 14:52:31.637900  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.639016  5678 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.639196  2611 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:31.641733  3546 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 14:52:31.642678  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:31.672752  3546 pending_rounds.cc:85] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:31.673534  3546 pending_rounds.cc:107] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:52:31.673952  3546 raft_consensus.cc:2889] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.645601  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:31.645892  5651 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.647038  5750 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:31.647990  5816 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:31.649109  5818 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:31.649880  5698 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:31.666285  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:31.728444  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1146) took 2512 ms. Trace:
I20260709 14:52:31.650899  5772 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.615218  5496 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.632957  5620 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:31.597934  5763 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.705155  5703 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:31.696524  5678 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 14:52:31.689499  5725 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:31.746337  5703 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:31.672776  5822 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:31.709774  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.604431  5788 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:31.715603  5852 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 14:52:31.717780  3543 pending_rounds.cc:85] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 3
I20260709 14:52:31.710486  5777 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.722229  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1153) took 1717 ms. Trace:
I20260709 14:52:31.726985  5088 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:31.728924  2610 rpcz_store.cc:276] 0709 14:52:29.216638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.870668 (+1654030us) service_pool.cc:224] Handling call
0709 14:52:30.873543 (+  2875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.068085 (+194542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.728136 (+660051us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54003c8554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.728366 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438543616}
I20260709 14:52:31.622421  5734 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:31.685307  2605 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:31.728653  3541 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:31.746089  5772 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.625000  5813 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.689805  5819 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:31.864301  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 522) took 4726 ms (client timeout 2221 ms). Trace:
I20260709 14:52:31.913512  2608 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 14:52:31.919214  2608 pending_rounds.cc:85] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 3
I20260709 14:52:31.919632  2608 pending_rounds.cc:107] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 14:52:31.919997  2608 raft_consensus.cc:2889] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.923162  2612 rpcz_store.cc:276] 0709 14:52:30.004977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.019818 (+ 14841us) service_pool.cc:224] Handling call
0709 14:52:31.721977 (+1702159us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540016dfd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:31.722154 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255880064}
W20260709 14:52:31.858001  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 260089us
I20260709 14:52:31.807642  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1145) took 2648 ms. Trace:
I20260709 14:52:31.925745  2607 rpcz_store.cc:276] 0709 14:52:29.159042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.735682 (+1576640us) service_pool.cc:224] Handling call
0709 14:52:30.833647 (+ 97965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.027567 (+193920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.807323 (+779756us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.807547 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444345088}
I20260709 14:52:31.864637  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 525) took 1143 ms. Trace:
I20260709 14:52:31.864557  2758 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 14:52:31.926978  2685 rpcz_store.cc:276] 0709 14:52:30.720992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.724569 (+  3577us) service_pool.cc:224] Handling call
0709 14:52:30.725014 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:30.763305 (+ 38291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.864370 (+1101065us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.864564 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20787456}
W20260709 14:52:31.800448  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.007s
I20260709 14:52:31.927197  2758 pending_rounds.cc:85] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
W20260709 14:52:31.928884  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.000s	sys 0.011s
I20260709 14:52:31.929267  2758 pending_rounds.cc:107] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:52:31.929733  2758 raft_consensus.cc:2889] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.931644  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1021) took 1084 ms. Trace:
I20260709 14:52:31.932195  3536 rpcz_store.cc:276] 0709 14:52:30.847225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.868410 (+ 21185us) service_pool.cc:224] Handling call
0709 14:52:30.868887 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.067210 (+198323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.713918 (+646708us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002017d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.931568 (+217650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433884416}
I20260709 14:52:31.934074  3543 pending_rounds.cc:107] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 14:52:31.934484  3543 raft_consensus.cc:2889] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:31.763424  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1156) took 1257 ms. Trace:
I20260709 14:52:31.922438  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:31.919019  2756 rpcz_store.cc:269] 0709 14:52:27.137705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:27.385814 (+248109us) service_pool.cc:224] Handling call
0709 14:52:31.863928 (+4478114us) spinlock_profiling.cc:243] Waited 3.28 s on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:31.864171 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7534563712}
I20260709 14:52:31.942363  4786 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.943092  4786 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:31.945035  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:31.945859  3409 connection.cc:256] RPC call timeout handler was delayed by -1.3662854819999666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 14:52:31.946584  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.955251  3540 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 14:52:31.868829  5685 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.792s	user 0.015s	sys 0.000s
W20260709 14:52:31.932435  5778 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.047s	user 0.012s	sys 0.004s
I20260709 14:52:31.759290  5763 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.759779  5725 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:31.972283  5725 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:31.972471  5506 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.973129  5506 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:31.762030  5678 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.960740  3541 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:31.881925  5831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.011s	sys 0.006s
W20260709 14:52:31.975565  5831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.012s	sys 0.007s
I20260709 14:52:31.975173  5678 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:31.965302  3546 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 14:52:31.759002  5620 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:31.977056  5703 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:31.978495  5703 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:31.981859  5811 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.935519  2609 rpcz_store.cc:276] 0709 14:52:30.506490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.086329 (+579839us) service_pool.cc:224] Handling call
0709 14:52:31.274940 (+188611us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:31.301146 (+ 26206us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:31.338504 (+ 37358us) log.cc:844] Serialized 14 byte log entry
0709 14:52:31.595388 (+256884us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:52:31.622905 (+ 27517us) log.cc:844] Serialized 78 byte log entry
0709 14:52:31.623216 (+   311us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:31.623568 (+   352us) log.cc:844] Serialized 14 byte log entry
0709 14:52:31.623856 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.623986 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:31.763029 (+139043us) raft_consensus.cc:1685] finished
0709 14:52:31.763064 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:31.763335 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25519,"prepare.queue_time_us":51063,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":108,"raft.queue_time_us":49011,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":30312,"thread_start_us":219412,"threads_started":1,"wal-append.queue_time_us":358120}
I20260709 14:52:31.983544  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:31.954664  5755 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:31.805269  5718 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.970278  5822 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:31.939699  5714 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 14:52:31.760486  5496 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:31.760222  5821 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:31.951515  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:32.077744  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.924726  5734 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:31.933887  5819 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:32.093884  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.985101  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:32.111222  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.112123  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:32.113158  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:32.115069  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 130055us
I20260709 14:52:32.116940  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 875) took 1040 ms. Trace:
I20260709 14:52:32.117409  3540 rpcz_store.cc:276] 0709 14:52:31.076862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.639502 (+562640us) service_pool.cc:224] Handling call
0709 14:52:31.639939 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.955845 (+315906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.116632 (+160787us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:32.116866 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723775744}
I20260709 14:52:31.993185  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.120860  2755 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:32.122927  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.003319  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:31.991086  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:32.124576  2759 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:32.003571  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1157) took 1440 ms. Trace:
I20260709 14:52:31.992081  2760 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:31.995412  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.131968  2754 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:31.991462  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1157) took 1456 ms. Trace:
I20260709 14:52:32.132925  2608 rpcz_store.cc:276] 0709 14:52:30.535231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.298372 (+763141us) service_pool.cc:224] Handling call
0709 14:52:31.298880 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.920555 (+621675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.991160 (+ 70605us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540042dbd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:31.991398 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412055424}
W20260709 14:52:32.007635  5854 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.384s	user 0.003s	sys 0.001s
I20260709 14:52:32.004976  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:32.097447  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.136169  5714 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.136924  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.136875  5714 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:32.138487  2610 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 14:52:32.139037  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:32.139856  2607 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:32.138960  2435 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 5, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:32.139915  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.140647  2610 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:32.141130  2435 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 5, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:32.016414  5788 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.142314  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:32.098949  2757 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:32.110103  5718 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.106750  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.133622  5819 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:32.004506  5766 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.143271  5748 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:32.143802  5818 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:31.986197  5799 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:32.141734  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.009024  5772 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:31.992476  5752 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:32.131325  2611 rpcz_store.cc:276] 0709 14:52:30.563389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.423523 (+860134us) service_pool.cc:224] Handling call
0709 14:52:31.638950 (+215427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.672658 (+ 33708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.003224 (+330566us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54003c7b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:32.003469 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494240000}
I20260709 14:52:32.108449  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 532) took 4762 ms. Trace:
I20260709 14:52:32.198860  2755 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:32.093086  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141676us
I20260709 14:52:32.199091  2758 rpcz_store.cc:276] 0709 14:52:27.346436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:28.696945 (+1350509us) service_pool.cc:224] Handling call
0709 14:52:29.614525 (+917580us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:31.864102 (+2249577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.939873 (+ 75771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.108102 (+168229us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400570b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:32.108347 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5196671744}
I20260709 14:52:32.198431  2610 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:32.200899  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:31.987653  5811 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:31.986865  5685 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.201035  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.203589  4786 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:32.209635  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 468510us
I20260709 14:52:32.207670  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:32.214448  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:32.215936  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:32.216890  2758 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:32.220041  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.220577  5822 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:32.221738  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.077420  5678 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.003587  5793 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:32.106276  5755 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:32.227769  3412 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.092s (SENT)
I20260709 14:52:31.984591  5831 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.223s
I20260709 14:52:32.132416  5697 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.230298  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:32.244314  2758 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:32.197813  5750 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:32.264712  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.264951  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:32.252242  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:32.271390  3538 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.010s	sys 0.026s
W20260709 14:52:32.271860  3538 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.010s	sys 0.026s
I20260709 14:52:32.271991  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:32.206250  5752 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:32.227384  5739 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.003s
I20260709 14:52:32.143340  5718 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:32.273475  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:32.265112  2435 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:52:32.277999  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.001s	sys 0.036s
W20260709 14:52:32.290956  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.002s	sys 0.036s
W20260709 14:52:32.293324  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:32.278105  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.276751  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.295617  2685 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:32.296505  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.289450  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:32.299360  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:32.302611  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.302914  3540 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 3.
I20260709 14:52:32.304344  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.197606  5818 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:32.311456  5818 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:32.305631  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.221794  5778 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 14:52:32.319525  2685 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:32.271023  5772 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:32.304646  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.336727  2679 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:32.336905  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:32.337651  3541 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:32.338037  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.233556  5755 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:32.339432  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:32.304527  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:32.301695  2609 raft_consensus.cc:2376] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:52:32.205423  5799 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:32.299589  3542 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 14:52:32.340456  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.298486  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.340910  3542 pending_rounds.cc:85] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:52:32.312786  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:32.251351  5822 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:32.341358  3542 pending_rounds.cc:107] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 14:52:32.341794  3542 raft_consensus.cc:2889] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:32.316959  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43609.3us
I20260709 14:52:32.343359  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.313190  2606 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:32.305855  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:32.271293  5788 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:32.320644  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.322788  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.346454  2680 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:32.324783  2685 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:32.327071  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:32.348980  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.327419  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:32.232949  5760 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 14:52:32.307686  5763 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:32.343753  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:32.359892  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.229346  5793 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:32.313684  2435 catalog_manager.cc:5697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:32.224331  5748 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:32.233798  5685 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:32.350283  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:32.354027  2575 connection.cc:256] RPC call timeout handler was delayed by -1.0703073100000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
I20260709 14:52:32.354250  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 750) took 1328 ms. Trace:
I20260709 14:52:32.357895  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:32.364043  2605 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:32.264544  5678 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 14:52:32.291883  5750 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:32.371140  2754 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:32.364113  2575 connection.cc:256] RPC call timeout handler was delayed by -1.0567174990001149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 14:52:32.373324  2575 connection.cc:256] RPC call timeout handler was delayed by -1.0370834440000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 14:52:32.229983  5811 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:32.312496  5088 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.375633  5088 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:32.377657  5811 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:32.379051  3540 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:32.380981  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:32.315006  5506 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.314167  5782 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:32.365658  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:32.374962  5698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.036s	sys 0.052s
I20260709 14:52:32.272704  5697 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.312785  5714 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.255263  5766 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 14:52:32.385053  2752 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 14:52:32.308060  5777 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 14:52:32.364467  3537 rpcz_store.cc:276] 0709 14:52:31.025789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.518347 (+492558us) service_pool.cc:224] Handling call
0709 14:52:31.518813 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.527070 (+  8257us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:31.672933 (+145863us) log.cc:844] Serialized 14 byte log entry
0709 14:52:32.353807 (+680874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:32.353887 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:32.353954 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.354035 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:32.354188 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11902,"prepare.queue_time_us":611730,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":47,"raft.queue_time_us":47049,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":811138,"threads_started":3,"wal-append.queue_time_us":543734}
I20260709 14:52:32.291046  5816 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 14:52:32.313091  5739 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.391343  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:32.391023  5698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.037s	sys 0.052s
I20260709 14:52:32.426779  3540 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:32.385898  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.007s	sys 0.017s
W20260709 14:52:32.453552  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.007s	sys 0.018s
W20260709 14:52:32.454205  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.387941  5788 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:52:32.398854  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:32.420213  5838 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.357s	user 0.006s	sys 0.012s
I20260709 14:52:32.444157  2435 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 7, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 7 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:52:32.348453  5755 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:32.351387  5799 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:32.359959  5763 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:32.455687  2612 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
I20260709 14:52:32.419839  5848 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:32.458813  2754 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:32.461277  2609 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:32.355100  5772 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.466986  2605 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 14:52:32.389662  5088 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.472225  2611 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:32.420109  5851 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.239s
I20260709 14:52:32.487584  2605 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:52:32.487524  2435 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 7, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:52:32.504338  2751 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:32.466130  5788 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:32.451639  5714 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 14:52:32.509707  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.008s	sys 0.021s
I20260709 14:52:32.517541  2609 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:32.477474  5698 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.527112  2435 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 4, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
W20260709 14:52:32.525815  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.009s	sys 0.021s
W20260709 14:52:32.518654  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 403844us
I20260709 14:52:32.519618  2754 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:32.481766  5763 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:32.393162  5752 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.493180  5778 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:32.550141  5698 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:32.555446  5860 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 14:52:32.557245  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1161) took 1070 ms. Trace:
I20260709 14:52:32.515410  5869 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.557629  5698 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.523792  5788 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 14:52:32.549486  5725 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
W20260709 14:52:32.564458  3409 connection.cc:256] RPC call timeout handler was delayed by -1.1695776489999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 14:52:32.558207  5869 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:32.557757  2603 rpcz_store.cc:276] 0709 14:52:31.486598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.629496 (+142898us) service_pool.cc:224] Handling call
0709 14:52:31.630167 (+   671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:31.633432 (+  3265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:31.633488 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:31.634267 (+   779us) log.cc:844] Serialized 26 byte log entry
0709 14:52:31.800890 (+166623us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:31.801392 (+   502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.801478 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:32.556913 (+755435us) raft_consensus.cc:1685] finished
0709 14:52:32.556938 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:32.557155 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2568,"raft.queue_time_us":2770,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"spinlock_wait_cycles":187264,"thread_start_us":165576,"threads_started":1,"wal-append.queue_time_us":689390}
I20260709 14:52:32.585866  2435 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 7, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:32.572024  2607 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:32.523790  5496 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.600246  5685 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:32.603292  5868 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.604028  5868 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:32.604835  5813 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:32.606518  5868 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.607259  5868 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:32.607743  5868 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.610127  5868 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:32.610689  5868 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.559233  5748 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.559357  5864 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.590699  5725 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.591774  5872 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:32.484869  5793 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.550595  5778 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:32.594599  5763 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.595299  5777 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.597070  5739 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:32.598155  5718 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:32.505018  5750 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.559167  5867 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:32.673346  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:32.635131  5871 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.619760  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:32.621397  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 414089us
I20260709 14:52:32.568235  5088 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:32.577293  5752 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:32.596990  5869 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:32.637961  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:32.624081  5859 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 14:52:32.644021  2435 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 7, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:52:32.674446  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:32.578958  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 442294us
I20260709 14:52:32.674412  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:32.679333  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 335712us
W20260709 14:52:32.984223  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 346375us
I20260709 14:52:32.628016  5868 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.985252  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:32.990193  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:32.987440  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 124825us
W20260709 14:52:32.991209  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 128517us
I20260709 14:52:32.991431  2435 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:32.671727  5739 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:32.990047  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 228359us
I20260709 14:52:32.695405  5793 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:33.005151  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:33.022531  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:33.028529  2435 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 6, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:32.984997  5718 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:32.759135  5782 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:32.634310  5864 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:32.653676  5777 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.637174  5872 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:32.984016  5816 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:33.070755  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.036s	sys 0.061s
I20260709 14:52:33.040464  3540 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:33.043295  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.072019  3537 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:33.044287  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.073151  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.073917  3545 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:33.047569  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:32.996752  5876 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:32.995867  5697 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.089144  2435 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:52:33.055150  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.030s	sys 0.081s
W20260709 14:52:33.091043  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.030s	sys 0.082s
W20260709 14:52:33.093969  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:33.037945  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:33.039139  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.034287  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 881) took 1948 ms. Trace:
W20260709 14:52:33.075727  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.037s	sys 0.062s
I20260709 14:52:33.077696  3546 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:33.082609  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.091s	sys 0.164s
W20260709 14:52:33.098665  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.092s	sys 0.164s
W20260709 14:52:33.098481  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 884083us
I20260709 14:52:33.098534  3546 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:33.099612  3538 rpcz_store.cc:276] 0709 14:52:31.085901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.711650 (+625749us) service_pool.cc:224] Handling call
0709 14:52:31.712123 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:31.764015 (+ 51892us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:32.272300 (+508285us) log.cc:844] Serialized 14 byte log entry
0709 14:52:32.643915 (+371615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:32.643993 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:32.644061 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:33.033937 (+389876us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:33.034007 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.034209 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":375117,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":340219,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14210,"spinlock_wait_cycles":118697728,"thread_start_us":879121,"threads_started":2,"wal-append.queue_time_us":776355}
I20260709 14:52:33.144003  2435 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 4, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:52:33.162756  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 94241us
I20260709 14:52:33.019646  5739 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.031091  5875 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:32.995280  5496 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 14:52:32.699129  5871 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.061088  5766 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:33.171852  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:33.049541  5718 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:33.105928  2609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.026s	sys 0.080s
W20260709 14:52:33.077723  5767 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.027s	sys 0.071s
I20260709 14:52:32.682700  5867 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.037014  5869 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 14:52:33.072778  5760 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.178859  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1017) took 2614 ms. Trace:
I20260709 14:52:33.226130  3547 rpcz_store.cc:276] 0709 14:52:30.564437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:30.564625 (+   188us) service_pool.cc:224] Handling call
0709 14:52:30.590224 (+ 25599us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:30.591313 (+  1089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:30.591365 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:30.592302 (+   937us) log.cc:844] Serialized 52 byte log entry
0709 14:52:31.260762 (+668460us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b180044f7a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:31.929517 (+668755us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:31.929562 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:31.929644 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:33.178396 (+1248752us) raft_consensus.cc:1685] finished
0709 14:52:33.178433 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.178741 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4603,"raft.queue_time_us":4972,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9712,"spinlock_wait_cycles":17749888,"thread_start_us":668510,"threads_started":1,"wal-append.queue_time_us":1740208}
W20260709 14:52:33.180047  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 505732us
I20260709 14:52:32.996342  5778 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.184533  2435 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:33.188131  3546 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.169858  5703 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:33.098155  5380 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.071460  5698 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.107s
W20260709 14:52:33.200343  2609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.032s	sys 0.104s
W20260709 14:52:33.182574  5819 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.977s	user 0.016s	sys 0.000s
I20260709 14:52:33.206903  5885 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:33.245862  5885 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:33.176527  5880 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:33.190716  5877 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:33.234354  2435 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:33.236781  5760 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:33.089496  5816 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.197999  5772 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.098367  5876 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:33.203903  5697 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:33.207414  5883 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:33.241935  5851 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:33.234498  5748 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:33.300724  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 310587us
I20260709 14:52:33.240163  5725 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:33.204103  5887 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:33.201344  5767 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.027s	sys 0.072s
I20260709 14:52:33.195725  5766 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:33.263625  5885 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:33.326794  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.024s	sys 0.048s
I20260709 14:52:33.285940  5778 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:33.318118  5851 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:33.333040  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 332497us
I20260709 14:52:33.346253  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:33.335085  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.024s	sys 0.048s
W20260709 14:52:33.347589  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 727904us
I20260709 14:52:33.312565  5763 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.294967  5883 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:33.319655  5874 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.636s	user 0.001s	sys 0.001s
I20260709 14:52:33.337586  5766 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:33.246477  5380 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.348538  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:33.381201  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:33.382313  2723 connection.cc:256] RPC call timeout handler was delayed by -1.1378853780000782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 14:52:33.312546  5880 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:33.322480  5697 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.352804  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:33.378629  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 206846us
I20260709 14:52:33.319766  5886 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:33.362476  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.289906  5772 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 14:52:33.399819  5871 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:33.393945  5697 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.401345  5867 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:33.330135  5088 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:33.320886  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.356051  5883 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:33.389629  3409 leader_election.cc:400] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9.
I20260709 14:52:33.324386  5748 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:33.319970  5891 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:33.319377  5882 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 14:52:33.398241  5799 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.915s	user 0.011s	sys 0.000s
W20260709 14:52:33.406790  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 367830us
W20260709 14:52:33.284576  5873 consensus_queue.cc:1175] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 14:52:33.280927  5718 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:33.403322  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 226370us
I20260709 14:52:33.466430  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.465248  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.473235  2679 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:33.474650  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.465909  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:33.468652  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 147916us
I20260709 14:52:33.469386  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:33.477133  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:33.477954  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:33.471508  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:33.482092  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.483023  2680 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:33.486150  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.487263  2683 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:33.431744  5772 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:33.388826  5868 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.498064  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:33.470875  5878 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
I20260709 14:52:33.501941  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.511718  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.460676  5867 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:33.489889  5879 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.312s	user 0.000s	sys 0.002s
I20260709 14:52:33.408561  5777 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:33.546345  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 193157us
W20260709 14:52:33.552591  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 190239us
I20260709 14:52:33.493022  5875 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:33.471369  5895 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:33.557085  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:33.558597  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:33.554261  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 170806us
I20260709 14:52:33.501611  5894 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.560139  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:33.503584  5739 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.587872   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:52:33.514886  5876 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:33.566126  3547 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:33.542074  5881 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 14:52:33.573064  3542 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:33.571332  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 556) took 1453 ms (client timeout 1536 ms). Trace:
W20260709 14:52:33.566968  2723 connection.cc:256] RPC call timeout handler was delayed by -1.0460853780000434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
W20260709 14:52:33.591748  2723 connection.cc:256] RPC call timeout handler was delayed by -1.0226853780001193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 14:52:33.594229  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 116356us
I20260709 14:52:33.595435  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.596194  3547 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:33.598037  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.598948  3545 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:33.482875  5799 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.599826  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.600507  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.601593  2683 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:33.541611  5896 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:52:33.279515  5877 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.390352  5880 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.335875  5887 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:33.575697  5813 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.970s	user 0.014s	sys 0.003s
I20260709 14:52:33.602941  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.560428  5899 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.603693  2682 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 14:52:33.631296  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:33.639971  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.641129  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1050) took 1652 ms. Trace:
I20260709 14:52:33.635512  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:33.642122  2681 raft_consensus.cc:2376] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:52:33.636344  2677 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:33.553781  5380 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:33.647998  5380 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:33.649492  5760 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.652642  5864 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:33.654114  5889 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.334s	user 0.000s	sys 0.002s
W20260709 14:52:33.658178  5788 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.051s	user 0.010s	sys 0.002s
I20260709 14:52:33.224040  5782 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.586905  5088 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.588593  5892 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 14:52:33.606987  5903 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:33.490327  5851 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:33.630133  5902 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 14:52:33.631378  5872 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.543136  5898 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:33.627391  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 155845us
I20260709 14:52:33.576344  5868 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:33.569504  5767 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:33.577925  5818 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.194s	user 0.014s	sys 0.002s
I20260709 14:52:33.263868  5819 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.626953  5876 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:33.800858  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:33.606016  5697 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:33.666327  5734 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.458s	user 0.011s	sys 0.003s
W20260709 14:52:33.646142  5897 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 14:52:33.694350  5651 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.551870  5867 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.659376  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1167) took 1460 ms. Trace:
I20260709 14:52:33.826898  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 883) took 2501 ms. Trace:
I20260709 14:52:33.692965  5868 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:33.659562  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1160) took 1524 ms (client timeout 1862 ms). Trace:
W20260709 14:52:33.670099  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1166) took 1533 ms (client timeout 1887 ms). Trace:
I20260709 14:52:33.667273  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:33.835743  2611 rpcz_store.cc:269] 0709 14:52:32.136922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.221442 (+ 84520us) service_pool.cc:224] Handling call
0709 14:52:33.669869 (+1448427us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:33.670027 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573265152}
W20260709 14:52:33.693288  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 216246us
I20260709 14:52:33.675033  2753 pending_rounds.cc:85] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:33.837239  2753 pending_rounds.cc:107] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 14:52:33.837756  2753 raft_consensus.cc:2889] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:33.838080  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:33.840190  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.684409  2603 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 14:52:33.667390  3536 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:33.841022  2603 pending_rounds.cc:85] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 3
I20260709 14:52:33.841289  2603 pending_rounds.cc:107] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 14:52:33.841516  2603 raft_consensus.cc:2889] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:33.842164  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:33.835278  2612 rpcz_store.cc:269] 0709 14:52:32.135265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.136593 (+  1328us) service_pool.cc:224] Handling call
0709 14:52:33.659394 (+1522801us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:33.659504 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730925824}
I20260709 14:52:33.674430  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:33.676192  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:33.603456  5752 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.000s
I20260709 14:52:33.572433  5895 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:33.677351  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:33.707645  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:33.700376  5884 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.421s	user 0.000s	sys 0.011s
W20260709 14:52:33.784176  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.002s	sys 0.014s
W20260709 14:52:33.847568  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.002s	sys 0.014s
W20260709 14:52:33.848075  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 619029us
I20260709 14:52:33.641662  2756 rpcz_store.cc:276] 0709 14:52:31.988456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.039999 (+ 51543us) service_pool.cc:224] Handling call
0709 14:52:32.104927 (+ 64928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:32.106779 (+  1852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:32.106821 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:32.133415 (+ 26594us) log.cc:844] Serialized 26 byte log entry
0709 14:52:32.635409 (+501994us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:32.635478 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.635592 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:33.640896 (+1005304us) raft_consensus.cc:1685] finished
0709 14:52:33.640920 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.641056 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1280,"raft.queue_time_us":1387,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":33595,"thread_start_us":382141,"threads_started":1,"wal-append.queue_time_us":784680}
I20260709 14:52:33.588629  5869 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 14:52:33.606024  5896 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:33.750905  3542 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:33.866315  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.595214  5894 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:33.801005  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 95477.2us
W20260709 14:52:33.882812  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:33.590752  2755 rpcz_store.cc:269] 0709 14:52:32.118084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.122573 (+  4489us) service_pool.cc:224] Handling call
0709 14:52:33.571006 (+1448433us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:33.571222 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172589952}
I20260709 14:52:33.670481  5405 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:33.806059  2681 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:33.645846  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.893494  3538 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:33.673746  5872 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:33.810170  2751 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:33.669335  5748 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:33.671638  5903 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:33.914477  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 93898.6us
I20260709 14:52:33.850937  5868 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:33.784726  5818 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.693317  5714 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.928406  2610 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:52:33.928601  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:33.929538  2681 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:33.810141  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 220632us
I20260709 14:52:33.932247  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:33.705056  5848 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:33.608736  5888 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.208s
I20260709 14:52:33.826534  2605 rpcz_store.cc:276] 0709 14:52:32.199198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.501685 (+302487us) service_pool.cc:224] Handling call
0709 14:52:32.502238 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:32.502779 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:32.502817 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:32.533293 (+ 30476us) log.cc:844] Serialized 26 byte log entry
0709 14:52:33.128575 (+595282us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:33.128636 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:33.128880 (+   244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:33.659077 (+530197us) raft_consensus.cc:1685] finished
0709 14:52:33.659113 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.659299 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150717,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":11151,"thread_start_us":526763,"threads_started":1,"wal-append.queue_time_us":626477}
I20260709 14:52:33.827659  3543 rpcz_store.cc:276] 0709 14:52:31.326013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.717060 (+391047us) service_pool.cc:224] Handling call
0709 14:52:31.717515 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:32.339110 (+621595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:32.339159 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:32.340016 (+   857us) log.cc:844] Serialized 52 byte log entry
0709 14:52:32.655695 (+315679us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:32.655776 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.655900 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:33.826548 (+1170648us) raft_consensus.cc:1685] finished
0709 14:52:33.826580 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.826794 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34326,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":58,"thread_start_us":684767,"threads_started":2,"wal-append.queue_time_us":687503}
I20260709 14:52:33.686122  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:33.934567  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:33.934958  3547 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:33.831396  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 155601us
I20260709 14:52:33.846591  5895 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:33.832054  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 155733us
W20260709 14:52:33.832836  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 192775us
W20260709 14:52:33.937408  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:33.934567  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.938226  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.938964  3543 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:33.788501  5892 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:33.835480  2752 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:33.842900  3536 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:33.668777  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.845372  2682 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:33.816061  5496 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:33.908584  5898 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:33.844625  2759 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:33.806321  5886 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:33.862344  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:33.963596  2684 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:33.627048  5739 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:33.870383  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:33.879850  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.658803  5788 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.786692  5911 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:52:33.894801  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54692.7us
I20260709 14:52:33.899330  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.903931  2751 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:33.671391  5904 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.233s
I20260709 14:52:33.670969  5703 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:33.710553  5905 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.201s
I20260709 14:52:33.784606  5767 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:33.918166  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:33.822983  5734 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.940124  2605 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:34.004850  2681 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:33.941128  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 893) took 1643 ms. Trace:
I20260709 14:52:33.941711  2678 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:33.942922  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.944514  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.939276  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:33.673828  5899 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.892620  5405 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:33.671875  5875 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:33.879076  5896 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:33.948278  3537 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 6: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:33.955541  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:34.044018  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 88611us
W20260709 14:52:33.930377  5885 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.005s
I20260709 14:52:33.831674  5893 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 14:52:34.060232  5893 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:34.060988  5925 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:34.061661  5925 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:34.062071  5925 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:34.063431  5816 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:33.966450  5683 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 14:52:34.048496  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:34.065181  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:33.921589  5697 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:34.065980  2612 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:33.889827  5088 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 14:52:33.940379  5714 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:33.974948  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59262.1us
I20260709 14:52:33.999354  5703 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:33.977511  2680 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:33.979171  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 108970us
I20260709 14:52:33.981940  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:33.936234  5848 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:33.980276  3547 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:33.914742  5872 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:33.989275  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:33.905646  5891 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 14:52:33.948189  5620 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:33.997781  2682 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:33.918367  5873 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 14:52:34.012259  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:33.984279  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:34.013173  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:34.011844  3541 rpcz_store.cc:276] 0709 14:52:32.297538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.338322 (+ 40784us) service_pool.cc:224] Handling call
0709 14:52:32.338663 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:32.350718 (+ 12055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:32.350769 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:32.367423 (+ 16654us) log.cc:844] Serialized 26 byte log entry
0709 14:52:32.686817 (+319394us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:32.686914 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:32.687027 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:33.940867 (+1253840us) raft_consensus.cc:1685] finished
0709 14:52:33.940897 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:33.941056 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10565,"raft.queue_time_us":201868,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":29327,"thread_start_us":222438,"threads_started":1,"wal-append.queue_time_us":777225}
I20260709 14:52:34.011646  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.985056  5739 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:33.914487  5915 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 14:52:33.977926  2751 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:34.015415  3545 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
W20260709 14:52:34.034229  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:34.034842  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:34.038147  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:34.039193  3536 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
W20260709 14:52:34.039698  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 100527us
I20260709 14:52:33.942807  5371 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 14:52:34.046788  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:33.961427  5898 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:33.971709  5901 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:34.051618  5914 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:33.965123  5752 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 14:52:34.057960  5918 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:34.058962  5885 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:33.833290  5651 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:34.064496  5683 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:33.960038  2603 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:33.973182  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:33.955276  5876 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 14:52:33.977612  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:33.992926  5772 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:33.925938  5903 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:34.098693  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:34.220012  5887 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:34.268036  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 290519us
I20260709 14:52:34.268522  5887 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:34.099331  2604 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:34.269205  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:34.135170  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:34.270154  2760 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:34.270689  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:34.271055  5888 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:34.271566  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:34.273617  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:34.099653  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:34.274338  2603 raft_consensus.cc:2376] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9. Re-sending same reply.
I20260709 14:52:34.267416  5788 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:34.275663  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:34.276885  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:34.277674  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:34.124069  2754 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:34.161923  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150385us
I20260709 14:52:34.281862  2759 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:34.266320  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:34.271447  5867 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:34.282945  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 340097us
I20260709 14:52:34.493468  5894 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.059314  5877 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:34.989600  5931 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.387s	user 0.000s	sys 0.000s
I20260709 14:52:34.180998  5799 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:35.061384  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1080) took 1455 ms. Trace:
W20260709 14:52:34.404630  5900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.006s	sys 0.020s
I20260709 14:52:35.062424  5380 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:34.495507  5911 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:35.063290  5380 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.063843  5911 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:35.061993  2683 rpcz_store.cc:276] 0709 14:52:33.606410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.606608 (+   198us) service_pool.cc:224] Handling call
0709 14:52:33.607101 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:33.607646 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:33.607693 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:33.636799 (+ 29106us) log.cc:844] Serialized 52 byte log entry
0709 14:52:33.905262 (+268463us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:33.905315 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:33.905411 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:35.061100 (+1155689us) raft_consensus.cc:1685] finished
0709 14:52:35.061125 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:35.061311 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43851,"raft.queue_time_us":202428,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":14663,"thread_start_us":94775,"threads_started":1,"wal-append.queue_time_us":564692}
W20260709 14:52:34.498625  5778 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.149s	user 0.015s	sys 0.001s
I20260709 14:52:34.948664  5734 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:34.285130  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:35.065770  2679 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:35.076756  5911 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:35.078212  2726 connection.cc:256] RPC call timeout handler was delayed by -1.0681943719998799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 14:52:34.365391  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:34.376973  5772 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:35.088130  5772 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:34.611001  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.002s	sys 0.030s
W20260709 14:52:35.097106  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.009s	sys 0.033s
W20260709 14:52:35.099555  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:34.320009  5620 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:34.286171  5882 raft_consensus.cc:2764] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 14:52:34.271678  5888 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:34.280766  2756 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:35.113206  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1064) took 1101 ms. Trace:
W20260709 14:52:34.539669  5793 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.436s	user 0.014s	sys 0.004s
W20260709 14:52:35.049166  5896 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.994s	user 0.008s	sys 0.006s
W20260709 14:52:35.071635  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1043) took 1269 ms (client timeout 1670 ms). Trace:
I20260709 14:52:34.123431  5909 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 1.036s
W20260709 14:52:34.299866  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 315697us
W20260709 14:52:34.735844  5755 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 2.246s	user 0.006s	sys 0.013s
I20260709 14:52:35.072443  5901 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:34.218499  5833 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.871s
I20260709 14:52:34.375308  5752 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:34.440026  5904 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:34.271483  3541 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:35.180563  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1040) took 1500 ms. Trace:
I20260709 14:52:35.181090  3541 rpcz_store.cc:276] 0709 14:52:33.680281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.270396 (+590115us) service_pool.cc:224] Handling call
0709 14:52:35.180476 (+910080us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:34.464682  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 248254us
I20260709 14:52:35.075018  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1046) took 1006 ms. Trace:
W20260709 14:52:34.792412  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.006s	sys 0.018s
W20260709 14:52:35.186076  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.148s	user 0.008s	sys 0.023s
I20260709 14:52:35.185865  3540 rpcz_store.cc:276] 0709 14:52:34.068159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.257639 (+189480us) service_pool.cc:224] Handling call
0709 14:52:35.074914 (+817275us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:35.186762  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.14942e+06us
I20260709 14:52:35.187984  3540 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:35.189139  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1036) took 1510 ms. Trace:
I20260709 14:52:35.189617  3540 rpcz_store.cc:276] 0709 14:52:33.678866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.187197 (+1508331us) service_pool.cc:224] Handling call
0709 14:52:35.187679 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:35.188745 (+  1066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:35.189056 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:34.320461  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:35.049937  5872 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:34.287256  5683 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:34.371474  5887 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:35.219398  5887 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:35.219997  5887 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:34.261428  5924 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 14:52:35.078969  5898 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:34.344538  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:34.492305  5899 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:34.376538  5819 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:35.081027  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1063) took 1092 ms. Trace:
I20260709 14:52:35.235430  2759 rpcz_store.cc:276] 0709 14:52:33.988613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.988949 (+   336us) service_pool.cc:224] Handling call
0709 14:52:35.080924 (+1091975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:35.226951  5952 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:35.259897  5952 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.263703  5752 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:35.265602  5752 consensus_queue.cc:1243] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 14:52:35.266844  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 208637us
I20260709 14:52:34.467967  5919 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:34.493763  5818 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:34.276664  5914 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:34.393939  2685 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:35.295065  2685 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:35.090052  3547 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:34.342332  5496 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.733s
I20260709 14:52:35.058809  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:34.101403  5902 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:35.067976  5918 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:35.070611  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1065) took 1054 ms. Trace:
I20260709 14:52:35.306854  2755 rpcz_store.cc:276] 0709 14:52:34.016157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.034448 (+ 18291us) service_pool.cc:224] Handling call
0709 14:52:35.070514 (+1036066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:34.332149  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:35.308777  2680 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 14:52:35.308853  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.0332e+06us
I20260709 14:52:35.309321  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1095) took 1388 ms. Trace:
I20260709 14:52:35.310029  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:35.048588  5933 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.261s	user 0.000s	sys 0.000s
I20260709 14:52:35.117475  2756 rpcz_store.cc:276] 0709 14:52:34.011799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.012032 (+   233us) service_pool.cc:224] Handling call
0709 14:52:35.113089 (+1101057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:35.309628  2680 rpcz_store.cc:276] 0709 14:52:33.920629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.319528 (+398899us) service_pool.cc:224] Handling call
0709 14:52:35.309238 (+989710us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:34.773149  5883 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.313s	user 0.011s	sys 0.003s
W20260709 14:52:35.174619  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 897712us
W20260709 14:52:35.176244  3536 rpcz_store.cc:269] 0709 14:52:33.802194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.841850 (+ 39656us) service_pool.cc:224] Handling call
0709 14:52:35.071530 (+1229680us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:35.092816  5895 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.128s	user 0.014s	sys 0.000s
I20260709 14:52:35.338752  2680 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:35.339604  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1089) took 1523 ms. Trace:
I20260709 14:52:35.061965  5885 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:34.319396  5816 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.790s
I20260709 14:52:35.178954  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:35.340191  5898 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:35.341099  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 162250us
I20260709 14:52:35.194084  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:35.194988  3541 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:35.343796  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1035) took 1665 ms. Trace:
I20260709 14:52:35.344295  3541 rpcz_store.cc:276] 0709 14:52:33.678524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.182396 (+1503872us) service_pool.cc:224] Handling call
0709 14:52:35.194635 (+ 12239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:35.343364 (+148729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:35.343696 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:35.346010  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:34.254307  5767 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:35.078228  5911 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:35.171527  5925 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.109s	user 0.012s	sys 0.004s
I20260709 14:52:35.373302  5925 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:35.203989  5620 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:35.090391  5714 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.192709  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:35.113610  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.008s	sys 0.011s
W20260709 14:52:35.381472  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.008s	sys 0.012s
I20260709 14:52:35.093684  5739 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:35.383816  5819 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:35.384776  5819 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.388797  5925 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.236627  2760 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:35.237174  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:35.399834  5925 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:35.192023  5896 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:35.191284  5882 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:35.191659  5793 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:35.401244  5793 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:35.403384  5925 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:35.404322  5793 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:35.405256  3537 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:35.402397  5896 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:34.274124  5886 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.199541  5755 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.433171  5755 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.273468  5901 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:35.278308  5766 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.874s	user 0.016s	sys 0.000s
I20260709 14:52:35.435207  5766 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.436725  5898 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.438246  5766 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.205444  5904 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:35.463008  5848 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.121s	user 0.007s	sys 0.008s
I20260709 14:52:35.296644  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 763) took 1563 ms. Trace:
I20260709 14:52:35.463887  5848 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:35.531920  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.25397e+06us
W20260709 14:52:34.356554  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:35.555262  2605 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:34.359882  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:35.340049  2680 rpcz_store.cc:276] 0709 14:52:33.816702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.337954 (+1521252us) service_pool.cc:224] Handling call
0709 14:52:35.338367 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:35.339300 (+   933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:35.339541 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:34.381752  2681 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:35.296186  2685 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.784283  5818 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 14:52:35.267839  2723 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.815s (SENT)
I20260709 14:52:35.785578  2685 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:35.786437  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:35.787017  5900 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.000s	sys 0.004s
I20260709 14:52:35.342136  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:35.400104  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1067) took 1304 ms. Trace:
I20260709 14:52:35.789039  2759 rpcz_store.cc:276] 0709 14:52:34.095465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.236770 (+1141305us) service_pool.cc:224] Handling call
0709 14:52:35.400029 (+163259us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:35.399936  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1068) took 1299 ms (client timeout 1596 ms). Trace:
I20260709 14:52:35.790339  5904 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:35.791112  5766 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 14:52:35.791190  2760 rpcz_store.cc:269] 0709 14:52:34.100114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.275916 (+175802us) service_pool.cc:224] Handling call
0709 14:52:35.399860 (+1123944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:35.323941  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:35.793673  2611 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:35.331542  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:35.794818  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 544) took 1972 ms (client timeout 1726 ms). Trace:
W20260709 14:52:35.795506  2611 rpcz_store.cc:269] 0709 14:52:33.821989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.903629 (+ 81640us) service_pool.cc:224] Handling call
0709 14:52:34.359555 (+455926us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:35.794590 (+1435035us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4562688}
W20260709 14:52:35.796602  5683 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.006s
I20260709 14:52:35.797411  5683 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.349109  3541 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:35.622722  2682 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:35.619189  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.003s	sys 0.011s
I20260709 14:52:35.802491  5959 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:52:35.728858  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 669929us
W20260709 14:52:35.565321  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 371108us
I20260709 14:52:35.809886  5896 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:35.300745  5703 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.964s	user 0.014s	sys 0.001s
I20260709 14:52:35.175046  5788 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:35.444131  5819 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:35.450716  5868 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.510s	user 0.015s	sys 0.001s
I20260709 14:52:35.811887  5788 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:35.812613  5788 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:35.812155  5703 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.812987  5868 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:35.356206  5883 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.817653  2608 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:52:35.241400  5887 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:35.227376  5888 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:35.339269  5895 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:36.110671  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 536) took 3806 ms (client timeout 2735 ms). Trace:
W20260709 14:52:36.111613  2686 rpcz_store.cc:269] 0709 14:52:32.303574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.306556 (+  2982us) service_pool.cc:224] Handling call
0709 14:52:36.109744 (+3803188us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:36.110240 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3618067840}
I20260709 14:52:36.113430  5895 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:36.113687  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:35.455175  5902 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:36.116875  5902 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:35.049412  5903 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:35.361884  5909 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:36.119827  5909 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:36.119916  5903 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:36.120635  5909 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:36.121320  5903 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:36.121982  5903 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:36.122495  5903 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:35.297346  5914 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:35.361027  5918 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.297520  5919 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:35.406569  5925 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:35.615163  2606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.005s	sys 0.005s
W20260709 14:52:36.171749  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1169) took 2228 ms (client timeout 1811 ms). Trace:
I20260709 14:52:35.357667  5952 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:35.338413  5953 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:35.728231  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 535376us
I20260709 14:52:35.753993  5734 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:35.790133  5793 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:35.791688  5885 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:35.464536  3547 rpcz_store.cc:276] 0709 14:52:33.733932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.284683 (+550751us) service_pool.cc:224] Handling call
0709 14:52:35.296558 (+1011875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:35.802186  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:35.803242  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.529s	user 0.012s	sys 0.015s
W20260709 14:52:35.709640  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 369221us
I20260709 14:52:35.873404  5883 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:35.798410  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:35.874142  5818 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.154s
I20260709 14:52:35.785645  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:35.295328  5913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.000s	sys 0.008s
I20260709 14:52:36.123423  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1034) took 2323 ms. Trace:
I20260709 14:52:35.875842  5766 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:36.172284  2606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.546s	user 0.006s	sys 0.006s
W20260709 14:52:36.246129  5913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.624s	user 0.000s	sys 0.009s
W20260709 14:52:36.170981  3537 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.613s	user 0.003s	sys 0.024s
I20260709 14:52:36.266144  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1048) took 1073 ms. Trace:
I20260709 14:52:36.301360  3536 rpcz_store.cc:276] 0709 14:52:35.193109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.797315 (+604206us) service_pool.cc:224] Handling call
0709 14:52:36.266055 (+468740us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:36.172526  2610 rpcz_store.cc:269] 0709 14:52:33.673835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.674084 (+   249us) service_pool.cc:224] Handling call
0709 14:52:35.769024 (+2094940us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x7b54001eb754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:35.902469 (+133445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194830592}
I20260709 14:52:36.244357  5887 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:36.267107  5734 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:36.246187  3545 rpcz_store.cc:276] 0709 14:52:33.678187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.678512 (+   325us) service_pool.cc:224] Handling call
0709 14:52:33.785473 (+106961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:34.273573 (+488100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:35.772439 (+1498866us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:36.001199 (+228760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528288512}
I20260709 14:52:36.359615  3537 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:36.246958  5793 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:36.324940  5848 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:36.414736  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 539) took 3873 ms. Trace:
I20260709 14:52:36.411245  5734 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:36.418068  5793 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:36.436239  3409 connection.cc:256] RPC call timeout handler was delayed by -1.15575259000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2735s for a response
I20260709 14:52:36.456739  2680 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:36.495234  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 428071us
W20260709 14:52:36.498476  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.15371e+06us
I20260709 14:52:36.433388  2609 rpcz_store.cc:276] 0709 14:52:32.225040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.525357 (+300317us) service_pool.cc:224] Handling call
0709 14:52:32.561754 (+ 36397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:33.242457 (+680703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:33.242498 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:33.243274 (+   776us) log.cc:844] Serialized 52 byte log entry
0709 14:52:33.490669 (+247395us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:33.490718 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:33.490808 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:35.758704 (+2267896us) raft_consensus.cc:1685] finished
0709 14:52:35.773911 (+ 15207us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:36.097767 (+323856us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16007,"raft.queue_time_us":563131,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5390,"thread_start_us":851522,"threads_started":2,"wal-append.queue_time_us":668790,"wal-append.run_cpu_time_us":2383,"wal-append.run_wall_time_us":1109596}
I20260709 14:52:36.507344  2686 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:36.509016  5886 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 14:52:36.512189  5960 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 14:52:36.516408  3412 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.193s (SENT)
I20260709 14:52:36.519438  5766 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:36.522423  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 2.42264e+06us
W20260709 14:52:36.527173  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 988) took 4158 ms (client timeout 3353 ms). Trace:
I20260709 14:52:36.529409  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:36.531062  5883 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:36.533044  2756 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:36.535437  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:36.542151  5911 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.170s	user 0.014s	sys 0.013s
I20260709 14:52:36.552569  3545 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:36.553447  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:36.554968  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 550) took 1151 ms. Trace:
I20260709 14:52:36.558332  3536 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:36.484853  5885 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:36.561305  5703 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:36.560418  5405 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:36.173051  5919 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:36.577988  2685 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.006s
I20260709 14:52:36.475665  5868 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:36.575528  2684 rpcz_store.cc:269] 0709 14:52:32.305584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.348681 (+ 43097us) service_pool.cc:224] Handling call
0709 14:52:36.362647 (+4013966us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400269654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:36.464231 (+101584us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3710954240}
I20260709 14:52:36.581447  2680 rpcz_store.cc:276] 0709 14:52:35.342012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.784279 (+442267us) service_pool.cc:224] Handling call
0709 14:52:36.493073 (+708794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:36.589151  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1094) took 2671 ms. Trace:
I20260709 14:52:36.592562  2686 rpcz_store.cc:276] 0709 14:52:33.901546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.473334 (+2571788us) service_pool.cc:224] Handling call
0709 14:52:36.506752 (+ 33418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.564814 (+ 58062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.572927 (+  8113us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:36.597533  2605 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.828s	user 0.015s	sys 0.006s
W20260709 14:52:36.598577  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.06699e+06us
I20260709 14:52:36.599241  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:36.593940  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:36.598029  2605 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:36.595537  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1041) took 2880 ms. Trace:
I20260709 14:52:36.603662  3536 rpcz_store.cc:276] 0709 14:52:33.708857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.419572 (+2710715us) service_pool.cc:224] Handling call
0709 14:52:36.557984 (+138412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.588649 (+ 30665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.588985 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:36.592675  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 767) took 2766 ms. Trace:
I20260709 14:52:36.606419  3545 rpcz_store.cc:276] 0709 14:52:33.819802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.551257 (+2731455us) service_pool.cc:224] Handling call
0709 14:52:36.551959 (+   702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.580383 (+ 28424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.586137 (+  5754us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:36.617406  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 546) took 2794 ms (client timeout 1933 ms). Trace:
W20260709 14:52:36.614683  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.067s	sys 0.117s
W20260709 14:52:36.618105  2605 rpcz_store.cc:269] 0709 14:52:33.822559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.037703 (+215144us) service_pool.cc:224] Handling call
0709 14:52:36.613572 (+2575869us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:36.629864  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.067s	sys 0.118s
I20260709 14:52:36.613025  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:36.619961  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.004s	sys 0.020s
W20260709 14:52:36.634063  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.004s	sys 0.021s
W20260709 14:52:36.634476  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 826898us
I20260709 14:52:36.640460  2685 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:36.643636  5405 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:36.648208  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:36.652100  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.3842e+06us
W20260709 14:52:36.653488  2723 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.841s (SENT) [suppressed 20 similar messages]
I20260709 14:52:36.660073  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1170) took 2985 ms. Trace:
I20260709 14:52:36.660778  2608 rpcz_store.cc:276] 0709 14:52:33.674208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.704039 (+ 29831us) service_pool.cc:224] Handling call
0709 14:52:33.704554 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:35.818247 (+2113693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.659557 (+841310us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:36.659966 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4857693568}
W20260709 14:52:36.667454  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 238512us
I20260709 14:52:36.667868  5763 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:36.668395  5763 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:36.668859  5952 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:36.650454  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:36.671676  5885 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:36.672252  2611 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:36.673182  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1176) took 2989 ms. Trace:
I20260709 14:52:36.673907  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:36.674639  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 762) took 2974 ms (client timeout 1783 ms). Trace:
W20260709 14:52:36.675280  3542 rpcz_store.cc:269] 0709 14:52:33.699826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.865942 (+166116us) service_pool.cc:224] Handling call
0709 14:52:36.674148 (+2808206us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400317c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:36.674355 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5469154048}
I20260709 14:52:36.676496  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1037) took 2997 ms. Trace:
I20260709 14:52:36.677192  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:36.677883  2756 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:36.679754  5959 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:36.680385  5959 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:36.681214  5959 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:36.673636  2611 rpcz_store.cc:276] 0709 14:52:33.683921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.797163 (+2113242us) service_pool.cc:224] Handling call
0709 14:52:36.487460 (+690297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:36.672798 (+185338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.673066 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:36.683676  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.051s	sys 0.081s
W20260709 14:52:36.684199  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.051s	sys 0.081s
I20260709 14:52:36.676964  5883 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:36.688396  2684 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 14:52:36.689248  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 549) took 1347 ms. Trace:
W20260709 14:52:36.689965  5914 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.005s
I20260709 14:52:36.690136  5953 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:36.690510  5886 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:36.679714  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:36.677052  3540 rpcz_store.cc:276] 0709 14:52:33.679193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.190738 (+1511545us) service_pool.cc:224] Handling call
0709 14:52:35.196340 (+  5602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:35.465566 (+269226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:35.465799 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:35.465922 (+   123us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:35.466018 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.675906 (+1209888us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:36.676012 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:36.676254 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618279296}
W20260709 14:52:36.694919  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.046s	sys 0.075s
W20260709 14:52:36.712296  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.049s	sys 0.075s
I20260709 14:52:36.689631  2684 rpcz_store.cc:276] 0709 14:52:35.341764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.634169 (+1292405us) service_pool.cc:224] Handling call
0709 14:52:36.687840 (+ 53671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.688921 (+  1081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.689180 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:36.677883  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:36.716965  2610 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:36.670064  5763 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:36.718017  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1177) took 3027 ms. Trace:
I20260709 14:52:36.718161  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1083) took 3043 ms. Trace:
I20260709 14:52:36.718472  2610 rpcz_store.cc:276] 0709 14:52:33.690324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.412532 (+2722208us) service_pool.cc:224] Handling call
0709 14:52:36.605301 (+192769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.717663 (+112362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.717912 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:37.227758  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:37.392870  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 676231us
W20260709 14:52:36.795020  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 118053us
I20260709 14:52:36.720026  2609 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
W20260709 14:52:37.327129  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.015s	sys 0.013s
W20260709 14:52:37.400053  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.016s	sys 0.014s
W20260709 14:52:37.352936  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.55103e+06us
W20260709 14:52:37.362602  2606 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.016s	sys 0.013s
I20260709 14:52:37.401238  2758 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 4: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:37.401727  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:37.256781  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:36.866436  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:36.787935  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120312us
I20260709 14:52:36.806370  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1175) took 3122 ms. Trace:
I20260709 14:52:36.839624  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1087) took 3052 ms. Trace:
I20260709 14:52:37.402712  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:37.401347  2606 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.016s	sys 0.015s
W20260709 14:52:37.298316  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.024s	sys 0.022s
W20260709 14:52:37.330643  3545 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.000s	sys 0.009s
W20260709 14:52:37.407809  3545 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.001s	sys 0.009s
I20260709 14:52:37.407176  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 543) took 3585 ms. Trace:
W20260709 14:52:37.408447  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.025s	sys 0.026s
W20260709 14:52:37.323153  2577 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.012s	sys 0.017s
I20260709 14:52:37.408718  2609 rpcz_store.cc:276] 0709 14:52:33.821704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.610221 (+2788517us) service_pool.cc:224] Handling call
0709 14:52:36.719510 (+109289us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:37.406590 (+687080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.407066 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:37.409011  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 905658us
I20260709 14:52:36.805615  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1039) took 3125 ms. Trace:
W20260709 14:52:37.234712  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.018s	sys 0.007s
W20260709 14:52:36.793478  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56137us
I20260709 14:52:36.718339  5763 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:37.437331  2577 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.012s	sys 0.025s
W20260709 14:52:37.282668  5967 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.208s	user 0.006s	sys 0.000s
I20260709 14:52:37.405297  2685 rpcz_store.cc:276] 0709 14:52:33.787856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.338759 (+550903us) service_pool.cc:224] Handling call
0709 14:52:34.339314 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.724642 (+2385328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.839529 (+114887us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:37.440958  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.026s
W20260709 14:52:37.449556  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.000s	sys 0.027s
I20260709 14:52:36.803957  5683 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:37.407652  5793 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.880s	user 0.017s	sys 0.001s
I20260709 14:52:36.791682  5885 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:37.404299  2603 rpcz_store.cc:276] 0709 14:52:33.683609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.275151 (+591542us) service_pool.cc:224] Handling call
0709 14:52:34.275543 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:34.614506 (+338963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:34.614550 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:35.788151 (+1173601us) log.cc:844] Serialized 26 byte log entry
0709 14:52:36.419144 (+630993us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:36.434878 (+ 15734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.470892 (+ 36014us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:36.805843 (+334951us) raft_consensus.cc:1685] finished
0709 14:52:36.805878 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:36.806103 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6,"raft.queue_time_us":783866,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":164,"spinlock_wait_cycles":178944,"thread_start_us":767835,"threads_started":2,"wal-append.queue_time_us":734119}
W20260709 14:52:37.449699  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 888399us
W20260709 14:52:37.441092  5888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.014s	sys 0.044s
W20260709 14:52:37.441978  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.019s	sys 0.009s
I20260709 14:52:37.437894  3538 rpcz_store.cc:276] 0709 14:52:33.679952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.126365 (+2446413us) service_pool.cc:224] Handling call
0709 14:52:36.127299 (+   934us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:36.128355 (+  1056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:36.128410 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:36.129297 (+   887us) log.cc:844] Serialized 26 byte log entry
0709 14:52:36.619538 (+490241us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:36.619647 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.619804 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:36.805036 (+185232us) raft_consensus.cc:1685] finished
0709 14:52:36.805072 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:36.805338 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35758,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":9074,"thread_start_us":372856,"threads_started":1,"wal-append.queue_time_us":610623}
W20260709 14:52:37.462491  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:37.467993  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:36.690666  5914 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:37.480662  2756 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:37.455381  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:36.901026  5873 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.679s
W20260709 14:52:37.467419  5888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.015s	sys 0.045s
I20260709 14:52:36.718696  2677 rpcz_store.cc:276] 0709 14:52:33.674506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.675212 (+   706us) service_pool.cc:224] Handling call
0709 14:52:33.675646 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:33.914994 (+239348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:33.915029 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:33.922226 (+  7197us) log.cc:844] Serialized 26 byte log entry
0709 14:52:34.341367 (+419141us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:34.341429 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.717250 (+2375821us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540022cc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:36.717353 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:36.717479 (+   126us) raft_consensus.cc:1685] finished
0709 14:52:36.717500 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:36.717889 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51023,"raft.queue_time_us":51827,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":13797,"spinlock_wait_cycles":431844736,"thread_start_us":361679,"threads_started":1,"wal-append.queue_time_us":1277504,"wal-append.run_cpu_time_us":2573,"wal-append.run_wall_time_us":1017462}
W20260709 14:52:37.466243  5959 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.005s
I20260709 14:52:36.690960  5953 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:37.494751  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:37.479206  5725 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 14:52:37.498764  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:37.461949  5885 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:37.495980  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e31634415994959847625e41eecf9b3"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:37.504992  5912 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.487242  5961 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 14:52:37.505304  2612 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:37.491312  5760 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.001s	sys 0.021s
I20260709 14:52:37.476624  5968 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.508421  5912 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:37.517025  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 533) took 5214 ms (client timeout 2190 ms). Trace:
I20260709 14:52:37.509608  2686 raft_consensus.cc:1217] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Deduplicated request from leader. Original: 5.4->[7.5-11.7]   Dedup: 7.5->[9.6-11.7]
I20260709 14:52:37.518095  2686 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
I20260709 14:52:37.517663  5912 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:37.518934  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1159) took 6186 ms. Trace:
I20260709 14:52:37.520370  5886 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:37.517635  2678 rpcz_store.cc:269] 0709 14:52:32.302581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.305282 (+  2701us) service_pool.cc:224] Handling call
0709 14:52:37.516547 (+5211265us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540024efd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.516735 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3504531328}
I20260709 14:52:37.519994  2604 rpcz_store.cc:276] 0709 14:52:31.332823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:31.602351 (+269528us) service_pool.cc:224] Handling call
0709 14:52:31.602924 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:34.269322 (+2666398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.518304 (+3248982us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b540042e0d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.518678 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5740469504}
W20260709 14:52:37.513428  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70978.1us
I20260709 14:52:37.528558  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:37.529954  5886 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:37.530441  5886 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:37.530799  5886 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:37.531162  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 549) took 3707 ms. Trace:
I20260709 14:52:37.531107  5968 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:37.507246  5760 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.002s	sys 0.021s
I20260709 14:52:37.531821  2608 rpcz_store.cc:276] 0709 14:52:33.823487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.662049 (+2838562us) service_pool.cc:224] Handling call
0709 14:52:36.662745 (+   696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:36.691699 (+ 28954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:36.691755 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:36.709651 (+ 17896us) log.cc:844] Serialized 26 byte log entry
0709 14:52:37.458477 (+748826us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:37.458550 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.458642 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:37.530875 (+ 72233us) raft_consensus.cc:1685] finished
0709 14:52:37.530897 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.531089 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1184,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":202,"thread_start_us":546809,"threads_started":1,"wal-append.queue_time_us":609774}
I20260709 14:52:37.534112  5968 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:37.466856  5793 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:37.536940  2680 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:52:37.538435  5793 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:37.539474  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1038) took 3860 ms. Trace:
I20260709 14:52:37.540187  3541 rpcz_store.cc:276] 0709 14:52:33.679564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.800679 (+2121115us) service_pool.cc:224] Handling call
0709 14:52:35.801648 (+   969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:35.802650 (+  1002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:35.802689 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:35.873656 (+ 70967us) log.cc:844] Serialized 26 byte log entry
0709 14:52:36.684706 (+811050us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:36.684765 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.684861 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:36.697191 (+ 12330us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:37.522336 (+825145us) raft_consensus.cc:1685] finished
0709 14:52:37.522362 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.539360 (+ 16998us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9006,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":529,"spinlock_wait_cycles":50363264,"thread_start_us":810325,"threads_started":1,"wal-append.queue_time_us":797352}
W20260709 14:52:37.543581  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46669us
I20260709 14:52:37.504561  5885 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:37.557297  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69001.4us
I20260709 14:52:37.556044  2758 raft_consensus.cc:606] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:37.567077  2758 raft_consensus.cc:614] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:37.500708  5959 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:37.499802  5900 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 14:52:37.574566  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.580327  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:37.581475  5698 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:37.581641  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.582710  2684 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:37.582036  5698 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:37.585119  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.586535  5718 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:37.585861  5698 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.587749  5698 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:37.591140  5983 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:52:37.592557  2726 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.811s (SENT)
I20260709 14:52:37.574620  5750 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.594718  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.586184  2608 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:37.601383  5750 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:37.605643  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.607612  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:37.608122  5750 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:37.608516  2677 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:37.609680  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1088) took 3806 ms. Trace:
I20260709 14:52:37.605841  2678 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:37.612790  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.620083  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 565) took 1074 ms. Trace:
I20260709 14:52:37.620187  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:37.638797  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55151.6us
I20260709 14:52:37.620777  2758 rpcz_store.cc:276] 0709 14:52:36.546242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.571176 (+1024934us) service_pool.cc:224] Handling call
0709 14:52:37.573168 (+  1992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:37.619552 (+ 46384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:37.619591 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:37.619669 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:37.619750 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.619837 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.619997 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:37.646406  2685 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:37.611361  2683 rpcz_store.cc:276] 0709 14:52:33.802998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:35.065079 (+1262081us) service_pool.cc:224] Handling call
0709 14:52:35.065607 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:35.225715 (+160108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:35.225761 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:35.225832 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:35.225891 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.609087 (+2383196us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.609163 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.609487 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19983,"raft.queue_time_us":51152,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"spinlock_wait_cycles":320920832}
W20260709 14:52:37.652571  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 175916us
I20260709 14:52:37.659355  2604 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 14:52:37.659085  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 162362us
W20260709 14:52:37.662313  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1174) took 3979 ms (client timeout 2180 ms). Trace:
W20260709 14:52:37.663046  2612 rpcz_store.cc:269] 0709 14:52:33.683283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.066783 (+383500us) service_pool.cc:224] Handling call
0709 14:52:37.661989 (+3595206us) spinlock_profiling.cc:243] Waited 3.23 s on lock 0x7b54002ff3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.662224 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7422823680}
W20260709 14:52:37.673177  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146424us
I20260709 14:52:37.678992  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.680233  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.680997  2605 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:37.682070  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.682235  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.682452  5777 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:37.682917  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:37.683225  2605 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:37.684738  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:37.687090  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1077) took 4087 ms (client timeout 2112 ms). Trace:
I20260709 14:52:37.687335  2605 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:37.687559  2679 rpcz_store.cc:269] 0709 14:52:33.599378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.599519 (+   141us) service_pool.cc:224] Handling call
0709 14:52:37.686635 (+4087116us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400376754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.686880 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3369547008}
I20260709 14:52:37.688434  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.689036  2605 raft_consensus.cc:2376] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:37.690039  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.691782  2605 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:37.692757  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.693416  2605 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:37.685846  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1097) took 3698 ms (client timeout 1598 ms). Trace:
I20260709 14:52:37.686523  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.694768  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.677650  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:37.698714  2608 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:52:37.695432  2681 rpcz_store.cc:269] 0709 14:52:33.987533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.011345 (+ 23812us) service_pool.cc:224] Handling call
0709 14:52:37.685460 (+3674115us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.685629 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258635136}
W20260709 14:52:37.705013  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84728.9us
I20260709 14:52:37.705163  2758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:37.682924  5777 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:37.714089  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68982.6us
I20260709 14:52:37.708011  2758 raft_consensus.cc:606] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:37.716379  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.722007  2612 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:37.721848  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:37.724114  2751 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:37.711131  5766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.008s	sys 0.012s
I20260709 14:52:37.727319  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.727350  2604 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:37.727747  5766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.008s	sys 0.012s
I20260709 14:52:37.728552  3541 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:37.729259  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.730054  3538 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:52:37.716414  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.730995  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.731494  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.731843  2609 raft_consensus.cc:2376] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5. Re-sending same reply.
I20260709 14:52:37.732353  3543 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:37.735975  5903 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.433s	user 0.017s	sys 0.001s
I20260709 14:52:37.731031  2610 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
W20260709 14:52:37.739850  2726 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.180s (SENT)
I20260709 14:52:37.745334  5905 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 14:52:37.746155  5818 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:37.746883  5818 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:37.747225  5766 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:37.747382  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.747874  5766 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:37.747330  5818 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.748317  3543 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:37.749472  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 547) took 2408 ms. Trace:
I20260709 14:52:37.749176  5766 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.749967  2680 rpcz_store.cc:276] 0709 14:52:35.341176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.605252 (+1264076us) service_pool.cc:224] Handling call
0709 14:52:36.663730 (+ 58478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:37.537365 (+873635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.749166 (+211801us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54000f0354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.749374 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2007417728}
I20260709 14:52:37.758517  2684 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:37.763782  5953 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:37.764322  5777 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.768260  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:37.769522  2760 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:37.753383  5903 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:37.770133  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1161) took 1996 ms (client timeout 2214 ms). Trace:
W20260709 14:52:37.770574  2760 rpcz_store.cc:269] 0709 14:52:35.799645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.767776 (+1968131us) service_pool.cc:224] Handling call
0709 14:52:37.770030 (+  2254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:37.769074  5760 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:37.717821  5914 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:37.766369  5777 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:37.775487  5919 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.019s	sys 0.014s
I20260709 14:52:37.779076  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:37.779317  5919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.019s	sys 0.014s
I20260709 14:52:37.780828  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:37.784628  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1154) took 4118 ms. Trace:
I20260709 14:52:37.785714  5959 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:37.782294  5880 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:37.781728  2758 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:37.724144  5496 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:37.795564  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52612us
I20260709 14:52:37.796607  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:37.796670  2575 connection.cc:256] RPC call timeout handler was delayed by -2.611536790999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33530000000000004s for a response
W20260709 14:52:37.777031  5909 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.655s	user 0.019s	sys 0.002s
W20260709 14:52:37.787559  5788 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.974s	user 0.019s	sys 0.004s
I20260709 14:52:37.798236  2751 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:37.791532  5718 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.802278  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.806912  5718 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:37.808534  5929 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 3.522s	user 0.011s	sys 0.032s
W20260709 14:52:37.809211  4786 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 14:52:37.809835  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.801718  5909 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:37.811164  5891 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.811321  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1172) took 4136 ms. Trace:
I20260709 14:52:37.811872  5891 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.812317  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:37.812999  5909 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:37.813362  5718 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:37.814388  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1061) took 5816 ms. Trace:
I20260709 14:52:37.814877  2752 rpcz_store.cc:276] 0709 14:52:31.997681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:32.420387 (+422706us) service_pool.cc:224] Handling call
0709 14:52:32.420795 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:33.949694 (+1528899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.795375 (+3845681us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.814159 (+ 18784us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3252942208}
I20260709 14:52:37.815330  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:37.815534  2610 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:37.816061  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 551) took 2473 ms. Trace:
I20260709 14:52:37.816339  3540 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.816557  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.817584  2752 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.792164  5496 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:37.816915  2755 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:37.814429  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:37.820524  3541 raft_consensus.cc:2376] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:37.820919  5867 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:37.819087  5877 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:37.822118  5877 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:37.822352  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 564) took 1278 ms. Trace:
W20260709 14:52:37.822983  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:37.823508  5877 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:37.824175  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 761) took 4145 ms (client timeout 2945 ms). Trace:
W20260709 14:52:37.824658  3537 rpcz_store.cc:269] 0709 14:52:33.679143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.046433 (+367290us) service_pool.cc:224] Handling call
0709 14:52:37.823915 (+3777482us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.824087 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2594428800}
I20260709 14:52:37.825233  5877 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.819121  5496 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.822162  5867 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:52:37.827907  5867 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:37.819643  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.829197  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.829545  3538 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:37.830204  3537 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:37.830796  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:37.831065  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1069) took 3706 ms (client timeout 1957 ms). Trace:
I20260709 14:52:37.785235  2753 rpcz_store.cc:276] 0709 14:52:33.665602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.666922 (+  1320us) service_pool.cc:224] Handling call
0709 14:52:33.667332 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:33.862339 (+195007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:33.862377 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:33.862982 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 14:52:34.280522 (+417540us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:34.377234 (+ 96712us) log.cc:844] Serialized 14 byte log entry
0709 14:52:34.377593 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.659053 (+3281460us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b180045b200. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:37.783947 (+124894us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.784014 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:37.784100 (+    86us) raft_consensus.cc:1685] finished
0709 14:52:37.784117 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.784355 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112806,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":12271,"raft.queue_time_us":16053,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":46944768,"thread_start_us":344687,"threads_started":1,"wal-append.queue_time_us":1282472,"wal-append.run_cpu_time_us":2826,"wal-append.run_wall_time_us":1041201}
I20260709 14:52:37.831713  3538 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:37.835291  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:37.835609  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.836246  2683 raft_consensus.cc:2376] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:37.838240  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.811760  2607 rpcz_store.cc:276] 0709 14:52:33.681561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.701026 (+ 19465us) service_pool.cc:224] Handling call
0709 14:52:33.701532 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:33.702065 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:33.702106 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:33.708019 (+  5913us) log.cc:844] Serialized 26 byte log entry
0709 14:52:33.976257 (+268238us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:33.976324 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:33.976420 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:37.807632 (+3831212us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:37.810695 (+  3063us) raft_consensus.cc:1685] finished
0709 14:52:37.810719 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.811101 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49268,"raft.queue_time_us":98123,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":28816,"spinlock_wait_cycles":48229248,"thread_start_us":267760,"threads_started":1,"wal-append.queue_time_us":496733}
I20260709 14:52:37.804499  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:37.839888  5970 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.249s	user 0.000s	sys 0.004s
I20260709 14:52:37.859056  3543 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:37.861187  3546 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:37.862331  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1093) took 3975 ms (client timeout 4408 ms). Trace:
W20260709 14:52:37.862890  2682 rpcz_store.cc:269] 0709 14:52:33.886738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.320090 (+433352us) service_pool.cc:224] Handling call
0709 14:52:37.862014 (+3541924us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400269154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.862227 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989448960}
W20260709 14:52:37.849799  2576 connection.cc:256] RPC call timeout handler was delayed by -2.7705139380000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
I20260709 14:52:37.839450  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.841101  4786 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.841655  3541 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:37.831530  2757 rpcz_store.cc:269] 0709 14:52:34.124526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:34.364987 (+240461us) service_pool.cc:224] Handling call
0709 14:52:37.830627 (+3465640us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.830773 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6195023232}
I20260709 14:52:37.835657  2681 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:37.805239  5788 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:37.816566  2686 rpcz_store.cc:276] 0709 14:52:35.342288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.644865 (+1302577us) service_pool.cc:224] Handling call
0709 14:52:36.646049 (+  1184us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:37.518547 (+872498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.815541 (+296994us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540024ead4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.815784 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985018624}
W20260709 14:52:37.846318  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1159) took 4147 ms (client timeout 2499 ms). Trace:
I20260709 14:52:37.815346  5891 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:37.846974  5974 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 14:52:37.847460  2759 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:37.847725  5380 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:37.813503  2756 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:52:37.832155  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:37.818758  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1160) took 4114 ms. Trace:
I20260709 14:52:37.881330  3540 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:37.850394  2755 rpcz_store.cc:276] 0709 14:52:36.543724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.598741 (+ 55017us) service_pool.cc:224] Handling call
0709 14:52:37.822054 (+1223313us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.822254 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797555968}
W20260709 14:52:37.870117  2754 rpcz_store.cc:269] 0709 14:52:33.699100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.699285 (+   185us) service_pool.cc:224] Handling call
0709 14:52:37.845831 (+4146546us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54000e6354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:37.846079 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739512704}
W20260709 14:52:37.878746  5941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.009s	sys 0.052s
I20260709 14:52:37.884369  3536 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:37.861155  5734 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.367s	user 0.016s	sys 0.001s
I20260709 14:52:37.884847  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:37.879884  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:37.884083  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 144284us
I20260709 14:52:37.881229  2751 rpcz_store.cc:276] 0709 14:52:33.704649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.704869 (+   220us) service_pool.cc:224] Handling call
0709 14:52:33.810019 (+105150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:37.723296 (+3913277us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b5400162c20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bba1d 00007fa16e5a21a4 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419
0709 14:52:37.800317 (+ 77021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.818362 (+ 18045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141591424}
I20260709 14:52:37.881071  2753 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:37.892783  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 217651us
I20260709 14:52:37.892867  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.894017  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:37.872293  4786 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:37.895205  2757 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:37.896193  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 568) took 1345 ms. Trace:
W20260709 14:52:37.896229  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96787.1us
I20260709 14:52:37.897069  4786 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:37.893029  5651 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:37.896788  2757 rpcz_store.cc:276] 0709 14:52:36.550539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.894535 (+1343996us) service_pool.cc:224] Handling call
0709 14:52:37.894998 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:37.895825 (+   827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.896090 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:37.899204  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.900717  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:37.901671  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:37.902607  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:37.903020  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1042) took 4089 ms. Trace:
W20260709 14:52:37.907493  2576 connection.cc:256] RPC call timeout handler was delayed by -1.7247139380000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2945s for a response
W20260709 14:52:37.908319  2576 connection.cc:256] RPC call timeout handler was delayed by -1.4070139379999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32980000000000004s for a response
I20260709 14:52:37.911176  3542 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:37.914774  2757 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:37.888978  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.916014  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.920922  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1050) took 2727 ms. Trace:
I20260709 14:52:37.903556  3547 rpcz_store.cc:276] 0709 14:52:33.790481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.517853 (+2727372us) service_pool.cc:224] Handling call
0709 14:52:36.518589 (+   736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:36.519343 (+   754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:36.519378 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:36.520323 (+   945us) log.cc:844] Serialized 26 byte log entry
0709 14:52:36.884536 (+364213us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:36.884620 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:36.884747 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:37.879887 (+995140us) raft_consensus.cc:1685] finished
0709 14:52:37.879912 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.880075 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1868,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":177,"thread_start_us":311716,"threads_started":1,"wal-append.queue_time_us":1037601}
I20260709 14:52:37.922256  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 569) took 1280 ms. Trace:
I20260709 14:52:37.922358  5983 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:37.931217  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.931221  5870 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:37.933979  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1171) took 4259 ms. Trace:
I20260709 14:52:37.935066  5953 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:52:37.885982  5941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.010s	sys 0.052s
I20260709 14:52:37.940197  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.940955  2757 rpcz_store.cc:276] 0709 14:52:36.642142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.898481 (+1256339us) service_pool.cc:224] Handling call
0709 14:52:37.904201 (+  5720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:37.921843 (+ 17642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.922150 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:37.918541  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:37.917719  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:37.980757  2604 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:37.978456  2608 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:37.951321  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:37.942477  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:37.954850  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:37.957001  2610 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:37.957973  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:37.959262  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.962456  5685 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:37.963066  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:37.963673  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:37.965144  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.965488  5875 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.965983  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:37.978422  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 173277us
I20260709 14:52:37.988667  2754 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:37.989063  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:37.989836  2603 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:37.991446  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:37.995375  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.000903  2758 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:37.987864  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
W20260709 14:52:38.002111  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 207731us
I20260709 14:52:38.004778  2612 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.006469  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:37.900592  5867 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.976962  5763 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:37.966409  5870 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.004516  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:37.913044  5734 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.001206  2609 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:38.000938  2607 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:38.016510  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 566) took 1454 ms. Trace:
I20260709 14:52:38.244634  2682 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 14:52:38.245096  2605 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
I20260709 14:52:38.068666  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.245663  2605 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:37.968108  2606 rpcz_store.cc:276] 0709 14:52:33.674549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.703913 (+ 29364us) service_pool.cc:224] Handling call
0709 14:52:33.810071 (+106158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:34.272124 (+462053us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:34.625318 (+353194us) log.cc:844] Serialized 14 byte log entry
0709 14:52:37.448690 (+2823372us) log.cc:844] Serialized 14 byte log entry
0709 14:52:37.464656 (+ 15966us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:37.473827 (+  9171us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:37.483412 (+  9585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.933664 (+450252us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400480654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:37.933728 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.933911 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15296,"prepare.queue_time_us":2007518,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":2360,"raft.queue_time_us":1158,"raft.run_cpu_time_us":785,"raft.run_wall_time_us":187074,"spinlock_wait_cycles":1061815040,"thread_start_us":2835185,"threads_started":3,"wal-append.queue_time_us":1129927}
I20260709 14:52:38.143246  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.247236  2604 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:38.156472  5875 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.248401  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.249297  2604 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.120457  2612 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 4. Current term is 5. Ops: []
I20260709 14:52:38.244496  2758 rpcz_store.cc:276] 0709 14:52:36.547761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.904987 (+1357226us) service_pool.cc:224] Handling call
0709 14:52:37.916514 (+ 11527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:38.001551 (+ 85037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.001810 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:38.067785  3409 leader_election.cc:400] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:38.254295  3409 leader_election.cc:403] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 14:52:38.183046  5900 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:38.255610  5875 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:37.977037  5994 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 14:52:38.184536  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:38.225909  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 216792us
I20260709 14:52:38.255465  5763 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:37.940582  3545 rpcz_store.cc:276] 0709 14:52:35.193624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.611792 (+1418168us) service_pool.cc:224] Handling call
0709 14:52:36.723368 (+111576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:36.723835 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:37.408525 (+684690us) log.cc:844] Serialized 14 byte log entry
0709 14:52:37.670940 (+262415us) log.cc:844] Serialized 14 byte log entry
0709 14:52:37.903386 (+232446us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b1800331660. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:37.920215 (+ 16829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:37.920311 (+    96us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:37.920418 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:37.920550 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:37.920818 (+   268us) inbound_call.cc:177] Queueing success response
0709 14:52:37.935066 (+ 14248us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b1800331660. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
Metrics: {"mutex_wait_us":202096,"prepare.queue_time_us":845513,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":92,"raft.queue_time_us":246441,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":29425,"spinlock_wait_cycles":15367808,"thread_start_us":831423,"threads_started":3,"wal-append.queue_time_us":257374}
I20260709 14:52:37.984654  5914 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:38.017984  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80332.7us
I20260709 14:52:37.984990  5793 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:38.007706  2684 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:38.179983  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 558) took 1636 ms. Trace:
I20260709 14:52:38.148846  2603 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:38.017355  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.004516  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.087325  3546 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:38.120467  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.121333  5777 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:38.196014  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 574) took 1514 ms. Trace:
I20260709 14:52:38.287297  2751 rpcz_store.cc:276] 0709 14:52:36.665675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.912489 (+1246814us) service_pool.cc:224] Handling call
0709 14:52:37.977906 (+ 65417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:38.179080 (+201174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.179130 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:38.179206 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:38.179268 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.179377 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:38.179578 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29450,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":56884}
I20260709 14:52:38.289583  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.290656  2603 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:38.252234  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:38.293807  2612 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:38.086416  5992 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 14:52:38.085587  5998 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.119593  5752 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:38.306999  5752 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:38.307474  5752 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:38.308460  5763 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:38.309134  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:38.309947  2682 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.312338  5683 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:38.313640  5683 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:38.118438  5818 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:38.314797  5651 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:38.315344  5867 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:52:38.016974  5788 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.317123  5818 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:38.270879  2606 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 14:52:38.318082  2606 pending_rounds.cc:85] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:52:38.318454  2606 pending_rounds.cc:107] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 14:52:38.318442  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.319113  2606 raft_consensus.cc:2889] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:38.319726  2752 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:38.306568  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.320921  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.321763  2606 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.322229  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:38.323001  2612 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:37.989894  5685 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:38.324867  5685 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.326368  5685 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:38.274223  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.315402  5651 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:38.327726  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.327973  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 897) took 5284 ms. Trace:
I20260709 14:52:38.328184  5651 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.328544  3546 rpcz_store.cc:276] 0709 14:52:33.043082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:33.043756 (+   674us) service_pool.cc:224] Handling call
0709 14:52:33.077512 (+ 33756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:37.515309 (+4437797us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b58003b18d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5a20aa 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4
0709 14:52:37.860470 (+345161us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b58007808d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5a20aa 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4
0709 14:52:38.279982 (+419512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.327852 (+ 47870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161009792}
I20260709 14:52:38.315886  2679 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:38.322717  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.087316  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.278650  2759 rpcz_store.cc:276] 0709 14:52:36.524601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:36.524953 (+   352us) service_pool.cc:224] Handling call
0709 14:52:37.846904 (+1321951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:37.875180 (+ 28276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.152784 (+277604us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54000e6350. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:38.160761 (+  7977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3038411008}
I20260709 14:52:38.277724  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.259658  5870 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.256206  5900 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.335086  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.335359  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:38.336735  2686 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:38.338202  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:38.304869  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.296075  5777 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.333305  5651 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.339491  5903 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:38.340597  5900 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:38.250495  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.356220  2604 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:38.343568  2609 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:38.333441  2606 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.306553  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:38.358021  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 419688us
I20260709 14:52:38.358615  2678 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.359699  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:38.361123  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:37.988562  5983 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:38.274164  5734 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.369081  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:38.373227  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 224495us
I20260709 14:52:38.374534  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.375864  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:38.376307  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.284547  5793 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.354888  5088 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:38.401800  2604 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:38.404946  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:38.364135  5777 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:38.410039  2606 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:38.414149  2685 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 14:52:38.399600  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.400454  2603 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:52:38.429548  2754 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 14:52:38.451220  2754 pending_rounds.cc:85] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
I20260709 14:52:38.451570  2754 pending_rounds.cc:107] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 14:52:38.451853  2754 raft_consensus.cc:2889] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:38.369652  5953 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:38.389165  5788 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:38.437417  2609 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:38.438907  2611 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:38.440871  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 163280us
I20260709 14:52:38.478782  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:38.482250  2605 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 14:52:38.487327  2757 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:52:38.487054  5986 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 14:52:38.488190  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.488391  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:38.488747  5986 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:38.489283  5986 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:38.489379  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.491086  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.491854  2752 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:38.489588  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:38.369124  5914 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.500391  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:38.429687  5763 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:38.495833  2606 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:38.498380  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.495344  2681 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:38.505658  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65234.2us
W20260709 14:52:38.477471  5496 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.001s
W20260709 14:52:38.508932  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.511907  2612 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:38.513056  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:38.515730  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.514067  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:38.514259  2682 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:38.519307  2605 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 14:52:38.521018  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 161453us
I20260709 14:52:38.524325  2603 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 14:52:38.368707  5998 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:38.528115  2726 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.727s (SENT)
I20260709 14:52:38.531679  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:38.440214  5088 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.496599  5788 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.541715  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.547919  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:38.551316  2682 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.553089  2754 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:38.506598  5763 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:38.526671  5734 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:38.555508  2608 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:38.555780  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.567308  2752 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 14:52:38.558547  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58260.2us
I20260709 14:52:38.559499  2609 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:52:38.559712  3543 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.550061  2610 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 14:52:38.541494  5405 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.897s	user 0.011s	sys 0.011s
W20260709 14:52:38.563446  2726 leader_election.cc:336] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.964s (SENT)
I20260709 14:52:38.572219  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:38.594762  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.573203  3536 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:38.599603  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:38.565941  5763 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:38.614310  2754 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:38.524271  5496 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:38.584493  5885 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.007s	user 0.013s	sys 0.004s
I20260709 14:52:38.606808  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:38.612634  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:38.554556  5788 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.623558  2685 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:38.590371  5953 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:38.636075  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:38.641700  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 124271us
I20260709 14:52:38.603519  5959 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:38.645499  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46044.3us
I20260709 14:52:38.647955  5963 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:38.650810  5963 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:38.651306  5963 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.651528  5871 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.652271  5871 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.589283  5718 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 14:52:38.659584  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 207538us
I20260709 14:52:38.660418  2684 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:38.655990  2679 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:38.638403  5937 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:38.674064  2651 connection.cc:256] RPC call timeout handler was delayed by -1.6814085439999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2577s for a response
I20260709 14:52:38.674985  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.639406  5885 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.694201  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:38.670962  5871 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.696249  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:38.669781  5895 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:38.668895  5914 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.701894  2682 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:38.656425  5959 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:38.705034  2678 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:38.692746  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:38.712934  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 179507us
I20260709 14:52:38.724668  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:38.724370  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 125878us
I20260709 14:52:38.490541  5986 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:38.722934  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:38.731572  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57169.3us
I20260709 14:52:38.727788  3545 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:38.736181  2751 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:52:38.734501  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:38.703275  5405 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:38.799034  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.715552  5895 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:38.735303  5914 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:38.801465  5718 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:38.842741  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 110619us
I20260709 14:52:38.855304  6015 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.856184  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:38.860646  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:38.861541  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.862253  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.805388  5405 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:38.812487  3543 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:38.863452  2605 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:38.863366  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:38.822527  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 253962us
I20260709 14:52:38.865376  2603 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.865507  5405 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.866195  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1172) took 1017 ms. Trace:
W20260709 14:52:38.866988  5886 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.336s	user 0.009s	sys 0.009s
I20260709 14:52:38.867273  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.868237  2606 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:38.867576  5886 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:38.868846  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.869412  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:38.870311  2611 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:38.871250  3536 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:52:38.871862  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:38.872303  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.872641  2608 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:38.873790  5995 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
W20260709 14:52:38.853794  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 129230us
I20260709 14:52:38.876338  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:38.876924  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:38.854862  6013 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:38.862254  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:38.829049  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 87924.5us
W20260709 14:52:38.868228  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.840497  5959 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:38.881393  5959 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:38.733624  5793 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.890499  5088 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.805194  5916 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.134s
I20260709 14:52:38.900094  5916 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:38.899966  5900 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:38.814353  5685 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:38.844699  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 555) took 1225 ms. Trace:
I20260709 14:52:38.901288  5405 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:38.901909  5405 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:38.901746  2677 rpcz_store.cc:276] 0709 14:52:37.619262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.619491 (+   229us) service_pool.cc:224] Handling call
0709 14:52:38.844442 (+1224951us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:38.844583 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2229238528}
I20260709 14:52:38.903184  5405 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:38.881266  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:38.906183  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:38.812467  6000 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 14:52:38.847900  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:38.853453  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.731153  5902 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:38.839340  5766 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:38.828390  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101336us
I20260709 14:52:38.821599  2681 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:38.881862  5959 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:38.891938  5913 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:52:38.893083  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:38.893234  6006 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 14:52:38.893765  5793 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 14:52:38.894196  6003 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 14:52:38.895277  6002 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 14:52:38.896178  6007 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:38.896704  5885 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:38.898070  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:38.814623  5895 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:38.901194  5877 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.075s	user 0.005s	sys 0.013s
I20260709 14:52:38.903784  5986 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:38.882863  5734 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:38.902510  3542 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:38.901099  5916 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:38.880513  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.866705  2760 rpcz_store.cc:276] 0709 14:52:37.848444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.985416 (+136972us) service_pool.cc:224] Handling call
0709 14:52:38.017338 (+ 31922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:38.121211 (+103873us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:38.568310 (+447099us) log.cc:844] Serialized 14 byte log entry
0709 14:52:38.677454 (+109144us) log.cc:844] Serialized 14 byte log entry
0709 14:52:38.800922 (+123468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:38.801016 (+    94us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:38.865871 (+ 64855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.865971 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:38.866097 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8472,"prepare.queue_time_us":511481,"raft.queue_time_us":30563,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":11593,"spinlock_wait_cycles":157709568,"thread_start_us":463159,"threads_started":3,"wal-append.queue_time_us":93203}
I20260709 14:52:38.870515  5886 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.929039  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:38.890676  5871 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 14:52:38.924263  5752 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.616s	user 0.013s	sys 0.003s
W20260709 14:52:38.936322  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 261451us
W20260709 14:52:38.949766  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:38.989746  2752 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:38.956492  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:38.958236  5998 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:38.920221  5902 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:38.926249  5959 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.973160  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.973313  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:38.854213  5718 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:38.956068  5088 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 14:52:38.913491  5914 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.015130  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:39.015849  2611 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:39.017006  5914 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:39.017740  5914 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:39.018105  5914 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:39.018489  5914 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:38.956352  5405 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.958736  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.005162  5902 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:39.022900  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:39.023324  5793 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:39.023838  5793 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:39.024245  5793 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:38.999782  5752 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.026332  2677 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:39.027407  5752 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:38.982436  5734 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:39.031724  5734 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.033006  5734 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:39.038820  5867 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.005s
I20260709 14:52:39.039220  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:39.039739  6008 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.227s	user 0.004s	sys 0.000s
I20260709 14:52:39.040225  3542 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:39.040078  2680 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:39.041280  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.041832  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:38.993052  3537 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:38.926934  5777 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.043035  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.043916  2758 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:39.043635  5963 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.044728  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.045012  5777 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:38.993872  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.045922  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.046368  3541 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:39.000959  2612 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:39.048370  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.048624  5718 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:39.005155  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.049779  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.050748  2760 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:52:39.051438  6014 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 14:52:39.051955  2752 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:39.006142  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 906) took 1179 ms. Trace:
I20260709 14:52:39.008284  5496 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:39.012614  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.050971  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.052486  3538 rpcz_store.cc:276] 0709 14:52:37.855332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.873008 (+ 17676us) service_pool.cc:224] Handling call
0709 14:52:37.904203 (+ 31195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:38.001108 (+ 96905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.001141 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:38.261147 (+260006us) log.cc:844] Serialized 26 byte log entry
0709 14:52:38.403538 (+142391us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:38.403587 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.403675 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.005867 (+602192us) raft_consensus.cc:1685] finished
0709 14:52:39.005894 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.006069 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":133095,"threads_started":1,"wal-append.queue_time_us":332502}
I20260709 14:52:39.045783  2759 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:39.058548  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 577) took 2381 ms. Trace:
I20260709 14:52:39.059314  2756 rpcz_store.cc:276] 0709 14:52:36.677321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.914993 (+1237672us) service_pool.cc:224] Handling call
0709 14:52:37.953761 (+ 38768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:37.964743 (+ 10982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:37.964776 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:37.965492 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 14:52:38.357232 (+391740us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:38.357285 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.357370 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.058213 (+700843us) raft_consensus.cc:1685] finished
0709 14:52:39.058247 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.058450 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8561,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":8753,"thread_start_us":391271,"threads_started":1,"wal-append.queue_time_us":680358}
I20260709 14:52:39.062183  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 560) took 1227 ms. Trace:
I20260709 14:52:39.062904  2683 rpcz_store.cc:276] 0709 14:52:37.849773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.864251 (+ 14478us) service_pool.cc:224] Handling call
0709 14:52:37.864901 (+   650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:37.865492 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:37.865526 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:37.866096 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 14:52:38.209355 (+343259us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:38.209407 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:38.209495 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.061858 (+852363us) raft_consensus.cc:1685] finished
0709 14:52:39.061895 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.062096 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6334,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":3419,"thread_start_us":342754,"threads_started":1,"wal-append.queue_time_us":529158}
I20260709 14:52:38.989581  6012 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:39.065366  6012 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.065986  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 996) took 1108 ms. Trace:
I20260709 14:52:39.066449  6012 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.068109  6012 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:39.066457  2680 rpcz_store.cc:276] 0709 14:52:37.957728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:37.977594 (+ 19866us) service_pool.cc:224] Handling call
0709 14:52:39.065802 (+1088208us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:39.065922 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748977920}
I20260709 14:52:39.071159  6016 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:38.986054  5916 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:39.072817  5916 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:38.990657  6011 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 14:52:39.075376  6015 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:38.991438  5885 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 14:52:39.080286  6009 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 14:52:39.083365  5870 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.749s	user 0.005s	sys 0.011s
I20260709 14:52:39.084138  5870 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.088548  5900 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.089545  5900 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.013800  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.090680  5793 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.014715  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.091956  3547 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:39.092154  5900 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:39.092944  5793 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:38.992452  5877 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
W20260709 14:52:39.096311  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59760us
I20260709 14:52:39.097846  5685 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:52:39.098908  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57317.6us
W20260709 14:52:39.102299  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 103465us
I20260709 14:52:39.102978  6018 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.103152  3538 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:39.104447  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:39.105757  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.106132  3543 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:39.008957  6020 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:39.107969  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.108573  6017 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.109464  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:39.147715  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 243048us
I20260709 14:52:39.116547  5895 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.117687  5983 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:39.121738  5953 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 14:52:39.128775  4786 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.011s	sys 0.012s
I20260709 14:52:39.146193  3547 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6. Re-sending same reply.
I20260709 14:52:39.155416  3541 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 14:52:39.159240  4786 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.012s	sys 0.012s
I20260709 14:52:39.160269  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:39.156908  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.012s	sys 0.015s
W20260709 14:52:39.161907  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.012s	sys 0.015s
I20260709 14:52:39.166031  2753 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:39.168664  3547 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:52:39.045852  5963 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:39.170928  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:39.160002  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.022s
I20260709 14:52:39.174819  3538 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:39.175026  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.005s	sys 0.022s
I20260709 14:52:39.149438  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.142189  5877 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:39.182201  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:39.179019  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:39.183352  3537 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:39.185042  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.179544  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:39.078342  5902 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.169572  6020 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:39.203967  6020 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.205497  5405 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:39.206035  5405 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:39.206415  5405 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:39.155640  6017 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.209236  6017 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:39.209466  3547 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:39.213994  3541 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 14:52:39.215663  3541 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:52:39.147464  5998 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 14:52:39.217417  3541 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 14:52:39.216943  5718 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 14:52:39.219383  5895 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:39.178781  5986 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 14:52:39.220866  6018 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:39.221599  6019 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:39.222105  6019 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:39.135800  5886 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 14:52:39.217924  3543 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:39.218750  3537 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:39.224205  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54186.1us
I20260709 14:52:39.224905  3547 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:39.226528  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.231027  3541 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:39.128815  5763 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:39.234130  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56117.8us
I20260709 14:52:39.230443  3540 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 4.
I20260709 14:52:39.163964  6007 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.225232  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:39.235486  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.003s	sys 0.014s
I20260709 14:52:39.240583  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.244223  2758 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:39.244732  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.004s	sys 0.015s
I20260709 14:52:39.246320  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.251904  2759 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:39.253651  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:39.241794  5937 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:39.254443  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.241791  5963 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 14:52:39.255115  5986 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:39.251415  6019 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.255767  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.255895  6011 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:39.256471  6019 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:39.256831  6011 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.257143  5986 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:39.257547  5986 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:39.257881  6011 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.255462  5963 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:39.261022  5651 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:39.261353  5963 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:52:39.262118  5651 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:39.262465  5683 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:39.262770  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:39.180702  5870 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:39.265347  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.199489  5793 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:39.268076  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.268774  3536 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:39.270934  2758 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:39.234031  5885 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:39.277814  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:39.278542  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.674s (SENT)
I20260709 14:52:39.299479  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.299610  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.300177  2752 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:39.300546  3543 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:39.299682  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.304534  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.306366  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.311038  2751 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:39.265666  6013 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.311617  5937 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.312242  5937 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.312270  6013 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.299111  6018 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:39.315318  5983 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:39.263017  5683 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:39.323972  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45483.9us
I20260709 14:52:39.279940  5380 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:39.354091  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49683.2us
I20260709 14:52:39.362393  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:39.375839  2756 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:39.366755  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:39.279860  6011 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:39.338873  5937 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:39.258482  5986 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.262354  5963 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 14:52:39.263195  5651 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 14:52:39.246304  5752 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 14:52:39.315670  6018 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:39.339339  5983 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.368674  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:39.246276  5914 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:39.378712  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.315589  5685 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 14:52:39.381173  3540 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:39.354483  5900 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:39.385848  3542 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:39.384013  5766 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 14:52:39.348110  5788 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.002s
W20260709 14:52:39.405860  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.026s	sys 0.010s
I20260709 14:52:39.343055  3538 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:39.400861  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.343497  5877 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:39.405453  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.020s	sys 0.015s
W20260709 14:52:39.446203  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.021s	sys 0.015s
W20260709 14:52:39.440490  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.026s	sys 0.010s
I20260709 14:52:39.364578  5763 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:39.370857  5380 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:39.437984  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.043s
W20260709 14:52:39.407497  6005 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.022s
W20260709 14:52:39.447369  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 561192us
I20260709 14:52:39.353390  5405 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:39.446842  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.018s	sys 0.031s
W20260709 14:52:39.479830  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.019s	sys 0.031s
W20260709 14:52:39.480494  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79758.3us
W20260709 14:52:39.462883  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.005s	sys 0.043s
I20260709 14:52:39.473902  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:39.404955  5913 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:39.475713  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100972us
W20260709 14:52:39.489617  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 596543us
I20260709 14:52:39.491786  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.386284  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:39.445521  5885 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:39.472908  6005 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.000s	sys 0.023s
I20260709 14:52:39.350451  5683 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.508605  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:39.513667  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:39.515146  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:39.517143  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:39.493216  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.002s	sys 0.023s
I20260709 14:52:39.474788  5986 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:39.525713  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44177.4us
W20260709 14:52:39.527701  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 621545us
W20260709 14:52:39.523118  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.002s	sys 0.023s
I20260709 14:52:39.523864  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:39.530977  2678 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:39.522435  2681 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:39.534797  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.936s (SENT)
W20260709 14:52:39.535192  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.022s	sys 0.035s
I20260709 14:52:39.535547  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:39.536509  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.022s	sys 0.035s
I20260709 14:52:39.558503  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:39.492326  5752 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:39.558457  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65586.4us
I20260709 14:52:39.460397  5788 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:39.576344  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48667.4us
W20260709 14:52:39.579145  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:39.566008  6019 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:39.563946  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46889.4us
I20260709 14:52:39.573967  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:39.574813  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:39.492969  5871 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:39.628888  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71057.1us
I20260709 14:52:39.630251  2685 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:39.626060  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:39.523610  6016 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:39.639847  2684 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:39.640867  2680 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:39.525997  5496 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:39.490411  5953 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 14:52:39.649611  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:39.651458  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73202.3us
W20260709 14:52:39.649492  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 772604us
I20260709 14:52:39.660395  2677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:52:39.613761  6019 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:39.494839  5916 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.664435  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:39.568693  5900 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.671597  3543 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:39.675309  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.663430  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.683120  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:39.680795  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.682404  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:39.682999  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68718.1us
I20260709 14:52:39.639344  5877 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:39.685772  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151044us
I20260709 14:52:39.660488  5718 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 14:52:39.701858  2611 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:39.702533  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 397014us
I20260709 14:52:39.702363  2684 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:39.708081  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.043s (SENT)
I20260709 14:52:39.684247  2605 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:39.708637  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:39.655603  5903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.050s	sys 0.092s
I20260709 14:52:39.686738  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:39.702230  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47448.7us
I20260709 14:52:39.690116  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:39.692143  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:39.705303  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 443351us
I20260709 14:52:39.518716  6029 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 14:52:39.717746  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:39.714526  2612 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 14:52:39.713795  2606 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:39.718623  2603 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:39.716044  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.582479  5885 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 14:52:39.582624  5793 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 14:52:39.725610  2612 pending_rounds.cc:85] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 3
I20260709 14:52:39.739305  2612 pending_rounds.cc:107] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 14:52:39.739674  2612 raft_consensus.cc:2889] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:39.665783  5902 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:39.679267  5963 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:39.737522  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 512369us
I20260709 14:52:39.688685  6019 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:39.740805  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:39.742084  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 92590.4us
I20260709 14:52:39.749943  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:39.755899  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 72868.2us
I20260709 14:52:39.759742  4786 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:39.690605  5959 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.000s
W20260709 14:52:39.718305  5903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.050s	sys 0.096s
I20260709 14:52:39.709342  6034 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.712957  5752 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:39.773648  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.775210  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:39.778311  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70261.2us
I20260709 14:52:39.781369  4786 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:39.785071  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.785718  2605 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:39.699446  6017 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
W20260709 14:52:39.787247  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106620us
I20260709 14:52:39.788959  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:39.583268  5380 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:39.753615  5496 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.794808  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:39.789489  4786 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:39.798002  5380 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:39.799268  5886 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:39.799785  6033 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:39.800312  5886 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:39.800709  5886 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.801131  6033 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:39.801568  6033 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:39.801704  5886 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.698460  6036 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 14:52:39.806077  6036 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.768829  6015 raft_consensus.cc:2764] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 14:52:39.807664  6024 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 14:52:39.808135  6025 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 14:52:39.808355  5914 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:39.761404  5913 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.809401  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1071) took 1104 ms. Trace:
I20260709 14:52:39.809964  2754 rpcz_store.cc:276] 0709 14:52:38.705192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.705357 (+   165us) service_pool.cc:224] Handling call
0709 14:52:38.726350 (+ 20993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:38.801060 (+ 74710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.801108 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:38.812457 (+ 11349us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.163572 (+351115us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:39.163620 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.164982 (+  1362us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.809148 (+644166us) raft_consensus.cc:1685] finished
0709 14:52:39.809177 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.809334 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1074,"raft.queue_time_us":93446,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":113,"thread_start_us":340454,"threads_started":1,"wal-append.queue_time_us":699039}
I20260709 14:52:39.782142  5734 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:39.809485  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1181) took 1727 ms. Trace:
W20260709 14:52:39.812646  6027 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 14:52:39.811868  2607 rpcz_store.cc:276] 0709 14:52:38.081797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.374140 (+292343us) service_pool.cc:224] Handling call
0709 14:52:38.414526 (+ 40386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:38.499304 (+ 84778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.499346 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:38.510720 (+ 11374us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.175588 (+664868us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:39.175636 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.175724 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.809238 (+633514us) raft_consensus.cc:1685] finished
0709 14:52:39.809261 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.809398 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5463,"raft.queue_time_us":65581,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8640,"thread_start_us":578391,"threads_started":1,"wal-append.queue_time_us":948333}
I20260709 14:52:39.780519  5959 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:39.814211  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.814397  5766 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:52:39.815183  6026 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 14:52:39.757694  5871 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.815387  5959 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:39.818392  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1004) took 1641 ms. Trace:
I20260709 14:52:39.756938  5963 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:39.822688  2607 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:39.825718  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.826704  2612 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:39.791054  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.828984  2756 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:39.814405  6023 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 14:52:39.834525  2684 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:39.770192  5877 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:39.773284  5902 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:39.839937  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73804.6us
I20260709 14:52:39.779183  5793 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:39.840840  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:39.853739  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64890.4us
W20260709 14:52:39.854092  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63188.3us
I20260709 14:52:39.629282  6018 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.785308  6020 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:39.818783  2686 rpcz_store.cc:276] 0709 14:52:38.177092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.341848 (+164756us) service_pool.cc:224] Handling call
0709 14:52:38.344229 (+  2381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:38.367743 (+ 23514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.367799 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:38.432223 (+ 64424us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.162588 (+730365us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:39.162656 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.167476 (+  4820us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:39.818165 (+650689us) raft_consensus.cc:1685] finished
0709 14:52:39.818186 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:39.818326 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13719,"raft.queue_time_us":17645,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":29296,"thread_start_us":645706,"threads_started":1,"wal-append.queue_time_us":1004696}
W20260709 14:52:39.842932  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 152893us
I20260709 14:52:39.762866  5900 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 14:52:39.891693  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:39.729404  6032 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 14:52:39.853782  4786 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:39.912902  4786 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:39.893235  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.900328  5963 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:39.793881  6016 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:39.911037  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118077us
W20260709 14:52:39.912786  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118092us
I20260709 14:52:39.806716  6036 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.917727  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:39.842514  5886 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:39.842569  6029 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:39.930930  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.928305  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:39.928519  2605 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:39.845016  5752 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:39.890622  5793 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:39.843605  6041 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:39.944171  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:39.903429  5903 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:39.936194  4786 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.895695  6020 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:39.824936  5496 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:39.818996  6034 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:39.917472  5870 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:39.889546  5902 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:39.968034  5880 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 14:52:39.969105  6028 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 14:52:39.924013  5959 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:39.943128  5734 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:39.925899  5877 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:39.931951  6047 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:52:40.014503  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96866.4us
I20260709 14:52:40.015547  2677 raft_consensus.cc:606] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:40.015772  3412 leader_election.cc:400] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 6 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 6. Current term is 7.
I20260709 14:52:40.010222  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:39.929414  5718 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 14:52:40.012660  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:39.970535  6020 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:39.961211  6017 raft_consensus.cc:2764] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 14:52:39.944185  5963 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:39.984265  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 243556us
W20260709 14:52:40.020494  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51344.5us
I20260709 14:52:40.022323  2612 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:40.012733  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:40.073364  2682 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 14:52:40.033583  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:39.983042  6018 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:40.067335  2684 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:40.070010  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.077785  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 910) took 1479 ms. Trace:
I20260709 14:52:39.968159  6037 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:52:40.082193  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 189036us
I20260709 14:52:40.078339  3546 rpcz_store.cc:276] 0709 14:52:38.598368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.609564 (+ 11196us) service_pool.cc:224] Handling call
0709 14:52:38.610177 (+   613us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:38.701852 (+ 91675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.701894 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:38.712330 (+ 10436us) log.cc:844] Serialized 52 byte log entry
0709 14:52:39.150316 (+437986us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:39.163208 (+ 12892us) log.cc:844] Serialized 14 byte log entry
0709 14:52:39.249984 (+ 86776us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b18003b7d00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:39.250279 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.250396 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.065634 (+815238us) raft_consensus.cc:1685] finished
0709 14:52:40.077481 (+ 11847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.077704 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92979,"prepare.queue_time_us":9487,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":5874,"raft.queue_time_us":202251,"raft.run_cpu_time_us":480,"raft.run_wall_time_us":73381,"spinlock_wait_cycles":17403648,"thread_start_us":408956,"threads_started":2,"wal-append.queue_time_us":741238}
I20260709 14:52:39.944752  6029 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.018083  5788 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:40.090248  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.097941  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.095731  5734 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.100088  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:40.105633  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:40.112419  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96811.6us
I20260709 14:52:40.041002  5880 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:40.080287  6047 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:40.108114  5937 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:40.150615  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53558.1us
I20260709 14:52:40.161324  2757 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:40.149499  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:40.172271  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:40.178372  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:40.190652  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1186) took 1889 ms. Trace:
I20260709 14:52:40.196674  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:40.120415  5734 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.161137  6017 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.151798  5937 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:40.191380  2604 rpcz_store.cc:276] 0709 14:52:38.301571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.428984 (+127413us) service_pool.cc:224] Handling call
0709 14:52:38.452058 (+ 23074us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:38.500007 (+ 47949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:38.500040 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:38.558458 (+ 58418us) log.cc:844] Serialized 52 byte log entry
0709 14:52:39.245372 (+686914us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:39.245440 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.245544 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.082803 (+837259us) raft_consensus.cc:1685] finished
0709 14:52:40.082859 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.171883 (+ 89024us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54001eb750. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:40.190530 (+ 18647us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52382,"raft.queue_time_us":51008,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5761,"spinlock_wait_cycles":51558144,"thread_start_us":686349,"threads_started":1,"wal-append.queue_time_us":1231160}
I20260709 14:52:40.083981  5793 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.120141  6053 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 14:52:40.238056  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:40.190766  5766 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:40.200224  5718 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 14:52:40.240151  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 90862.8us
I20260709 14:52:40.205742  5867 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:40.200719  6012 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.135067  5752 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.206156  6047 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:40.259889  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 159936us
I20260709 14:52:40.171316  5913 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 14:52:40.248338  5788 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 14:52:40.267263  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:40.262039  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162443us
I20260709 14:52:40.097036  6016 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.160333  5877 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.180001  6052 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 14:52:40.313761  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:40.316350  2607 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:40.247895  6046 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:40.313859  5913 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:40.205283  6059 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 14:52:40.295004  5766 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.292742  5914 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.340140  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:40.353842  2611 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.002s
I20260709 14:52:40.366227  2611 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:40.320921  6036 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:40.369612  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:40.297297  5885 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 14:52:40.324584  6030 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:40.378988  2607 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
I20260709 14:52:40.290660  6017 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.341149  5496 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:40.346879  6046 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.394480  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:40.396646  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.362980  5766 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:40.305002  6012 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.359525  5788 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:40.415381  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:40.415452  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:40.377797  5902 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:40.415113  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.376960  5871 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:40.418021  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.395107  6034 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:40.415621  6041 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:40.415472  6050 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
I20260709 14:52:40.292987  5880 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.372754  5914 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.406456  6046 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:40.428696  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:40.469976  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:40.376855  5937 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:40.448165  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.008s
W20260709 14:52:40.477317  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.008s	sys 0.008s
W20260709 14:52:40.478221  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 703043us
I20260709 14:52:40.479255  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:40.425412  6051 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.338s	user 0.000s	sys 0.002s
I20260709 14:52:40.472273  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:40.443019  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.474136  2612 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:40.424026  5867 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:40.393883  6030 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:40.492105  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 77097.3us
I20260709 14:52:40.408109  6064 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 14:52:40.475325  6070 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 14:52:40.498229  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.430094  6047 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:40.428289  6069 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:40.530355  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:40.539438  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:40.539814  6049 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 14:52:40.435981  5983 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.540863  5983 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:40.541721  2606 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 14:52:40.438110  6041 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.543226  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1094) took 1266 ms. Trace:
I20260709 14:52:40.544659  2603 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:40.501535  6039 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.332s	user 0.000s	sys 0.002s
I20260709 14:52:40.545172  6041 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:40.546314  6033 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.546885  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1204) took 1567 ms. Trace:
I20260709 14:52:40.547592  6033 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:40.547453  2608 rpcz_store.cc:276] 0709 14:52:38.979137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:38.980354 (+  1217us) service_pool.cc:224] Handling call
0709 14:52:38.986880 (+  6526us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:39.011016 (+ 24136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:39.011061 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:39.099340 (+ 88279us) log.cc:844] Serialized 52 byte log entry
0709 14:52:39.530400 (+431060us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:39.530463 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.530574 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.546549 (+1015975us) raft_consensus.cc:1685] finished
0709 14:52:40.546582 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.546773 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6225,"raft.queue_time_us":51663,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":430477,"threads_started":1,"wal-append.queue_time_us":935188}
W20260709 14:52:40.504536  6056 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
W20260709 14:52:40.505816  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.508205  5788 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.551417  6069 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:40.552733  6069 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.553901  5405 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:40.510085  6057 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 14:52:40.511391  6063 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
W20260709 14:52:40.514314  6060 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 14:52:40.430236  5871 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.543771  2753 rpcz_store.cc:276] 0709 14:52:39.276965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:39.292355 (+ 15390us) service_pool.cc:224] Handling call
0709 14:52:39.339565 (+ 47210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:39.340084 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:39.340133 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:39.367058 (+ 26925us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.717831 (+350773us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:39.718018 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.718234 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.542919 (+824685us) raft_consensus.cc:1685] finished
0709 14:52:40.542952 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.543146 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":301159,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":349936,"threads_started":1,"wal-append.queue_time_us":766854}
I20260709 14:52:40.515962  6017 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:40.518036  2607 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:40.520296  6012 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:40.521297  5937 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:40.522079  6029 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:40.523037  6038 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:40.524017  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:40.596086  5986 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.039s	user 0.012s	sys 0.004s
I20260709 14:52:40.524272  5914 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:40.524842  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:40.530786  5734 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:40.530243  5867 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:40.531839  5903 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:40.539695  2610 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:40.585978  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:40.624588  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.625144  2683 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.625512  2753 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 14:52:40.593704  2685 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:40.595281  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:40.596246  2754 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
W20260709 14:52:40.596582  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.598266  2606 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:40.498935  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:40.627362  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:40.628131  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 129349us
I20260709 14:52:40.632777  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.628100  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:40.600466  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 494924us
W20260709 14:52:40.510169  5380 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.004s
W20260709 14:52:40.603430  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121029us
I20260709 14:52:40.606125  2681 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:40.613384  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.638345  2759 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:40.587970  5900 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:40.601476  6018 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.517s	user 0.015s	sys 0.000s
W20260709 14:52:40.617075  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 303376us
I20260709 14:52:40.618335  2680 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:52:40.617440  2612 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.646759  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:52:40.601526  6019 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.819s	user 0.015s	sys 0.000s
I20260709 14:52:40.624125  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.644721  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1078) took 1923 ms. Trace:
I20260709 14:52:40.640434  2757 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 14:52:40.652187  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.649437  2760 rpcz_store.cc:276] 0709 14:52:38.721522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:39.051551 (+330029us) service_pool.cc:224] Handling call
0709 14:52:39.051980 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:39.104188 (+ 52208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:39.104222 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:39.104774 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.493634 (+388860us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:39.493705 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.493835 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.560521 (+1066686us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b24001ccf88. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b597fb0 00007fa16e20e488 00007fa16e20ebe8 00007fa16e1e6398 00007fa16e1e176a 00007fa16e1e0bb6
0709 14:52:40.636833 (+ 76312us) raft_consensus.cc:1685] finished
0709 14:52:40.636860 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.644641 (+  7781us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51688,"raft.queue_time_us":53039,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":165,"spinlock_wait_cycles":42847104,"thread_start_us":388374,"threads_started":1,"wal-append.queue_time_us":791189}
I20260709 14:52:40.652686  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.658124  2758 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:40.659343  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.635671  5734 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:40.647089  6020 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.002s
W20260709 14:52:40.649819  5886 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
I20260709 14:52:40.644286  5900 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:40.661404  2752 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:40.640674  6012 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:40.635787  6037 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:40.670329  2753 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 14:52:40.706540  2753 pending_rounds.cc:85] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:40.661116  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.676529  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1198) took 1041 ms. Trace:
I20260709 14:52:40.706853  2753 pending_rounds.cc:107] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 14:52:40.702416  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.707161  2753 raft_consensus.cc:2889] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:40.707019  2609 rpcz_store.cc:276] 0709 14:52:39.635160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:39.635324 (+   164us) service_pool.cc:224] Handling call
0709 14:52:39.661225 (+ 25901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:39.723328 (+ 62103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:39.723388 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:39.724219 (+   831us) log.cc:844] Serialized 26 byte log entry
0709 14:52:39.921673 (+197454us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:39.921735 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:39.921838 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:40.558283 (+636445us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b180008d220. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:40.660315 (+102032us) raft_consensus.cc:1685] finished
0709 14:52:40.660338 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:40.676402 (+ 16064us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72245,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":51414,"spinlock_wait_cycles":8509696,"thread_start_us":101659,"threads_started":1,"wal-append.queue_time_us":407729}
I20260709 14:52:40.648542  6038 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:40.708932  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.709277  6038 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:40.707767  2752 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 7.
I20260709 14:52:40.709666  2753 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.710695  2756 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:40.701761  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55107.9us
I20260709 14:52:40.669552  5734 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.653820  6019 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.712407  5685 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:40.720088  5685 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.720568  5685 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:40.699303  6011 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.308s	user 0.012s	sys 0.006s
I20260709 14:52:40.705090  2612 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:40.723018  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.700693  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.712390  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:40.713621  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.731737  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:40.734090  6019 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:40.738868  5877 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:40.740489  6030 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:40.734225  6031 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:40.715492  5734 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.750442  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:40.733672  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:40.746716  2753 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:40.738265  6058 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:40.759402  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:40.762295  6055 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.001s	sys 0.005s
I20260709 14:52:40.767820  6018 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:40.768328  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:40.752049  6031 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.767881  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.700156  6020 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.715466  5380 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:52:40.813640  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45436.3us
I20260709 14:52:40.774158  5900 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.771893  6030 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:40.795483  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44984 (request call id 13) took 1221 ms. Trace:
I20260709 14:52:40.798192  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:40.804744  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:40.826413  2677 rpcz_store.cc:276] 0709 14:52:39.626058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:39.626229 (+   171us) service_pool.cc:224] Handling call
0709 14:52:40.795227 (+1168998us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:40.795349 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811110272}
W20260709 14:52:40.807889  6055 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.001s	sys 0.006s
I20260709 14:52:40.806401  2756 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:40.782049  5870 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.814680  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:40.814884  6031 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:40.819844  6085 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 14:52:40.820338  2757 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 14:52:40.821460  6092 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 14:52:40.821792  5685 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:40.742324  6054 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:52:40.826262  6030 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:40.832612  5870 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.833354  6091 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.809155  6018 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:40.836042  6020 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:40.841954  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:40.842974  2576 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.600s (SENT)
I20260709 14:52:40.851670  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:40.826931  5963 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:40.850812  2753 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
W20260709 14:52:40.854255  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:40.859359  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:40.860435  3543 pending_rounds.cc:85] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:40.860831  3543 pending_rounds.cc:107] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 14:52:40.861100  3543 raft_consensus.cc:2889] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:40.845050  5651 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:52:40.843348  6054 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:40.848688  6018 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:40.846092  6030 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.787599  6038 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:40.845297  5877 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:40.885598  2683 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:40.872284  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:40.827267  5380 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.875382  6020 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:40.859472  6093 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.150s	user 0.002s	sys 0.000s
I20260709 14:52:40.847050  5871 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:40.850291  5900 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:40.887472  3545 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:40.890370  3547 raft_consensus.cc:1217] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[7.5-12.7]   Dedup: 7.5->[10.6-12.7]
I20260709 14:52:40.875361  5718 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:40.894286  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.783s (SENT)
W20260709 14:52:40.902555  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 423405us
I20260709 14:52:40.908591  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:40.904969  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:40.909430  3547 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:52:40.849263  6091 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:40.914954  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:40.914034  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:40.895463  6082 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.912482  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:40.920807  2679 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 14:52:40.917297  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:40.917590  3547 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 14:52:40.890929  5496 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.919255  2752 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:40.970701  3547 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:40.971539  2683 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:40.904343  5902 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:40.917809  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:40.990231  2680 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:40.903381  5877 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:40.990665  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96409.1us
I20260709 14:52:40.911289  6096 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:40.913467  6105 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:40.999904  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104606us
I20260709 14:52:40.910998  6020 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:41.021138  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.023532  2679 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [4.3-8.4]
I20260709 14:52:41.007879  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:41.001221  5766 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.010s
I20260709 14:52:41.021136  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:41.026458  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.025849  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.062739  2681 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:41.028160  2677 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:40.993530  5902 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:41.090200  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 186908us
I20260709 14:52:41.090652  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.091360  2679 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
W20260709 14:52:41.092649  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 172223us
I20260709 14:52:40.981118  5496 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.061197  5752 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:41.093860  2680 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:41.004024  6077 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:52:41.098701  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77664.3us
I20260709 14:52:41.030560  6029 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:40.968107  6046 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:41.105585  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:40.982506  6082 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.096866  6019 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:41.107558  6019 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:41.108981  6078 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 14:52:41.112401  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:41.105168  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:41.116737  2683 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:41.118955  3545 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:41.111074  5685 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:40.967334  6054 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.061861  5871 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 14:52:41.028525  6041 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 14:52:41.138952  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.026320  6105 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:41.062330  5983 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 14:52:41.099829  5405 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:41.029356  6045 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.261s
I20260709 14:52:41.100809  6077 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:41.133359  2758 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:41.031504  5734 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:41.106065  6055 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:41.102980  5788 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 14:52:41.135640  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.157430  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.102631  6087 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:41.085080  5766 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.129266  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:41.127297  6094 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.336s	user 0.000s	sys 0.002s
I20260709 14:52:41.100082  6036 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 14:52:41.146289  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:41.122113  5380 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:41.158792  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:41.159972  2680 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:41.137831  5870 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:41.166375  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:41.169996  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.732s (SENT)
I20260709 14:52:41.200340  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:41.201637  6087 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:52:41.171852  6099 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 14:52:41.174731  6082 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:41.150532  6077 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:41.176942  5788 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:41.180712  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.181720  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:41.103575  5752 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.181109  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:41.156677  6102 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 14:52:41.113366  6029 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.189213  6020 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:41.190948  6015 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:41.270910  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89256.4us
I20260709 14:52:41.130894  5877 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.150355  5405 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.191675  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:41.192006  3540 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:41.158936  5871 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:41.192744  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:41.298188  3541 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:41.193378  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.194777  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:41.299492  2679 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:41.299821  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 105151us
I20260709 14:52:41.300923  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.195668  6053 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:41.302186  2679 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:41.302333  6053 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:41.196316  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:41.198137  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:41.198446  5937 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.005s
W20260709 14:52:41.199186  6107 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 14:52:41.154078  6055 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:41.305684  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:41.305760  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.160400  6054 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:52:41.306196  2678 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:52:41.230780  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.241667  6087 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:41.307571  6041 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:41.307067  6020 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:41.308703  6020 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:41.305737  5937 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 14:52:41.307986  5651 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:41.191608  6105 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:41.205199  6019 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:41.306285  6055 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:41.312060  5937 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:41.312376  6055 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:41.291721  2681 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:41.296445  5405 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:41.314129  5937 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:41.196043  5867 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:41.225075  3545 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:41.315271  5867 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:41.316025  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:41.316390  5867 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.317052  6079 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 14:52:41.241601  6109 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:52:41.245977  6097 raft_consensus.cc:993] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: : Instructing follower c1b0e692e4294ee3b8694bbada174ba1 to start an election
I20260709 14:52:41.301067  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:41.250753  6036 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:41.247269  5683 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:41.308030  6041 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:41.309247  6020 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:41.316869  6087 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:41.317260  5405 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:41.307080  2679 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:41.317695  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.884s (SENT)
I20260709 14:52:41.324847  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:41.273617  6082 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:41.336683  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:41.328963  6102 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 14:52:41.319254  5871 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:41.330711  5870 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:41.350942  2755 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:41.315913  5937 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:41.335530  6041 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:41.333163  6097 raft_consensus.cc:1081] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Signalling peer c1b0e692e4294ee3b8694bbada174ba1 to start an election
W20260709 14:52:41.363754  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.290701  6015 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.290560  6018 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.386071  6068 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:41.387297  6068 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:41.388796  6068 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.344332  6069 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:41.379577  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:41.380663  3542 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:41.349344  6036 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:41.359078  6091 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:41.325827  5651 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.347282  6082 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:41.361485  6053 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:41.390563  6106 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 14:52:41.914393  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 469128us
W20260709 14:52:41.734802  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.007612  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.663031  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1193) took 1006 ms. Trace:
W20260709 14:52:42.011399  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 815073us
I20260709 14:52:41.447954  2757 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:42.019842  2753 rpcz_store.cc:276] 0709 14:52:40.656999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.805997 (+148998us) service_pool.cc:224] Handling call
0709 14:52:40.806419 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:40.853176 (+ 46757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:41.662445 (+809269us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:41.662917 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101313280}
W20260709 14:52:41.601975  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.022920  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.020979  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:42.012192  2679 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:41.518582  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:41.754796  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:41.365271  5870 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:41.360917  6079 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:41.365729  5900 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:41.451808  2686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.017s	sys 0.044s
W20260709 14:52:42.019737  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.004171  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 611426us
W20260709 14:52:41.703922  6116 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 14:52:41.441365  2760 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a57a97528f31400dbc0900e76292b021"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:41.613340  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1014) took 1601 ms. Trace:
W20260709 14:52:41.957647  3541 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.659s	user 0.015s	sys 0.000s
I20260709 14:52:41.754012  6018 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:41.481127  5903 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.837s	user 0.008s	sys 0.008s
W20260709 14:52:41.935883  6054 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.007s
I20260709 14:52:41.580211  6080 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:41.473841  5405 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.049288  6011 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:41.754382  6031 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.909s	user 0.015s	sys 0.000s
I20260709 14:52:41.801165  6032 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.033578  2686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.020s	sys 0.084s
I20260709 14:52:42.012566  5867 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:41.574602  6033 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.026s	user 0.002s	sys 0.015s
I20260709 14:52:41.397070  6069 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:41.598589  5983 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:41.511554  5871 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.065359  6011 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.036722  5900 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:41.422763  5788 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:41.932588  6097 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:41.572528  6037 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.039050  2760 raft_consensus.cc:606] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:42.028676  2576 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.752s (SENT) [suppressed 40 similar messages]
I20260709 14:52:42.129245  2760 raft_consensus.cc:614] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:52:42.039584  3541 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
W20260709 14:52:42.129979  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100998us
I20260709 14:52:42.023731  6091 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.016990  6052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.003s	sys 0.008s
I20260709 14:52:42.033509  5870 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:42.030650  5914 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.058871  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1017) took 1797 ms. Trace:
W20260709 14:52:42.040510  6077 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.803s	user 0.013s	sys 0.003s
I20260709 14:52:41.835698  5793 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:41.474749  6046 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.080348  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 931) took 1225 ms. Trace:
I20260709 14:52:42.090454  6082 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.084115  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 588) took 2014 ms. Trace:
W20260709 14:52:42.087469  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.098789  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.096223  2679 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
W20260709 14:52:42.153988  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.153504  2754 rpcz_store.cc:276] 0709 14:52:40.069848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.070028 (+   180us) service_pool.cc:224] Handling call
0709 14:52:40.502344 (+432316us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521f8c 00007fa16b521ef6 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:40.502515 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:40.633341 (+130826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.083814 (+1450473us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.084035 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124145024}
I20260709 14:52:42.039170  2684 rpcz_store.cc:276] 0709 14:52:40.011962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.012119 (+   157us) service_pool.cc:224] Handling call
0709 14:52:41.599699 (+1587580us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:41.613220 (+ 13521us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86498176}
I20260709 14:52:42.051508  5903 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:42.050890  6079 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:42.058169  6074 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:42.050134  6080 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:42.077226  5871 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:42.152653  6077 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.168445  6077 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.146114  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 575) took 1561 ms. Trace:
I20260709 14:52:42.076283  5983 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.149326  3542 rpcz_store.cc:276] 0709 14:52:40.855383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.855552 (+   169us) service_pool.cc:224] Handling call
0709 14:52:40.859422 (+  3870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.015255 (+1155833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.080016 (+ 64761us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400224cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.080261 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197914368}
I20260709 14:52:42.082218  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:42.072433  6032 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.147181  2686 rpcz_store.cc:276] 0709 14:52:40.261472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.261866 (+   394us) service_pool.cc:224] Handling call
0709 14:52:40.311236 (+ 49370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:40.311610 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:40.340731 (+ 29121us) log.cc:844] Serialized 14 byte log entry
0709 14:52:40.866446 (+525715us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b1800373de0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:42.057822 (+1191376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:42.057912 (+    90us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:42.057985 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.058535 (+   550us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:42.058738 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171792,"prepare.queue_time_us":32492,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":57,"raft.queue_time_us":564727,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8439,"spinlock_wait_cycles":9701632,"thread_start_us":1191004,"threads_started":1}
I20260709 14:52:42.075644  6069 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:42.074020  6119 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:42.142628  6091 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.180907  2685 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
I20260709 14:52:42.085197  5734 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.088743  6062 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:42.138245  6052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.003s	sys 0.009s
I20260709 14:52:42.081096  5900 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.109987  6011 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.225833  2677 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:42.135732  6037 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.125743  6054 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.109853  6015 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.049769  5405 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.066594  6031 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.171056  2682 rpcz_store.cc:276] 0709 14:52:40.585135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.585303 (+   168us) service_pool.cc:224] Handling call
0709 14:52:40.585888 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:41.021417 (+435529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:41.021448 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:41.061740 (+ 40292us) log.cc:844] Serialized 26 byte log entry
0709 14:52:41.260859 (+199119us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:41.260909 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:41.260986 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:42.145853 (+884867us) raft_consensus.cc:1685] finished
0709 14:52:42.145877 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:42.146028 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":438246,"raft.queue_time_us":468977,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":26123,"thread_start_us":155569,"threads_started":1,"wal-append.queue_time_us":267775}
I20260709 14:52:42.091056  6016 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:42.173000  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1078) took 1273 ms (client timeout 1546 ms). Trace:
I20260709 14:52:42.174393  2679 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:52:42.076076  6038 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.157763  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.195024  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1105) took 1297 ms. Trace:
I20260709 14:52:42.247116  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1114) took 1146 ms. Trace:
I20260709 14:52:42.247874  2682 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 14:52:42.170310  6080 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.136853  6097 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.164080  6074 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.226496  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:42.088011  5916 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.151611  5914 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.231266  2684 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:42.103572  6036 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.247603  2679 rpcz_store.cc:276] 0709 14:52:41.100349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.163627 (+1063278us) service_pool.cc:224] Handling call
0709 14:52:42.165255 (+  1628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.246709 (+ 81454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.247040 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:42.248090  3536 rpcz_store.cc:269] 0709 14:52:40.889258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.901658 (+ 12400us) service_pool.cc:224] Handling call
0709 14:52:42.098582 (+1196924us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:42.162973 (+ 64391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10871680}
I20260709 14:52:42.249066  2685 rpcz_store.cc:276] 0709 14:52:40.897366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.897540 (+   174us) service_pool.cc:224] Handling call
0709 14:52:42.082065 (+1184525us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:52:42.194935 (+112870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2588928}
W20260709 14:52:42.197660  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.190335  6030 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.289s	user 0.018s	sys 0.000s
I20260709 14:52:42.225731  5903 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.231854  6020 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.888s	user 0.009s	sys 0.014s
I20260709 14:52:42.253350  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1117) took 1084 ms. Trace:
I20260709 14:52:42.232692  5734 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.257340  6127 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 14:52:42.257493  6132 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 14:52:42.258041  5983 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:42.258287  6128 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 14:52:42.175858  6126 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 14:52:42.229139  6046 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.269024  2685 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 14:52:42.204742  6062 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.273036  6037 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.250303  6016 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:42.305554  6016 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.306407  6062 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.253129  6038 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:42.252135  6055 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.939s	user 0.017s	sys 0.000s
I20260709 14:52:42.236223  6054 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.260613  6015 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.277885  6041 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.008s
I20260709 14:52:42.255860  6011 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.319535  6011 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.320464  6031 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.321587  6055 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.322176  6011 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.322278  6055 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.324645  6054 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.325582  6054 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.326095  6054 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.327483  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1115) took 1211 ms. Trace:
I20260709 14:52:42.327988  2684 rpcz_store.cc:276] 0709 14:52:41.116500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.155334 (+1038834us) service_pool.cc:224] Handling call
0709 14:52:42.173347 (+ 18013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.262373 (+ 89026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.322767 (+ 60394us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.327390 (+  4623us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132501888}
I20260709 14:52:42.329072  5902 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.330955  6041 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.324728  6055 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.332868  6055 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.333600  6041 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.339396  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.281001  6030 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.284448  6097 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.287798  2682 rpcz_store.cc:276] 0709 14:52:41.164884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.247173 (+1082289us) service_pool.cc:224] Handling call
0709 14:52:42.247594 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.248674 (+  1080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.248938 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:42.176126  6079 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:42.261451  5683 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.925s	user 0.016s	sys 0.000s
I20260709 14:52:42.259289  6033 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.300172  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1024) took 1102 ms. Trace:
I20260709 14:52:42.376176  2685 rpcz_store.cc:276] 0709 14:52:41.197636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.263493 (+1065857us) service_pool.cc:224] Handling call
0709 14:52:42.268601 (+  5108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.299879 (+ 31278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.300116 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:42.377264  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.294713  6133 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:42.333366  6055 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.278980  5914 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.291957  5903 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:42.396128  3412 connection.cc:256] RPC call timeout handler was delayed by -1.1664024599999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 14:52:42.375537  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 347871us
I20260709 14:52:42.293707  6080 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:42.293916  5734 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.388571  2755 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 14:52:42.330677  5902 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:42.295914  6020 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.355283  6016 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:42.399413  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1197) took 1739 ms. Trace:
I20260709 14:52:42.404991  2756 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:52:42.405602  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.435528  3545 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:42.434715  2758 rpcz_store.cc:276] 0709 14:52:40.659778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.828496 (+168718us) service_pool.cc:224] Handling call
0709 14:52:40.829047 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:41.154908 (+325861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.399042 (+1244134us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400423bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.399346 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699139456}
I20260709 14:52:42.406033  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.317940  6062 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 14:52:42.406196  6133 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:42.421461  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.421461  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:42.392529  5877 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 14:52:42.368427  6015 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.399964  6011 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.380712  3536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3"
dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:37148
I20260709 14:52:42.404763  6110 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.303880  6046 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.422109  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:42.398398  5885 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.426429  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.436576  3541 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:42.441681  3537 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:42.441552  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:42.437758  6129 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 14:52:42.402541  5683 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:42.481293  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.473827  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.447284  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:42.390566  6033 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:42.403571  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:42.588371  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1199) took 1918 ms (client timeout 2094 ms). Trace:
I20260709 14:52:42.444310  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:42.608357  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 161224us
W20260709 14:52:42.546274  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94011.9us
I20260709 14:52:42.615605  2758 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:42.554067  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.616412  3543 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:42.450549  6134 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 14:52:42.569721  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.016s	sys 0.051s
W20260709 14:52:42.626525  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.016s	sys 0.053s
I20260709 14:52:42.484998  2751 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:42.530105  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.453661  3536 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:52:42.578665  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.630753  3536 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:42.631470  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:42.557197  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:42.616925  2724 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:42.486199  2685 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 14:52:42.613168  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 211079us
W20260709 14:52:42.614041  2752 rpcz_store.cc:269] 0709 14:52:40.670013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.710508 (+ 40495us) service_pool.cc:224] Handling call
0709 14:52:42.588039 (+1877531us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54000695d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:42.588247 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440254720}
I20260709 14:52:42.607364  2760 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:42.636937  2724 leader_election.cc:403] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:52:42.637650  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.637989  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:42.638221  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:42.638495  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:42.586807  5963 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.712s	user 0.012s	sys 0.002s
I20260709 14:52:42.639154  5651 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:42.586674  6053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.005s	sys 0.006s
I20260709 14:52:42.638927  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:42.610904  5959 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.490s	user 0.010s	sys 0.013s
W20260709 14:52:42.745390  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106982us
W20260709 14:52:42.745983  2723 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.713s (SENT)
I20260709 14:52:42.637714  2760 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:56734
W20260709 14:52:42.660457  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.557469  6091 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.723549  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.450098  5914 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.235s
W20260709 14:52:42.725281  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.673465  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.687080  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1205) took 1503 ms. Trace:
I20260709 14:52:42.765404  2755 rpcz_store.cc:276] 0709 14:52:41.184035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:41.392053 (+208018us) service_pool.cc:224] Handling call
0709 14:52:41.392433 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.403422 (+1010989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.686630 (+283208us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.686990 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289898240}
W20260709 14:52:42.766968  2681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.003s	sys 0.016s
W20260709 14:52:42.767391  2681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.003s	sys 0.016s
W20260709 14:52:42.767467  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 123998us
I20260709 14:52:42.448698  5496 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.638761  2685 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:42.563614  6045 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.580359  5405 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.685686  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:42.744302  5959 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.635s	user 0.010s	sys 0.013s
I20260709 14:52:42.551002  6097 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.558818  6062 consensus_queue.cc:1243] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 14:52:42.543244  5877 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:42.749109  2760 raft_consensus.cc:606] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
W20260709 14:52:42.440881  6078 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.004s	sys 0.003s
I20260709 14:52:42.822981  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1021) took 1778 ms. Trace:
W20260709 14:52:42.750346  6061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.017s	sys 0.031s
I20260709 14:52:42.750320  6140 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.750939  6149 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 10
I20260709 14:52:42.752696  6145 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:42.750859  6137 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.626987  6080 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:42.762656  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.898120  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.551584  6032 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.738796  3541 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:42.851620  6061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.021s	sys 0.031s
W20260709 14:52:42.725029  5963 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.850s	user 0.013s	sys 0.005s
W20260709 14:52:42.730043  6053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.007s
I20260709 14:52:42.777710  2751 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:42.548121  5885 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.729833  5651 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:42.801635  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.803885  3543 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:42.945765  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.491621  6011 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.762549  6041 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:42.542920  6110 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.749786  6139 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.792662  5880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.010s	sys 0.004s
I20260709 14:52:42.832734  2681 rpcz_store.cc:276] 0709 14:52:41.024322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.042774 (+1018452us) service_pool.cc:224] Handling call
0709 14:52:42.072022 (+ 29248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.072367 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:42.090782 (+ 18415us) log.cc:844] Serialized 14 byte log entry
0709 14:52:42.801828 (+711046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:42.801900 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:42.802012 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.802104 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:42.802283 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48854,"prepare.queue_time_us":16038,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":34682,"thread_start_us":676632,"threads_started":1}
I20260709 14:52:42.761658  6054 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:42.829216  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.958099  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.958285  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:42.960011  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 581) took 1864 ms. Trace:
W20260709 14:52:42.834524  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:42.798185  6141 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.190s	user 0.001s	sys 0.003s
I20260709 14:52:42.623855  5683 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.796674  6136 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:42.852005  4786 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:42.786262  6037 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:42.824640  6030 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.625402  6133 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.951995  5651 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:42.860800  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:42.952589  6080 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:42.865161  5870 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 14:52:42.864912  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.865911  5903 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:43.044425  5683 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.868462  6033 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:43.034500  6148 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 14:52:42.868505  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 587) took 1538 ms. Trace:
I20260709 14:52:42.869462  6140 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:42.873414  6046 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:42.873946  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.015s	sys 0.017s
I20260709 14:52:42.874421  6015 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:42.887509  2576 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.525s (SENT)
W20260709 14:52:42.840473  6078 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.009s	sys 0.016s
W20260709 14:52:42.889757  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 172822us
I20260709 14:52:42.805086  5959 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:42.808549  6111 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:42.802928  5914 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:42.890808  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.786069  5871 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.083s
W20260709 14:52:42.913805  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 148727us
W20260709 14:52:42.914258  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:42.845371  6079 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.854660  5877 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:42.815578  6068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.001s	sys 0.020s
I20260709 14:52:42.843442  6055 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:42.802629  6045 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:42.954375  2679 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:42.890869  5734 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:42.877673  6020 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:42.960448  2677 rpcz_store.cc:276] 0709 14:52:41.095482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.083334 (+987852us) service_pool.cc:224] Handling call
0709 14:52:42.083798 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.226644 (+142846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.959745 (+733101us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:42.959944 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326077184}
W20260709 14:52:42.971971  5880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.010s	sys 0.005s
I20260709 14:52:42.983407  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.984318  6054 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:42.985342  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:42.992594  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.944028  6032 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.000146  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:42.926729  5963 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:43.002043  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100759us
I20260709 14:52:42.863461  5766 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:42.951839  6041 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:42.949575  6145 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:42.828004  6097 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.000828  5405 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.033277  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1027) took 1542 ms. Trace:
W20260709 14:52:43.035836  6144 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 14:52:43.037880  6016 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.002s
W20260709 14:52:43.039952  6150 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 14:52:43.040661  6069 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.000s
I20260709 14:52:43.041040  6080 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:43.041967  5870 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.043057  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:42.912015  6137 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:42.988889  6139 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:43.230671  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:43.327229  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.212167  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.328910  2681 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:43.329614  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.024s	sys 0.023s
W20260709 14:52:43.118913  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1019) took 2095 ms (client timeout 1898 ms). Trace:
I20260709 14:52:43.135627  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:43.330932  2683 rpcz_store.cc:269] 0709 14:52:41.023293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:41.026165 (+  2872us) service_pool.cc:224] Handling call
0709 14:52:43.118470 (+2092305us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:43.118756 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122748288}
I20260709 14:52:43.331743  2611 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 14:52:43.242057  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.328871  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:43.066219  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:43.080430  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 122276us
W20260709 14:52:43.333453  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 445966us
I20260709 14:52:43.328425  2682 rpcz_store.cc:276] 0709 14:52:41.491394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.387034 (+895640us) service_pool.cc:224] Handling call
0709 14:52:42.387710 (+   676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.397457 (+  9747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:42.397488 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:42.428729 (+ 31241us) log.cc:844] Serialized 26 byte log entry
0709 14:52:42.808860 (+380131us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:42.808915 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.808998 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:43.032994 (+223996us) raft_consensus.cc:1685] finished
0709 14:52:43.033017 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:43.033197 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9175,"raft.queue_time_us":13736,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":25413,"thread_start_us":249229,"threads_started":1,"wal-append.queue_time_us":578304}
W20260709 14:52:43.334748  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.327230  3538 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:43.280763  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1077) took 2392 ms. Trace:
W20260709 14:52:43.227316  3536 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.001s
W20260709 14:52:43.099258  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:43.335615  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:43.341112  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:43.191979  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.336479  3546 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:43.335481  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:43.339712  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:43.328070  2680 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:43.342630  2612 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:43.338661  3541 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 14:52:43.341784  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.064456  2684 rpcz_store.cc:276] 0709 14:52:41.330277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.329371 (+999094us) service_pool.cc:224] Handling call
0709 14:52:42.329859 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.330881 (+  1022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:42.330919 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:42.331540 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 14:52:42.555678 (+224138us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:42.555750 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.555881 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:42.868172 (+312291us) raft_consensus.cc:1685] finished
0709 14:52:42.868208 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:42.868410 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"thread_start_us":223569,"threads_started":1,"wal-append.queue_time_us":533570}
I20260709 14:52:43.326767  3537 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 14:52:43.334863  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.088s (SENT)
I20260709 14:52:43.343350  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:43.230556  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1200) took 2560 ms. Trace:
I20260709 14:52:43.345361  2756 rpcz_store.cc:276] 0709 14:52:40.670390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.861186 (+190796us) service_pool.cc:224] Handling call
0709 14:52:40.861658 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.434878 (+1573220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:43.214556 (+779678us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:43.230420 (+ 15864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3548776320}
I20260709 14:52:43.344743  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:43.285118  5867 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.211s	user 0.015s	sys 0.000s
I20260709 14:52:43.338099  3547 rpcz_store.cc:276] 0709 14:52:40.888856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.889356 (+   500us) service_pool.cc:224] Handling call
0709 14:52:40.889965 (+   609us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:40.972263 (+ 82298us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:40.972310 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:40.973178 (+   868us) log.cc:844] Serialized 52 byte log entry
0709 14:52:42.303391 (+1330213us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18003b7700. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:42.610609 (+307218us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:42.610664 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.610757 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:43.276213 (+665456us) raft_consensus.cc:1685] finished
0709 14:52:43.276250 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:43.280665 (+  4415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1120,"raft.queue_time_us":3417,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":4075,"spinlock_wait_cycles":3129216,"thread_start_us":306918,"threads_started":1,"wal-append.queue_time_us":2153972}
I20260709 14:52:43.352653  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:43.338657  3536 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:43.353444  2603 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:43.355347  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:43.356190  2608 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 5.
W20260709 14:52:43.215807  6157 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 14:52:43.256937  6158 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 14:52:43.381652  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:43.202205  5885 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:43.374131  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.123s (SENT)
I20260709 14:52:43.380728  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.206094  6096 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:43.380766  2754 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:52:43.406750  2756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:56734
I20260709 14:52:43.381156  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.047215  6091 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:43.407423  2609 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 14:52:43.382561  2606 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:43.405716  2610 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:43.381549  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.405689  2612 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:43.381027  3540 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:43.207830  6011 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:43.415856  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:43.417529  3536 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:43.329329  6068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.001s	sys 0.027s
I20260709 14:52:43.287374  6029 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:43.325819  5877 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:43.422382  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:43.356431  6079 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:43.459385  2751 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:43.456928  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.472044  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.153401  6033 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.260s
I20260709 14:52:43.119431  6018 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.348464  6149 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:43.351388  6015 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.336861  6041 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:43.458303  4786 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.337684  6055 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.480268  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.589268  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1031) took 1129 ms. Trace:
I20260709 14:52:43.589025  4786 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.589684  2677 rpcz_store.cc:276] 0709 14:52:42.459605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.357564 (+897959us) service_pool.cc:224] Handling call
0709 14:52:43.589180 (+231616us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:43.590281  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.591217  3538 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:43.591147  6018 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.591660  6137 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:43.592450  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:43.593055  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.593297  6033 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:43.593669  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.594403  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.595023  6055 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:43.020049  6037 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.593747  6033 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:43.596231  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:43.596884  2756 raft_consensus.cc:606] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:43.337651  6032 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.599994  2680 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:43.605250  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.605589  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.338137  5914 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.600080  3537 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 6.
I20260709 14:52:43.584134  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.590701  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.595561  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:37148 (request call id 935) took 1256 ms. Trace:
I20260709 14:52:43.611613  3536 rpcz_store.cc:276] 0709 14:52:42.374979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.380523 (+  5544us) service_pool.cc:224] Handling call
0709 14:52:43.595465 (+1214942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:43.347535  6069 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:43.588027  6160 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:52:43.620651  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.337886  5405 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.606003  2756 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:43.536468  5683 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.406550  6077 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:43.572854  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1109) took 2660 ms (client timeout 2512 ms). Trace:
W20260709 14:52:43.606490  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 277563us
I20260709 14:52:43.535610  5885 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.606331  2680 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:43.601778  6096 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:43.015955  6136 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.611117  2758 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:43.484469  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.613206  2755 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:43.663549  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:43.612460  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 147910us
I20260709 14:52:43.665940  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:43.666899  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.667347  3536 raft_consensus.cc:2368] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:52:43.281512  5651 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:43.615859  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:43.602514  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:43.674328  3538 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:43.381026  6161 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:43.621628  2677 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:43.621743  4786 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:43.623740  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 249662us
I20260709 14:52:43.624030  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:43.611559  6148 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:52:43.627709  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:43.597537  3546 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:43.618870  5959 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.612614  5871 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 14:52:43.586388  6140 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:43.357712  5734 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:43.481426  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.577495  5766 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:43.621057  5963 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.632344  2607 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:43.595052  6079 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:43.608554  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:43.657857  2681 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
W20260709 14:52:43.658501  2678 rpcz_store.cc:269] 0709 14:52:40.912845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:40.984065 (+ 71220us) service_pool.cc:224] Handling call
0709 14:52:43.572474 (+2588409us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400423954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:43.572734 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644649344}
I20260709 14:52:43.678349  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.356631  6168 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.221s
I20260709 14:52:43.675624  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:43.617311  6091 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:43.457701  5880 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.628475  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:43.573357  5788 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.607456  6015 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.624686  5877 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.617951  6145 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:43.966661  2609 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:52:43.867723  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.474061  6046 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.961078  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 590) took 1314 ms. Trace:
I20260709 14:52:43.833058  3538 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:43.762624  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:43.724148  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:43.701289  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:43.874473  2608 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 14:52:43.680267  2723 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.841s (SENT)
W20260709 14:52:43.965543  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.191s (SENT)
I20260709 14:52:43.965684  3547 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 14:52:43.768452  2612 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:43.907583  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1202) took 3114 ms. Trace:
I20260709 14:52:43.764382  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:43.818548  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 939) took 1442 ms. Trace:
I20260709 14:52:43.960515  3537 raft_consensus.cc:2376] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10. Re-sending same reply.
I20260709 14:52:43.680052  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:43.483264  5903 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.962144  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 593) took 1277 ms. Trace:
W20260709 14:52:43.656965  5902 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.234s	user 0.014s	sys 0.002s
I20260709 14:52:43.761960   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:52:43.973665  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.997357  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:43.995925  2757 rpcz_store.cc:276] 0709 14:52:40.793302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:41.239725 (+446423us) service_pool.cc:224] Handling call
0709 14:52:41.240191 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:42.020865 (+780674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:43.907127 (+1886262us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:43.907471 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468944640}
I20260709 14:52:43.998952  2681 rpcz_store.cc:276] 0709 14:52:42.684917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.984979 (+300062us) service_pool.cc:224] Handling call
0709 14:52:43.962066 (+977087us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:43.996127  3540 rpcz_store.cc:276] 0709 14:52:42.376633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.378218 (+  1585us) service_pool.cc:224] Handling call
0709 14:52:42.378755 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:43.597871 (+1219116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:43.818221 (+220350us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:43.818465 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304355584}
I20260709 14:52:44.000955  2677 rpcz_store.cc:276] 0709 14:52:42.646392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.620272 (+973880us) service_pool.cc:224] Handling call
0709 14:52:43.961007 (+340735us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.002982  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:44.004009  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1083) took 1044 ms. Trace:
I20260709 14:52:44.004410  3538 rpcz_store.cc:276] 0709 14:52:42.959543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.594725 (+635182us) service_pool.cc:224] Handling call
0709 14:52:44.003937 (+409212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.005590  2606 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:44.006412  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 801) took 1007 ms. Trace:
I20260709 14:52:44.006718  3537 rpcz_store.cc:276] 0709 14:52:42.999520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.610698 (+611178us) service_pool.cc:224] Handling call
0709 14:52:44.006353 (+395655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.007143  2758 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:44.010975  2680 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:44.011261  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1125) took 1109 ms. Trace:
W20260709 14:52:44.012450  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.012247  2677 rpcz_store.cc:276] 0709 14:52:42.901790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.002603 (+1100813us) service_pool.cc:224] Handling call
0709 14:52:44.011188 (+  8585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.007584  2758 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:44.013567  3536 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:43.986446  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:44.015771  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:44.013494  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.016639  3540 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:44.014698  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:44.017405  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:44.018356  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1116) took 2875 ms. Trace:
I20260709 14:52:44.018299  2681 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:44.020789  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1030) took 1561 ms. Trace:
I20260709 14:52:44.021062  2681 rpcz_store.cc:276] 0709 14:52:42.459282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.017093 (+1557811us) service_pool.cc:224] Handling call
0709 14:52:44.020731 (+  3638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.022450  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.023316  3538 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 6: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:44.023921  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 791) took 1589 ms. Trace:
I20260709 14:52:44.024474  3538 rpcz_store.cc:276] 0709 14:52:42.434526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.022228 (+1587702us) service_pool.cc:224] Handling call
0709 14:52:44.023841 (+  1613us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.024942  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:44.025466  6181 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:44.025725  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.018800  2686 rpcz_store.cc:276] 0709 14:52:41.143079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.179876 (+1036797us) service_pool.cc:224] Handling call
0709 14:52:42.205913 (+ 26037us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:42.224596 (+ 18683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:42.224619 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:42.231262 (+  6643us) log.cc:844] Serialized 26 byte log entry
0709 14:52:42.586288 (+355026us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:42.614184 (+ 27896us) log.cc:844] Serialized 14 byte log entry
0709 14:52:42.614491 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:42.614572 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:44.018128 (+1403556us) raft_consensus.cc:1685] finished
0709 14:52:44.018153 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:44.018293 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2652,"prepare.queue_time_us":29894,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":65,"raft.queue_time_us":209,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":9506,"spinlock_wait_cycles":21450496,"thread_start_us":325157,"threads_started":1,"wal-append.queue_time_us":565904}
W20260709 14:52:44.028834  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63332.1us
I20260709 14:52:44.028956  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 784) took 1596 ms. Trace:
W20260709 14:52:44.029563  2576 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.206s (SENT)
I20260709 14:52:44.026648  6182 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:43.846709  6020 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.855111  6069 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.018857  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.029704  3538 rpcz_store.cc:276] 0709 14:52:42.432184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.025389 (+1593205us) service_pool.cc:224] Handling call
0709 14:52:44.028866 (+  3477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:43.961597  6068 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:44.031567  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:43.845301  6165 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:44.032174  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 370221us
W20260709 14:52:44.049912  2723 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.914s (SENT)
I20260709 14:52:44.050426  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:43.961727  6016 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.717097  6029 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.969515  6079 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:43.674003  5683 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:44.019454  2758 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:44.032790  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1124) took 1131 ms. Trace:
I20260709 14:52:44.018234  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1121) took 1118 ms. Trace:
I20260709 14:52:44.075029  2677 rpcz_store.cc:276] 0709 14:52:42.900282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.014163 (+1113881us) service_pool.cc:224] Handling call
0709 14:52:44.018142 (+  3979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.075591  2686 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:52:43.717726  6038 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:43.811144  6080 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:43.812479  6037 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:43.924422  6137 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:43.810151  6032 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:44.034185  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.945s (SENT)
I20260709 14:52:44.090794  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:44.026373  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 398023us
W20260709 14:52:44.093679  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.033161  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 800) took 1080 ms. Trace:
I20260709 14:52:44.010299  5734 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.971376  5766 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:44.035862  6045 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:44.035948  5405 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.036309  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:44.017230  6046 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:44.034739  6182 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:44.037448  6185 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:43.930398  5870 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:43.970084  5752 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.045011  2607 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:44.046571  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.105962  3537 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:44.105806  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.106549  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 787) took 1673 ms. Trace:
I20260709 14:52:44.106796  2609 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:44.106798  3537 rpcz_store.cc:276] 0709 14:52:42.433206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.022179 (+1588973us) service_pool.cc:224] Handling call
0709 14:52:44.106478 (+ 84299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.047351  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.047684  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1094) took 1060 ms. Trace:
I20260709 14:52:44.050800  3536 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: [6.5-10.7]
I20260709 14:52:43.664872  5885 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:44.032466  3540 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:44.110011  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.110661  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 782) took 1703 ms. Trace:
I20260709 14:52:44.108031  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 788) took 1674 ms. Trace:
I20260709 14:52:44.111088  3540 rpcz_store.cc:276] 0709 14:52:42.407354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.031888 (+1624534us) service_pool.cc:224] Handling call
0709 14:52:44.032253 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.110357 (+ 78104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.110599 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.028082  6052 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:44.064131  2681 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 14:52:44.069520  6091 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:43.932796  6096 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:43.673444  6077 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:44.071707  6136 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.115298  6052 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:44.118424  5885 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.072356  6015 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.015340  5903 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:44.100049  5734 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:44.120961  5903 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.034965  6147 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:44.121228  5734 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.074275  2680 rpcz_store.cc:276] 0709 14:52:42.901422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.679190 (+777768us) service_pool.cc:224] Handling call
0709 14:52:44.032724 (+353534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.079325  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 592) took 1432 ms. Trace:
I20260709 14:52:44.046188  5877 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:44.038079  5902 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:43.970981  5788 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:44.084693  2723 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.112s (SENT)
I20260709 14:52:44.133661  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:44.079120  6029 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.085225  5651 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.085309  6054 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:44.060374  6165 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.027211  6181 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:44.080204  6168 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:44.089452  2606 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:44.034756  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:44.094108  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.094511  2751 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:44.094203  3547 rpcz_store.cc:276] 0709 14:52:42.953206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.608220 (+655014us) service_pool.cc:224] Handling call
0709 14:52:44.033056 (+424836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.073169  6061 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 14:52:44.095211  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.077039  6183 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:44.099509  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.046944  6020 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:44.100826  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.103425  2605 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:44.104763  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.105017  2608 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:44.108181  3546 rpcz_store.cc:276] 0709 14:52:42.987516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.031214 (+1043698us) service_pool.cc:224] Handling call
0709 14:52:44.047588 (+ 16374us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:44.108569  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74430.6us
I20260709 14:52:44.108701  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 793) took 1672 ms. Trace:
I20260709 14:52:44.052372  6068 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:44.111698  3542 rpcz_store.cc:276] 0709 14:52:42.433551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.017109 (+1583558us) service_pool.cc:224] Handling call
0709 14:52:44.107979 (+ 90870us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.112244  2681 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 4
I20260709 14:52:44.075119  6016 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:43.971220  5880 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.074463  2758 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:44.114511  6096 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:52:44.124814  3412 connection.cc:256] RPC call timeout handler was delayed by -1.7014378169999418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 14:52:44.122221  2759 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:44.126892  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.082124  6041 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:44.124848  2686 rpcz_store.cc:276] 0709 14:52:42.647016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.034316 (+1387300us) service_pool.cc:224] Handling call
0709 14:52:44.073881 (+ 39565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.076015 (+  2134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.079224 (+  3209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304}
I20260709 14:52:44.134994  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:44.094879  6137 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:44.136077  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:44.321801  6052 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.322661  3542 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:44.323637  6184 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 14:52:44.324510  2612 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:44.325268  3536 rpcz_store.cc:276] 0709 14:52:42.436062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.046793 (+1610731us) service_pool.cc:224] Handling call
0709 14:52:44.050499 (+  3706us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:44.108368 (+ 57869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.108643 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:44.326021  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.327858  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.327953  3536 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:44.107486  6161 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:44.335319  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.348583  6181 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:44.386024  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.341550  6016 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.345057  6068 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:44.343979  6041 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:44.337157  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:44.337953  3538 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 3. Current term is 4. Ops: []
I20260709 14:52:44.339382  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1091) took 1352 ms. Trace:
I20260709 14:52:44.336017  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:44.397363  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60379.1us
I20260709 14:52:44.397864  2611 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:44.397598  3547 rpcz_store.cc:276] 0709 14:52:42.986678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.338256 (+1351578us) service_pool.cc:224] Handling call
0709 14:52:44.338611 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.338963 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:44.338994 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:44.339046 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:44.339087 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.339158 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:44.339306 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:44.340152  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.340786  6183 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.141921  5903 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:44.113687  6185 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.162127  5902 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.108021  6182 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:44.324359  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1087) took 1363 ms. Trace:
I20260709 14:52:44.352006  3543 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 7. Current term is 8.
I20260709 14:52:44.338014  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:44.353756  2577 connection.cc:256] RPC call timeout handler was delayed by -1.80672727700005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 14:52:44.350075  2603 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:44.350212  2681 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 14:52:44.350507  2678 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:44.349606  6168 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:44.346809  5880 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.348205  5788 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:44.349258  6137 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.396171  2612 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:44.343710  6031 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:44.220129  5766 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.398957  2606 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:44.404183  2758 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:44.356549  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:44.362463  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:44.361572  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:44.358814  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.401080  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:44.322103  2757 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:44.889500  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.003s	sys 0.021s
W20260709 14:52:44.282153  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 202169us
W20260709 14:52:45.199887  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.004s	sys 0.022s
I20260709 14:52:45.200832  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1084) took 2241 ms. Trace:
I20260709 14:52:45.190500  2607 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:45.190399  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 588) took 1068 ms. Trace:
I20260709 14:52:45.204457  2612 rpcz_store.cc:276] 0709 14:52:44.122440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.325707 (+203267us) service_pool.cc:224] Handling call
0709 14:52:45.190333 (+864626us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:44.782449  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.207726  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:44.780118  3545 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:44.409955  3547 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 14:52:45.210387  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.210282  3412 connection.cc:256] RPC call timeout handler was delayed by -1.6600378169999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 14:52:44.463989  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.211306  3412 connection.cc:256] RPC call timeout handler was delayed by -1.4737378170000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 14:52:45.212162  3412 connection.cc:256] RPC call timeout handler was delayed by -1.3756378169999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 14:52:45.212472  3412 connection.cc:256] RPC call timeout handler was delayed by -1.1668378169999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2263s for a response
W20260709 14:52:45.212539  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.213129  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:45.193122  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 612) took 1612 ms (client timeout 1796 ms). Trace:
W20260709 14:52:45.211125  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.213807  2758 rpcz_store.cc:269] 0709 14:52:43.580768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.663030 (+ 82262us) service_pool.cc:224] Handling call
0709 14:52:45.193045 (+1530015us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:45.214398  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 857896us
I20260709 14:52:45.193349  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 608) took 1613 ms. Trace:
I20260709 14:52:44.458688  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.201231  3537 rpcz_store.cc:276] 0709 14:52:42.959799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.114009 (+1154210us) service_pool.cc:224] Handling call
0709 14:52:44.114429 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.114666 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:44.305865 (+191199us) log.cc:844] Serialized 14 byte log entry
0709 14:52:45.200411 (+894546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:45.200476 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:45.200532 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.200608 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:45.200768 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":639368,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"thread_start_us":1081123,"threads_started":2}
I20260709 14:52:44.221293  2677 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:45.190788  2681 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:45.215078  2757 rpcz_store.cc:276] 0709 14:52:43.579601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.111502 (+531901us) service_pool.cc:224] Handling call
0709 14:52:45.193270 (+1081768us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:45.191778  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.192387  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 791333us
I20260709 14:52:44.778201  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 786) took 2345 ms. Trace:
I20260709 14:52:45.075047  3546 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 14:52:45.209865  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1096) took 2115 ms. Trace:
I20260709 14:52:45.219398  3545 rpcz_store.cc:276] 0709 14:52:43.094956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.779052 (+1684096us) service_pool.cc:224] Handling call
0709 14:52:44.779596 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.209425 (+429829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.209797 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.219395  3546 pending_rounds.cc:85] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:52:45.221010  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.209851  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1093) took 2222 ms. Trace:
I20260709 14:52:45.189899  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 586) took 1085 ms. Trace:
I20260709 14:52:45.191135  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1128) took 1862 ms. Trace:
W20260709 14:52:44.227798  6188 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 14:52:45.224393  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:45.191584  3542 rpcz_store.cc:276] 0709 14:52:42.960591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.321477 (+1360886us) service_pool.cc:224] Handling call
0709 14:52:44.322240 (+   763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.323687 (+  1447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.324130 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:44.932338  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 785) took 2499 ms (client timeout 2582 ms). Trace:
W20260709 14:52:45.233289  3541 rpcz_store.cc:269] 0709 14:52:42.432549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.950710 (+518161us) service_pool.cc:224] Handling call
0709 14:52:44.931953 (+1981243us) spinlock_profiling.cc:243] Waited 9.69 ms on lock 0x7b54002ba2d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:44.932241 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22284544}
I20260709 14:52:44.970366  6046 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:45.235134  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:45.235977  3541 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:44.957559  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 586) took 3765 ms. Trace:
I20260709 14:52:44.924886  6033 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:45.237282  6097 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:45.237529  2679 rpcz_store.cc:276] 0709 14:52:41.192504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.262729 (+1070225us) service_pool.cc:224] Handling call
0709 14:52:42.268161 (+  5432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:43.330409 (+1062248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.957151 (+1626742us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:44.957467 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577586048}
I20260709 14:52:45.238180  6033 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:45.223026  6054 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:45.239760  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 605) took 1661 ms (client timeout 2049 ms). Trace:
W20260709 14:52:44.858760  2576 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.013s	sys 0.010s
I20260709 14:52:44.529372  6055 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:45.240338  2576 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.016s	sys 0.010s
W20260709 14:52:45.240267  2759 rpcz_store.cc:269] 0709 14:52:43.578622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.024641 (+446019us) service_pool.cc:224] Handling call
0709 14:52:45.239556 (+1214915us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.239695 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215007488}
W20260709 14:52:45.240779  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 911358us
I20260709 14:52:44.626425  2604 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
W20260709 14:52:45.197721  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 836184us
I20260709 14:52:45.243137  6074 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:45.229341  6137 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:44.544165  6126 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.700s
I20260709 14:52:44.515547  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.244117  6137 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.244494  2608 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:45.244570  6137 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:44.965481  2611 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.005s
W20260709 14:52:45.189203  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.245348  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:45.223277  2678 rpcz_store.cc:276] 0709 14:52:43.328594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.012878 (+684284us) service_pool.cc:224] Handling call
0709 14:52:45.191064 (+1178186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.241328  6055 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 14:52:44.449888  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1090) took 1463 ms. Trace:
I20260709 14:52:45.154333  6077 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:45.246919  6096 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.918s	user 0.012s	sys 0.004s
I20260709 14:52:45.221534  3546 pending_rounds.cc:107] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 14:52:44.531373  6189 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 14:52:45.195775  6031 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:45.247032  3536 rpcz_store.cc:276] 0709 14:52:42.986393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.326793 (+1340400us) service_pool.cc:224] Handling call
0709 14:52:44.327502 (+   709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.400070 (+ 72568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.449779 (+ 49709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.248449  6031 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:45.248910  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.249650  3536 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:45.249948  6031 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:45.247532  6097 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 14:52:45.250741  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.252602  3536 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:45.253611  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.245072  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 589) took 1082 ms. Trace:
I20260709 14:52:45.254552  2608 rpcz_store.cc:276] 0709 14:52:44.163513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.402352 (+238839us) service_pool.cc:224] Handling call
0709 14:52:45.245003 (+842651us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.210023  6118 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:45.222141  3547 rpcz_store.cc:276] 0709 14:52:42.987223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.399811 (+1412588us) service_pool.cc:224] Handling call
0709 14:52:44.400412 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.209517 (+809105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.209781 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:45.241360  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.214s (SENT)
I20260709 14:52:45.257611  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.257714  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:44.535669  6120 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 2.398s	user 0.009s	sys 0.000s
I20260709 14:52:44.543974  6052 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.695s
I20260709 14:52:45.239539  6054 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:45.277841  3543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 790) took 2843 ms (client timeout 2945 ms). Trace:
I20260709 14:52:44.929574  6139 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:44.625029  6091 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.022s	sys 0.043s
I20260709 14:52:44.972695  6110 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:45.222762  2603 rpcz_store.cc:276] 0709 14:52:44.104279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.104459 (+   180us) service_pool.cc:224] Handling call
0709 14:52:45.189816 (+1085357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.247514  3546 raft_consensus.cc:2889] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:45.286873  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.287767  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:45.288048  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1095) took 2300 ms. Trace:
I20260709 14:52:45.228178  6062 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:52:44.545969  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.232893  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 589) took 2587 ms. Trace:
I20260709 14:52:44.334579  6018 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:44.957628  6079 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.885s	user 0.003s	sys 0.011s
I20260709 14:52:44.543680  6059 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.738s
I20260709 14:52:45.227564  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.263708  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:45.264603  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.304273  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1227) took 1010 ms. Trace:
I20260709 14:52:45.305102  2606 rpcz_store.cc:276] 0709 14:52:44.294043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.227149 (+933106us) service_pool.cc:224] Handling call
0709 14:52:45.304183 (+ 77034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32896}
I20260709 14:52:44.382025  5718 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:45.258251  3547 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:45.308512  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:45.267777  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 607) took 1688 ms (client timeout 1510 ms). Trace:
W20260709 14:52:45.309743  2751 rpcz_store.cc:269] 0709 14:52:43.579291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.594127 (+ 14836us) service_pool.cc:224] Handling call
0709 14:52:45.267560 (+1673433us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.267689 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1145031936}
I20260709 14:52:45.268733  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 585) took 1169 ms. Trace:
I20260709 14:52:45.311336  2610 rpcz_store.cc:276] 0709 14:52:44.099669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.099862 (+   193us) service_pool.cc:224] Handling call
0709 14:52:45.268547 (+1168685us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.268666 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54867840}
W20260709 14:52:45.269558  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 602) took 1798 ms (client timeout 1832 ms). Trace:
I20260709 14:52:45.271749  2752 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 14:52:45.273032  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 579) took 1180 ms. Trace:
W20260709 14:52:45.312556  2755 rpcz_store.cc:269] 0709 14:52:43.471028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.471675 (+   647us) service_pool.cc:224] Handling call
0709 14:52:45.269386 (+1797711us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.269497 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39534848}
I20260709 14:52:45.313234  2605 rpcz_store.cc:276] 0709 14:52:44.092431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.095305 (+  2874us) service_pool.cc:224] Handling call
0709 14:52:44.103029 (+  7724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.324918 (+221889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.272689 (+947771us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b54000e27a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e649a02 00007fa16e645dd6 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:52:45.272958 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133206784}
I20260709 14:52:45.273571  3541 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:45.316397  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.317498  3541 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:45.256338  6118 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:45.319171  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.319990  3542 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:45.276176  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.321077  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:44.346204  6020 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:44.447533  6081 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:52:45.279011  2576 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 3.298s (SENT)
I20260709 14:52:45.327881  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:45.219007  3538 rpcz_store.cc:276] 0709 14:52:42.432873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.039312 (+1606439us) service_pool.cc:224] Handling call
0709 14:52:44.071328 (+ 32016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:44.397224 (+325896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:44.777753 (+380529us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54004b3cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:44.778090 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612574080}
I20260709 14:52:45.328058  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:45.226720  5877 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.084s	user 0.003s	sys 0.011s
I20260709 14:52:45.274461   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:52:45.282761  6190 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:45.209794  6016 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:44.894771  6165 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:44.796830  6183 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:44.812436  5683 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:45.278465  3543 rpcz_store.cc:269] 0709 14:52:42.434213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.955296 (+521083us) service_pool.cc:224] Handling call
0709 14:52:45.277537 (+2322241us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.277750 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349153664}
I20260709 14:52:45.288488  3546 rpcz_store.cc:276] 0709 14:52:42.987781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.436425 (+1448644us) service_pool.cc:224] Handling call
0709 14:52:45.050140 (+613715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.287691 (+237551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.287973 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:45.326758  6046 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:45.225848  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 767189us
I20260709 14:52:45.287156  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.335601  6081 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.291862  6168 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:45.286592  5880 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:45.297274  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.293823  5902 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:44.537305  6078 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.768s
I20260709 14:52:45.358003  6069 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:45.286557  6091 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.023s	sys 0.044s
I20260709 14:52:45.322247  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.385594  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.387674  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.278524  5766 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:45.232191  5788 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:45.367576  5877 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:45.355985  6190 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:45.405937  6190 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.374676  6077 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 14:52:45.410228  2680 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:45.265311  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.326678  6018 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:45.331343  6118 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.329909  3547 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:45.332600  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:45.425634  2679 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:45.337472  3542 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:45.339514  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.427843  3541 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:45.297888  6074 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 14:52:45.341471  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1211) took 2902 ms (client timeout 2399 ms). Trace:
W20260709 14:52:45.431870  2754 rpcz_store.cc:269] 0709 14:52:42.438584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.490471 (+ 51887us) service_pool.cc:224] Handling call
0709 14:52:45.341119 (+2850648us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.341347 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704785792}
I20260709 14:52:45.298382  6139 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:44.220098  6029 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.298585  6062 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:45.358549  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.293255  6054 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:45.449249  6054 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 14:52:45.449579  3547 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 14:52:45.450361  6054 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:45.368597  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89608.4us
I20260709 14:52:45.455933  6185 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:45.457702  2576 connection.cc:256] RPC call timeout handler was delayed by -2.7775043890000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23450000000000004s for a response
I20260709 14:52:45.457944  6011 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:45.458391  6011 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:45.369284  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.374783  2684 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:45.352221  6016 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.298866  2681 rpcz_store.cc:276] 0709 14:52:42.646014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.033295 (+1387281us) service_pool.cc:224] Handling call
0709 14:52:44.063602 (+ 30307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.232573 (+1168971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.232828 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16256}
I20260709 14:52:45.382774  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.430902  6052 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:45.323094  6079 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:45.396070  6069 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:45.335810  5718 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:45.397902  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:45.430511  6018 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:45.327961  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.372201  6081 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:45.416596  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1086) took 2456 ms. Trace:
W20260709 14:52:45.420680  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 155415us
I20260709 14:52:45.372552  6046 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:45.377053  6059 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 14:52:45.400561  5766 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:45.441376  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.441493  3538 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
W20260709 14:52:45.458781  2576 connection.cc:256] RPC call timeout handler was delayed by -2.5678043890000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2578s for a response
W20260709 14:52:45.480960  2576 connection.cc:256] RPC call timeout handler was delayed by -1.514576565000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.5476s for a response
W20260709 14:52:45.481329  2576 connection.cc:256] RPC call timeout handler was delayed by -1.4886157309999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 14:52:45.481678  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2708157309999706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 14:52:45.461771  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.483860  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:45.463742  3543 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:45.466300  2683 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:45.466653  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.488827  2681 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:45.468349  3540 rpcz_store.cc:276] 0709 14:52:42.960325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.122075 (+1161750us) service_pool.cc:224] Handling call
0709 14:52:44.228105 (+106030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:44.228607 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:44.632107 (+403500us) log.cc:844] Serialized 14 byte log entry
0709 14:52:45.179446 (+547339us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:45.199415 (+ 19969us) log.cc:844] Serialized 26 byte log entry
0709 14:52:45.199593 (+   178us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:45.199637 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.199732 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:45.416214 (+216482us) raft_consensus.cc:1685] finished
0709 14:52:45.416247 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:45.416491 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":525792,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":733,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":46587,"thread_start_us":865319,"threads_started":2,"wal-append.queue_time_us":697203}
W20260709 14:52:45.471103  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99013.9us
I20260709 14:52:45.468786  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:45.495847  2680 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:45.473764  3542 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:52:45.494689  3541 raft_consensus.cc:1217] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Deduplicated request from leader. Original: 4.2->[6.3-6.3]   Dedup: 6.3->[]
I20260709 14:52:45.494656  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:45.498047  3541 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 14:52:45.463706  6062 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:45.466287  6079 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:45.466652  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:45.430797  6074 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:45.520361  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:45.527070  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:45.464213  5793 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:45.462076  6016 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:45.486076  5766 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:45.552073  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.588850  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.589752  2684 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:52:45.553833  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.556217  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:45.558676  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.592005  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:45.564788  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49157.7us
I20260709 14:52:45.463596  6052 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:45.577574  2610 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:45.589157  2612 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:45.553148  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:45.564113  6145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.020s	sys 0.039s
I20260709 14:52:45.470763  6139 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:45.508755  6054 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:45.615154  6145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.021s	sys 0.040s
I20260709 14:52:45.616070  2611 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:45.597934  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.616961  6139 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.617380  6054 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:45.617811  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.618644  2609 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:45.572153  5793 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:45.619752  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1123) took 2718 ms. Trace:
I20260709 14:52:45.620257  2677 rpcz_store.cc:276] 0709 14:52:42.901009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.075855 (+1174846us) service_pool.cc:224] Handling call
0709 14:52:44.076206 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.217296 (+1141090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.619494 (+402198us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400377154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:45.619682 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332595456}
I20260709 14:52:45.620666  5793 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:45.605175  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.621556  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.621937  2684 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:45.621768  6137 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:45.623029  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:45.610268  5718 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.611872  6018 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:45.594432  2608 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:45.624300  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.625216  3541 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:45.625803  2683 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:45.573756  6185 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:45.628675  6185 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.619755  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.629208  6185 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:45.630754  2609 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 14:52:45.631708  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.639339  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.640569  2612 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
W20260709 14:52:45.644501  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48324us
W20260709 14:52:45.647333  2651 connection.cc:256] RPC call timeout handler was delayed by -1.4525532690000773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 14:52:45.648232  2651 connection.cc:256] RPC call timeout handler was delayed by -1.3580532690000382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 14:52:45.651952  2651 connection.cc:256] RPC call timeout handler was delayed by -1.1096532690000913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 14:52:45.653153  2677 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:45.662425  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.646409  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.665493  3540 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:45.668161  6111 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:45.668726  6111 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.672643  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.673640  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:45.677429  6111 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:45.686692  2679 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:45.700007  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.718381  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 603) took 2140 ms. Trace:
I20260709 14:52:45.682850  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.724094  2752 rpcz_store.cc:276] 0709 14:52:43.577788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:43.582952 (+  5164us) service_pool.cc:224] Handling call
0709 14:52:43.583387 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.312791 (+1729404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.717956 (+405165us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:45.718275 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3881836928}
I20260709 14:52:45.690313  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.720762  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.692691  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.459430  6011 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:45.646644  6046 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:45.669633  5902 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:45.682003  5914 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:45.663143  6059 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 14:52:45.728212  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.715777  6077 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 14:52:45.784843  2680 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:45.750205  2605 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 14:52:45.750859  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:45.746313  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.005s	sys 0.018s
W20260709 14:52:45.787072  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.005s	sys 0.018s
W20260709 14:52:45.787804  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.021s
W20260709 14:52:45.788215  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.004s	sys 0.021s
W20260709 14:52:45.757273  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.001s	sys 0.021s
I20260709 14:52:45.789289  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:45.790244  2752 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:45.762140  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.032s
W20260709 14:52:45.791455  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.009s	sys 0.032s
I20260709 14:52:45.793476  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.794303  2753 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:52:45.724131  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:45.795902  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:45.796586  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.797008  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:45.797649  2758 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:45.767671  2607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.023s	sys 0.018s
W20260709 14:52:45.799170  2607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.023s	sys 0.018s
I20260709 14:52:45.795020  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:45.799736  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.800143  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 590) took 1618 ms. Trace:
I20260709 14:52:45.800722  2607 rpcz_store.cc:276] 0709 14:52:44.182257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.227151 (+1044894us) service_pool.cc:224] Handling call
0709 14:52:45.227671 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.799624 (+571953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:45.799671 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:45.799743 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:45.799807 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.799904 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:45.800067 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107422,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":17430,"spinlock_wait_cycles":24448,"thread_start_us":571340,"threads_started":1}
W20260709 14:52:45.788817  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.002s	sys 0.022s
W20260709 14:52:45.802951  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 589470us
W20260709 14:52:45.777531  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94823.5us
I20260709 14:52:45.726186  3542 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:45.730237  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:45.769515  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.021s	sys 0.019s
W20260709 14:52:45.805061  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.021s	sys 0.020s
I20260709 14:52:45.805454  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:45.805812  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 592739us
I20260709 14:52:45.806625  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:45.806761  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:45.779625  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51542.7us
I20260709 14:52:45.627283  6052 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:45.772543  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 476621us
I20260709 14:52:45.811285  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:45.780694  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.004s	sys 0.038s
I20260709 14:52:45.748036  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 581) took 1654 ms. Trace:
I20260709 14:52:45.770300  6011 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:45.745703  6097 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:45.807652  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.808061  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:45.808218  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:45.797341  3540 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:45.810487  2753 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:45.812094  2604 rpcz_store.cc:276] 0709 14:52:44.093205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:44.093339 (+   134us) service_pool.cc:224] Handling call
0709 14:52:45.747706 (+1654367us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:45.747940 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657102208}
I20260709 14:52:45.820446  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:45.813011  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.004s	sys 0.038s
I20260709 14:52:45.822088  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.746949  6126 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:45.806954  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.784449  6078 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:45.828262  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:45.851676  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:45.833247  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.827240  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.853068  2751 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:45.830984  2753 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:45.833207  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:45.826619  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96447.9us
W20260709 14:52:45.832878  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 570694us
I20260709 14:52:45.856902  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.843539  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:45.835100  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.847514  2752 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:45.838086  2759 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:45.859331  2757 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:45.791594  6029 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:45.848781  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.793670  6183 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:45.861145  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.864841  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.865900  2755 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:45.855561  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.868436  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.861228  2754 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:45.765926  5902 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:45.833891  5959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.052s	sys 0.053s
I20260709 14:52:45.871742  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.884859  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.886173  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:45.821976  5788 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:45.874001  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:45.874368  6018 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:45.874853  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:45.837584  6126 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.899005  2759 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:52:45.822422  6054 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:45.833976  6097 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:45.900987  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.013s	sys 0.043s
I20260709 14:52:45.831365  5880 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:45.893694  2605 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:45.879316  6139 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:45.831389  5718 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 14:52:45.913743  2604 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:45.894270  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.852954  6145 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:45.850607  6078 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:45.880714  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:45.901466  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.013s	sys 0.043s
I20260709 14:52:45.745697  6074 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:45.887923  5788 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:45.912298  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.901438  6126 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:45.916699  2758 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:45.917523  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:45.839118  6202 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:45.884203  6046 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:45.892927  6018 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:45.919406  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:45.916275  2755 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:45.870788  6183 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:45.895018  6020 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 14:52:45.888037  5959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.053s	sys 0.053s
I20260709 14:52:45.912633  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:45.967623  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.988214  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.988754  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:45.967785  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 87132.3us
I20260709 14:52:45.990406  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.973927  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:45.908519  6097 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:45.993314  2682 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.013s	sys 0.019s
W20260709 14:52:45.993801  2682 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.013s	sys 0.019s
I20260709 14:52:45.973094  2759 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:45.983433  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:45.996281  2752 raft_consensus.cc:2376] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8. Re-sending same reply.
I20260709 14:52:45.918759  6145 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:46.038427  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:45.987485  2757 raft_consensus.cc:2376] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9. Re-sending same reply.
I20260709 14:52:45.935670  6078 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:46.005355  2605 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:45.994257  6074 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:45.971372  5788 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:45.969467  6036 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:45.936995  6054 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:45.981196  6183 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:46.056200  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:46.074237  2752 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:45.977794  6146 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:45.980970  6137 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:46.036652  5718 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:45.976430  6201 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:46.076053  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 219228us
I20260709 14:52:45.980584  6018 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:45.977034  6202 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:46.077557  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 244337us
I20260709 14:52:45.998908  6046 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:46.071465  2754 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:52:46.044407  6145 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:45.986063  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.028s	sys 0.059s
W20260709 14:52:46.086226  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 278061us
W20260709 14:52:46.712262  2609 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.009s	sys 0.025s
I20260709 14:52:46.105856  5718 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:46.893348  6059 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:46.744565  5902 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:46.744642  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:46.897900  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:46.062954  5959 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 14:52:46.747776  6016 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.138s	user 0.012s	sys 0.013s
I20260709 14:52:46.817371  5880 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:52:46.747540  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 643479us
W20260709 14:52:46.749199  6033 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.510s	user 0.023s	sys 0.005s
W20260709 14:52:46.728585  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.005s	sys 0.003s
I20260709 14:52:46.751652  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:46.752166  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:46.069123  6036 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:46.317220  6046 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:46.755288  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:46.078806  6054 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:46.758594  2680 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 14:52:46.721787  2610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.008s	sys 0.015s
W20260709 14:52:46.724009  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.007s	sys 0.021s
W20260709 14:52:46.722709  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.000s	sys 0.023s
I20260709 14:52:46.158197  6118 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:47.018468  6054 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:47.022112  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47091.1us
I20260709 14:52:47.023860  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1036) took 1650 ms. Trace:
I20260709 14:52:47.024488  2683 rpcz_store.cc:276] 0709 14:52:45.373056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.626543 (+253487us) service_pool.cc:224] Handling call
0709 14:52:45.626906 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:45.650612 (+ 23706us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:45.650662 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:45.668457 (+ 17795us) log.cc:844] Serialized 26 byte log entry
0709 14:52:46.001569 (+333112us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:46.001623 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:46.001854 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.023196 (+1021342us) raft_consensus.cc:1685] finished
0709 14:52:47.023259 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.023540 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17505,"raft.queue_time_us":18411,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":17729,"thread_start_us":332628,"threads_started":1,"wal-append.queue_time_us":503917}
W20260709 14:52:46.711787  2603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.061s	user 0.015s	sys 0.019s
W20260709 14:52:47.027230  2603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.377s	user 0.019s	sys 0.019s
W20260709 14:52:46.711813  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:46.092444  6201 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:46.111740  6202 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:46.161022  6203 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:46.037745  6205 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:46.151515  6207 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:46.713120  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:46.183202  6199 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.375s	user 0.002s	sys 0.000s
I20260709 14:52:46.061084  6210 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:47.329548  6054 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:46.874379  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:46.878011  2609 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.199s	user 0.017s	sys 0.029s
W20260709 14:52:46.878499  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.010s	sys 0.008s
I20260709 14:52:46.094977  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:47.349910  6111 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.672s	user 0.006s	sys 0.028s
W20260709 14:52:47.350891  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.25578e+06us
I20260709 14:52:47.351485  6244 raft_consensus.cc:2764] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 14:52:47.352496  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:45.986781  6020 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:47.355470  6020 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:47.355985  6079 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:52:46.911543  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.030s	sys 0.047s
I20260709 14:52:46.913154  5959 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:47.359311  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.031s	sys 0.048s
I20260709 14:52:47.360457  5959 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:47.361672  6097 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.290s	user 0.033s	sys 0.019s
I20260709 14:52:46.913699  6016 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:47.362947  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.011s	sys 0.008s
W20260709 14:52:47.364238  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.37432e+06us
W20260709 14:52:46.922217  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.006s	sys 0.004s
I20260709 14:52:46.871989  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:46.934234  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:46.938107  6059 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:46.939571  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.012s	sys 0.028s
I20260709 14:52:46.919770  5718 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:46.939890  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:46.940210  6069 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:46.940910  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1217) took 1111 ms. Trace:
I20260709 14:52:46.942687  6229 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:46.974159  6036 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:46.967403  5902 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 14:52:46.915584  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:47.003690  2726 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.000s	sys 0.012s
W20260709 14:52:47.004523  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.013s	sys 0.000s
I20260709 14:52:47.004081  3536 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 14:52:47.012141  2686 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
W20260709 14:52:47.043257  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.011s	sys 0.016s
W20260709 14:52:47.056114  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.285s	user 0.000s	sys 0.027s
W20260709 14:52:47.065399  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.008s	sys 0.025s
I20260709 14:52:47.143025  6201 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:47.147923  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:47.149008  2651 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 3.781s (SENT) [suppressed 31 similar messages]
I20260709 14:52:47.115953  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:47.176656  3540 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:52:47.057991  5880 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:47.404978  6077 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 14:52:47.240756  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 263825us
W20260709 14:52:47.405900  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:47.407966  6011 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.563s	user 0.020s	sys 0.015s
I20260709 14:52:47.408919  6059 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.362370  6097 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.243443  6231 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:47.244158  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:47.255256  6253 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:47.266057  6018 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.113s	user 0.008s	sys 0.017s
I20260709 14:52:47.309738  6203 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:47.311391  6055 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:47.315481  5914 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:46.713675  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:47.347872  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1108) took 1692 ms. Trace:
W20260709 14:52:47.348519  5766 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.743s	user 0.016s	sys 0.016s
W20260709 14:52:47.349505  6126 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.362s	user 0.011s	sys 0.010s
W20260709 14:52:47.384594  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.524s	user 0.013s	sys 0.028s
W20260709 14:52:47.330766  5788 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.262s	user 0.011s	sys 0.015s
I20260709 14:52:47.385530  2604 rpcz_store.cc:276] 0709 14:52:45.828868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.873600 (+ 44732us) service_pool.cc:224] Handling call
0709 14:52:46.899734 (+1026134us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:46.940529 (+ 40795us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58628352}
I20260709 14:52:47.331095  6207 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:47.350526  6111 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:47.402443  6081 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.935s	user 0.022s	sys 0.011s
I20260709 14:52:47.403656  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 598) took 2143 ms. Trace:
W20260709 14:52:47.403908  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 488708us
W20260709 14:52:47.404479  2726 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.594s	user 0.001s	sys 0.013s
W20260709 14:52:47.404879  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.508s	user 0.015s	sys 0.000s
I20260709 14:52:47.411971  6097 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:47.412639  2754 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:47.413720  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1242) took 1732 ms. Trace:
W20260709 14:52:47.414199  6145 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.214s	user 0.031s	sys 0.035s
I20260709 14:52:47.414947  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:47.417510  2607 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:52:47.418201  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.61269e+06us
I20260709 14:52:47.414252  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1234) took 1834 ms. Trace:
W20260709 14:52:47.419118  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 392236us
W20260709 14:52:47.423614  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58275.8us
I20260709 14:52:47.370270  5959 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:47.434806  2681 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.503s	user 0.022s	sys 0.013s
I20260709 14:52:47.466532  2681 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:47.365381  6016 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:47.399895  6244 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:47.450548  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 96501.8us
I20260709 14:52:47.463233  2755 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:47.419485  6069 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:47.436236  6203 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:47.497453  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1212) took 1975 ms. Trace:
W20260709 14:52:47.498311  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48329us
I20260709 14:52:47.498544  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:47.485483  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1222) took 1470 ms (client timeout 1789 ms). Trace:
W20260709 14:52:47.510926  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.70052e+06us
I20260709 14:52:47.449937  6011 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:47.535761  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 407705us
I20260709 14:52:47.437366  3541 rpcz_store.cc:276] 0709 14:52:45.655183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.658088 (+  2905us) service_pool.cc:224] Handling call
0709 14:52:45.721753 (+ 63665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.722063 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:45.753550 (+ 31487us) log.cc:844] Serialized 14 byte log entry
0709 14:52:46.048088 (+294538us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.269743 (+1221655us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.307455 (+ 37712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:47.332927 (+ 25472us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:47.347047 (+ 14120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.347256 (+   209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.347535 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":694057,"prepare.queue_time_us":281954,"prepare.run_cpu_time_us":192,"prepare.run_wall_time_us":2458,"raft.queue_time_us":1194122,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":33631,"thread_start_us":678291,"threads_started":3}
I20260709 14:52:47.452636  6207 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:47.533600  2607 rpcz_store.cc:269] 0709 14:52:45.994918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:46.939035 (+944117us) service_pool.cc:224] Handling call
0709 14:52:47.465339 (+526304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:47.485169  6244 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:47.465728  2608 rpcz_store.cc:276] 0709 14:52:45.558738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.624441 (+ 65703us) service_pool.cc:224] Handling call
0709 14:52:45.624801 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:45.625236 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:45.969520 (+344284us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.360120 (+1390600us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:47.360999 (+   879us) log.cc:844] Serialized 26 byte log entry
0709 14:52:47.361292 (+   293us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:47.361338 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.361489 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.365636 (+  4147us) raft_consensus.cc:1685] finished
0709 14:52:47.365662 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.393372 (+ 27710us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34865,"prepare.queue_time_us":1349359,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":120,"raft.queue_time_us":42545,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":148,"thread_start_us":1604682,"threads_started":2,"wal-append.queue_time_us":1254178}
W20260709 14:52:47.552235  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53940.6us
I20260709 14:52:47.541790  2758 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:47.456346  5766 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.499137  6205 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:47.505255  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:47.459463  2611 rpcz_store.cc:276] 0709 14:52:45.681888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.730013 (+ 48125us) service_pool.cc:224] Handling call
0709 14:52:45.730434 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:45.743260 (+ 12826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:45.743307 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:45.761346 (+ 18039us) log.cc:844] Serialized 52 byte log entry
0709 14:52:47.412468 (+1651122us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:47.412915 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.413232 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.413361 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.413397 (+    36us) raft_consensus.cc:1685] finished
0709 14:52:47.413408 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.413592 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12107,"prepare.queue_time_us":297,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":12310,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":10353,"thread_start_us":1623238,"threads_started":1,"wal-append.queue_time_us":1477260}
I20260709 14:52:47.458277  2682 rpcz_store.cc:276] 0709 14:52:45.260655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.260821 (+   166us) service_pool.cc:224] Handling call
0709 14:52:45.261232 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.261524 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:45.994229 (+732705us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.402821 (+1408592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:47.402978 (+   157us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:47.403170 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.403272 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.403535 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":847905,"prepare.run_cpu_time_us":23378,"prepare.run_wall_time_us":651938,"raft.queue_time_us":1624,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":196,"thread_start_us":2103630,"threads_started":2}
I20260709 14:52:47.504158  2603 rpcz_store.cc:276] 0709 14:52:45.522877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.552150 (+ 29273us) service_pool.cc:224] Handling call
0709 14:52:45.552631 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.553009 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:47.028187 (+1475178us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.496372 (+468185us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.496728 (+   356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:47.496850 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:47.496941 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.497091 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.497350 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11208,"prepare.queue_time_us":1465982,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":89580,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":1738008,"threads_started":3,"wal-append.queue_time_us":222630}
I20260709 14:52:47.544175  6011 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:47.452865  6018 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:47.578770  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:47.579002  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:47.571564  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:47.585126  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44095.2us
I20260709 14:52:47.456377  6126 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.588795  2604 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:47.557919  6244 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:47.571269  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.752s (SENT)
I20260709 14:52:47.588564  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 961) took 1212 ms. Trace:
I20260709 14:52:47.488445  6081 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:47.590622  5902 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:47.594591  6182 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 2.362s	user 0.034s	sys 0.039s
W20260709 14:52:47.595247  6215 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.187s	user 0.000s	sys 0.002s
I20260709 14:52:47.595328  6182 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.587791  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:47.463554  6253 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.591712  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:47.578408  5877 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.966s	user 0.021s	sys 0.017s
I20260709 14:52:47.605346  6261 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:47.606029  5877 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:47.606427  6261 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.606973  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1240) took 1951 ms. Trace:
I20260709 14:52:47.607434  2612 rpcz_store.cc:276] 0709 14:52:45.560650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.642404 (+ 81754us) service_pool.cc:224] Handling call
0709 14:52:45.643006 (+   602us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:45.666354 (+ 23348us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:46.049518 (+383164us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.269604 (+1220086us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:47.411573 (+141969us) log.cc:844] Serialized 52 byte log entry
0709 14:52:47.416283 (+  4710us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:47.433334 (+ 17051us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.433721 (+   387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.451908 (+ 18187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.451974 (+    66us) raft_consensus.cc:1685] finished
0709 14:52:47.451991 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.511908 (+ 59917us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13829,"prepare.queue_time_us":1353331,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":139,"raft.queue_time_us":116267,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":46368,"thread_start_us":561061,"threads_started":3,"wal-append.queue_time_us":610876}
W20260709 14:52:47.609437  6074 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.548s	user 0.013s	sys 0.009s
I20260709 14:52:47.604912  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1099) took 2348 ms. Trace:
I20260709 14:52:47.598716  5880 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.610152  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:47.610383  6139 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.154s
I20260709 14:52:47.594169  6168 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:47.499915  6145 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.553480  6055 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.610203  2606 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:47.610903  6227 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 14:52:47.611791  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:47.612083  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:47.599475  5902 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:47.583633  6011 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:47.594774  6245 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 14:52:47.615285  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:47.594810  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:47.501406  5788 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.657598  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:47.654997  2577 leader_election.cc:336] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.566s (SENT)
I20260709 14:52:47.708508  2678 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:47.591857  3538 rpcz_store.cc:276] 0709 14:52:46.171119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:46.806307 (+635188us) service_pool.cc:224] Handling call
0709 14:52:46.962785 (+156478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:46.963809 (+  1024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:46.963848 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:46.964688 (+   840us) log.cc:844] Serialized 26 byte log entry
0709 14:52:47.130439 (+165751us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:47.130507 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.135882 (+  5375us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.253150 (+117268us) raft_consensus.cc:1685] finished
0709 14:52:47.253180 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.383336 (+130156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":286574,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":163949,"threads_started":1,"wal-append.queue_time_us":280171}
W20260709 14:52:47.716259  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141413us
I20260709 14:52:47.541352  5914 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.707240  3545 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:47.780707  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 326975us
I20260709 14:52:47.711221  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 618) took 1904 ms. Trace:
I20260709 14:52:47.712389  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:47.714042  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:47.710124  2651 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.557s (SENT)
I20260709 14:52:47.579402  6031 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:47.630394  2752 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 7. Current term is 8. Ops: [3.2-7.4]
I20260709 14:52:47.821030  2677 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 14:52:47.818296  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 230598us
W20260709 14:52:47.712057  6078 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.464s	user 0.019s	sys 0.004s
I20260709 14:52:47.830902  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:47.711829  6062 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 2.206s	user 0.019s	sys 0.011s
W20260709 14:52:47.832386  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:47.779726  6054 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:47.716655  6016 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:47.834704  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:47.835152  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:47.837653  2612 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:52:47.842212  6260 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 14:52:47.844288  6029 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:47.850432  6202 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.703s	user 0.014s	sys 0.000s
I20260709 14:52:47.850988  6253 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:47.851852  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 288030us
I20260709 14:52:47.588316  6069 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.853195  6126 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:47.847584  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:47.856405  5959 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:47.854089  6079 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:47.857555  6111 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.859961  5766 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:47.955046  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 245036us
W20260709 14:52:47.864195  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:47.956373  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:47.831882  6185 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.896s	user 0.018s	sys 0.015s
W20260709 14:52:47.956959  2651 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.662s (SENT)
I20260709 14:52:47.957531  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:47.957392  6185 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.959612  6055 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.960664  6185 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.864682  6081 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:47.961778  3411 connection.cc:256] RPC call timeout handler was delayed by -2.3391773749999629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 14:52:47.821303  5877 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:47.962779  5877 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.866746  5788 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:47.967499  5788 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:47.968258  5788 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:47.809127  5902 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:47.708287  6205 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:47.868283  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:47.962379  3411 connection.cc:256] RPC call timeout handler was delayed by -2.0736773749999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 14:52:47.976927  3411 connection.cc:256] RPC call timeout handler was delayed by -2.0070773749999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 14:52:47.718525  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.86677e+06us
I20260709 14:52:47.877743  2604 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:47.779145  6182 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:47.831514  2611 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:47.707913  6096 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:47.892270  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 262180us
W20260709 14:52:47.892858  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 307421us
W20260709 14:52:47.894619  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:47.623179  3547 rpcz_store.cc:276] 0709 14:52:45.224920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.450687 (+225767us) service_pool.cc:224] Handling call
0709 14:52:45.451124 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.451394 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:45.451791 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.229768 (+1777977us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.433877 (+204109us) log.cc:844] Serialized 14 byte log entry
0709 14:52:47.469550 (+ 35673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:47.486299 (+ 16749us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:47.534563 (+ 48264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.548634 (+ 14071us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.570697 (+ 22063us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32282,"prepare.queue_time_us":178335,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":141,"raft.queue_time_us":404952,"raft.run_cpu_time_us":666,"raft.run_wall_time_us":43063,"thread_start_us":1789077,"threads_started":4,"wal-append.queue_time_us":1893881}
I20260709 14:52:47.900525  6059 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:47.901405  6231 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:47.854419  6183 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.762s	user 0.006s	sys 0.013s
I20260709 14:52:47.809554  2756 rpcz_store.cc:276] 0709 14:52:45.806361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.813461 (+  7100us) service_pool.cc:224] Handling call
0709 14:52:45.813886 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:45.836676 (+ 22790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:45.836723 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:45.837328 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 14:52:47.496424 (+1659096us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:47.496536 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.496728 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:47.710776 (+214048us) raft_consensus.cc:1685] finished
0709 14:52:47.710809 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:47.711089 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22223,"raft.queue_time_us":22407,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":135,"thread_start_us":1561845,"threads_started":1,"wal-append.queue_time_us":1718901}
I20260709 14:52:47.710446  6201 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:47.907575  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:47.908586  2758 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:47.909197  6031 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:47.911818  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:47.912658  2755 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
W20260709 14:52:47.929798  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 338804us
I20260709 14:52:47.852843  6018 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.931798  2608 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:47.940941  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:47.924937  6029 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:47.949088  2605 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:47.849189  5914 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.856439  6036 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:47.954151  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:47.977469  3411 connection.cc:256] RPC call timeout handler was delayed by -1.6569773749999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24350000000000002s for a response
I20260709 14:52:48.015763  2607 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:47.924253  6016 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:47.878100  6139 consensus_queue.cc:1243] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:52:47.827188  6168 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:48.005333  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:47.935480  6261 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:47.966634  5877 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.013025  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1221) took 2043 ms. Trace:
W20260709 14:52:48.006009  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 411410us
I20260709 14:52:48.075714  2608 rpcz_store.cc:276] 0709 14:52:45.969922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.621407 (+1651485us) service_pool.cc:224] Handling call
0709 14:52:48.012927 (+391520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:47.976928  6079 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:47.961817  6185 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:48.026986  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.027197  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.027882  6234 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:48.029053  6213 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.063s	user 0.005s	sys 0.001s
I20260709 14:52:48.015389  6183 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:48.002879  6231 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.001453  6059 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.023262  6097 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 14:52:48.012635  6029 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.031893  2752 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 14:52:48.032449  6200 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.085s	user 0.000s	sys 0.001s
W20260709 14:52:48.041985  2726 connection.cc:256] RPC call timeout handler was delayed by -2.2124257709999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3251s for a response
I20260709 14:52:48.027043  2759 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:48.086578  6231 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.046096  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 596) took 2786 ms. Trace:
I20260709 14:52:48.047219  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.095628  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.047991  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:48.049080  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 947) took 2828 ms (client timeout 2125 ms). Trace:
I20260709 14:52:48.096807  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.050936  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.051625  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.098284  2611 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:48.098778  2754 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.099514  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.099823  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.100350  2611 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:48.100718  2754 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.101737  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.102407  2754 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:48.102986  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.103268  6111 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:48.103976  2757 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:48.104401  2607 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:48.097295  3536 rpcz_store.cc:269] 0709 14:52:45.220231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.253314 (+ 33083us) service_pool.cc:224] Handling call
0709 14:52:48.048700 (+2795386us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400317c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.048910 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3935667584}
I20260709 14:52:48.105055  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.105365  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.106009  2607 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:48.107617  2604 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:48.093726  6059 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.108678  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.109249  2604 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:48.095731  2678 rpcz_store.cc:276] 0709 14:52:45.259986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.260390 (+   404us) service_pool.cc:224] Handling call
0709 14:52:45.287353 (+ 26963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:47.828644 (+2541291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.045703 (+217059us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:48.046017 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5279075328}
I20260709 14:52:48.112380  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 599) took 2851 ms. Trace:
W20260709 14:52:48.113381  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65605.9us
I20260709 14:52:48.114272  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:48.115228  6182 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:48.116405  6182 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:48.116850  6182 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.117287  6182 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.115199  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:48.053380  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.054137  5788 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:47.934029  6202 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.000957  6096 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:48.126479  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.024s	sys 0.025s
I20260709 14:52:48.055163  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:46.895486   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:52:48.056952  6203 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:48.058616  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1220) took 2143 ms (client timeout 1848 ms). Trace:
I20260709 14:52:48.059564  6036 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.060487  6207 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.062120  6031 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.064561  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:48.024322  6016 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.065724  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.067294  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1045) took 1310 ms. Trace:
I20260709 14:52:48.068418  6244 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:48.069118  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.071125  6069 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.071012  6268 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:48.072706  6145 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.073133  6201 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.073213  5793 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:48.073980  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 166549us
I20260709 14:52:47.913617  6078 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:47.924749  6062 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.076879  2606 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:48.077387  6079 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:48.065043  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.400s (SENT)
W20260709 14:52:48.086215  6222 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.092s	user 0.004s	sys 0.000s
I20260709 14:52:48.085971  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.081434  5959 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:48.106734  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.052922  6168 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:48.173972  2651 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.819s (SENT)
I20260709 14:52:48.158116  2680 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
W20260709 14:52:48.159933  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.025s	sys 0.025s
I20260709 14:52:48.197793  2607 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:52:48.157109  2609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.030s
W20260709 14:52:48.325253  2609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.013s	sys 0.047s
W20260709 14:52:48.295913  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 453087us
W20260709 14:52:48.294757  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114185us
I20260709 14:52:48.174531  2608 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.112870  2685 rpcz_store.cc:276] 0709 14:52:45.260986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.261317 (+   331us) service_pool.cc:224] Handling call
0709 14:52:45.261727 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:45.297502 (+ 35775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:45.297539 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:45.299758 (+  2219us) log.cc:844] Serialized 52 byte log entry
0709 14:52:45.771896 (+472138us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:45.771976 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:45.772097 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:48.028026 (+2255929us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:48.111821 (+ 83795us) raft_consensus.cc:1685] finished
0709 14:52:48.111845 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.112161 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56216,"raft.queue_time_us":38700,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":26705,"spinlock_wait_cycles":38419328,"thread_start_us":464709,"threads_started":1,"wal-append.queue_time_us":773523}
W20260709 14:52:48.190421  2577 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.930s (SENT)
W20260709 14:52:48.305099  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.300591  2754 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:48.184725  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.310487  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:48.303754  2677 rpcz_store.cc:276] 0709 14:52:46.756990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:46.811808 (+ 54818us) service_pool.cc:224] Handling call
0709 14:52:48.067018 (+1255210us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54000c9ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.067205 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103229696}
I20260709 14:52:48.320914  2604 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:52:48.160631  2605 rpcz_store.cc:269] 0709 14:52:45.915442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:46.054114 (+138672us) service_pool.cc:224] Handling call
0709 14:52:48.058347 (+2004233us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54003c8054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.058525 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965001600}
I20260709 14:52:48.120411  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
W20260709 14:52:48.323540  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2525601799999322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 14:52:48.333076  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2057601799999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 14:52:48.333494  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1373601799999733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 14:52:48.328197  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:48.304896  6020 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.949s	user 0.014s	sys 0.006s
I20260709 14:52:48.329929  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:48.288193  6183 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:48.329195  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 211326us
I20260709 14:52:48.188402  6203 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.344523  2603 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 8: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:48.347572  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1218) took 2492 ms. Trace:
I20260709 14:52:48.348135  2603 rpcz_store.cc:276] 0709 14:52:45.854895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.075687 (+2220792us) service_pool.cc:224] Handling call
0709 14:52:48.347495 (+271808us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:48.333961  2752 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:48.265959  6185 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 14:52:48.329476  6097 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:48.349715  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:48.345173  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.034s	sys 0.048s
I20260709 14:52:48.349469  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.350222  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.355080  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:48.356782  2757 raft_consensus.cc:2376] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12. Re-sending same reply.
I20260709 14:52:48.357108  2754 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.357067  2760 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:48.358132  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.330165  5959 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:48.278053  5902 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:48.358055  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.280388  6011 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.361289  2604 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:48.349925  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 333040us
W20260709 14:52:48.362048  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.034s	sys 0.049s
I20260709 14:52:48.347415  5880 raft_consensus.cc:2764] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 14:52:48.366730  2757 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:48.376027  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1046) took 1003 ms. Trace:
I20260709 14:52:48.371809  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1239) took 2811 ms. Trace:
I20260709 14:52:48.368638  2753 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:48.379809  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:48.381078  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 220925us
I20260709 14:52:48.421017  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.415795  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1107) took 1406 ms. Trace:
W20260709 14:52:48.416832  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63405.5us
I20260709 14:52:48.369364  2607 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:48.414216  2754 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:48.417557  2603 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 14:52:48.351060  6261 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:48.428176  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 363814us
W20260709 14:52:48.428704  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.366657  2605 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:52:48.413970  2680 rpcz_store.cc:276] 0709 14:52:47.372308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.390032 (+ 17724us) service_pool.cc:224] Handling call
0709 14:52:48.375715 (+985683us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.375933 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363672960}
I20260709 14:52:48.428810  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:48.433470  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1226) took 1303 ms. Trace:
I20260709 14:52:48.433974  2604 rpcz_store.cc:276] 0709 14:52:47.131511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.345427 (+1213916us) service_pool.cc:224] Handling call
0709 14:52:48.361059 (+ 15632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.412305 (+ 51246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.412560 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:48.436556  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.437958  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1223) took 1720 ms. Trace:
I20260709 14:52:48.438462  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1214) took 2686 ms. Trace:
I20260709 14:52:48.368983  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.438531  2605 rpcz_store.cc:276] 0709 14:52:46.711911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.333849 (+1621938us) service_pool.cc:224] Handling call
0709 14:52:48.355673 (+ 21824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.431209 (+ 75536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.431407 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:48.440379  2607 rpcz_store.cc:276] 0709 14:52:45.745155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.339372 (+2594217us) service_pool.cc:224] Handling call
0709 14:52:48.369134 (+ 29762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.431118 (+ 61984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.431441 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:48.426105  2759 rpcz_store.cc:276] 0709 14:52:47.018916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.081354 (+ 62438us) service_pool.cc:224] Handling call
0709 14:52:47.092519 (+ 11165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.095346 (+1002827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.415455 (+320109us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:48.415714 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129907072}
I20260709 14:52:48.342769  6029 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.443637  2682 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:48.445755  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 945) took 3226 ms (client timeout 1991 ms). Trace:
I20260709 14:52:48.445868  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:48.446894  6029 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:48.447451  3537 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:48.447839  2686 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:48.369251  6168 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:48.444897  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:48.448823  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.413702  2610 rpcz_store.cc:276] 0709 14:52:45.560353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.633135 (+ 72782us) service_pool.cc:224] Handling call
0709 14:52:45.633575 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:45.634086 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:45.634128 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:45.643556 (+  9428us) log.cc:844] Serialized 26 byte log entry
0709 14:52:47.523946 (+1880390us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:47.541637 (+ 17691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:47.563634 (+ 21997us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:48.077528 (+513894us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:48.371436 (+293908us) raft_consensus.cc:1685] finished
0709 14:52:48.371460 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.371718 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1162,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":119,"spinlock_wait_cycles":12652416,"thread_start_us":1647926,"threads_started":1,"wal-append.queue_time_us":1303645}
I20260709 14:52:48.371021  5766 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:48.456971  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:48.426620  2603 pending_rounds.cc:85] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 6
W20260709 14:52:48.446264  3545 rpcz_store.cc:269] 0709 14:52:45.219700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.220661 (+   961us) service_pool.cc:224] Handling call
0709 14:52:48.439547 (+3218886us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54003df9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.445664 (+  6117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5469183360}
I20260709 14:52:48.450610  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1237) took 1263 ms. Trace:
I20260709 14:52:48.456084  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1047) took 1022 ms. Trace:
I20260709 14:52:48.373199  6078 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.448871  6097 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:48.346501  6020 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.459676  2611 rpcz_store.cc:276] 0709 14:52:47.177297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.428452 (+1251155us) service_pool.cc:224] Handling call
0709 14:52:48.428851 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.440303 (+ 11452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:48.440353 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:48.440406 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:48.440459 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.440543 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.440702 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":855,"raft.queue_time_us":19163,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":135}
I20260709 14:52:48.458639  2752 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:48.346776  6268 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.461319  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:48.462037  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 397027us
I20260709 14:52:48.462791  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.464262  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.465411  6081 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:48.469193  6244 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:48.473479  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.471184  2757 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.460347  2603 pending_rounds.cc:107] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 14:52:48.475080  2603 raft_consensus.cc:2889] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:48.471939  2754 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:48.471882  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.463384  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1236) took 2904 ms. Trace:
I20260709 14:52:48.469434  6062 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.431334  6231 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.483878  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:48.491577  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 315725us
I20260709 14:52:48.453363  6168 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.474498  6081 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.469884  6126 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:48.501322  5902 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:48.487298  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.488678  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.502516  2608 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:48.491719  2758 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:48.460255  2683 rpcz_store.cc:276] 0709 14:52:47.436282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.446787 (+ 10505us) service_pool.cc:224] Handling call
0709 14:52:47.487259 (+ 40472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:47.598857 (+111598us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:47.715095 (+116238us) log.cc:844] Serialized 14 byte log entry
0709 14:52:48.442051 (+726956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:48.442110 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:48.449183 (+  7073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.449262 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.449385 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10413,"prepare.queue_time_us":104136,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":5548,"raft.queue_time_us":14645,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":6975,"spinlock_wait_cycles":10771328,"thread_start_us":696766,"threads_started":1,"wal-append.queue_time_us":727936}
W20260709 14:52:48.494258  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.578s (SENT)
I20260709 14:52:48.494402  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:48.495325  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:48.495735  6272 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:52:48.493608  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.467077  5880 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:48.370565  5959 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.498279  2760 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:48.498704  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.499359  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.500023  6207 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.500464  6081 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.500804  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.502367  2607 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:48.489188  6097 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:48.503849  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44408.6us
I20260709 14:52:48.506564  2755 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.539628  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.501961  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.363131  6096 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.500255  6260 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:48.540385  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.540913  6260 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.541543  2755 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:48.509732  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:48.543264  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1049) took 1084 ms. Trace:
I20260709 14:52:48.543437  6261 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:52:48.543749  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49540.8us
I20260709 14:52:48.544390  6231 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:48.544584  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.582s (SENT)
I20260709 14:52:48.545063  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:48.546038  6011 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:48.519620  6244 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 14:52:48.548022  6244 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:48.548483  6244 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:48.549453  6244 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:48.541554  6096 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.551313  6020 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.543758  2679 rpcz_store.cc:276] 0709 14:52:47.458711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.459031 (+   320us) service_pool.cc:224] Handling call
0709 14:52:47.524862 (+ 65831us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:47.590771 (+ 65909us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 14:52:47.613336 (+ 22565us) log.cc:844] Serialized 14 byte log entry
0709 14:52:48.324837 (+711501us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:48.347436 (+ 22599us) log.cc:844] Serialized 52 byte log entry
0709 14:52:48.347635 (+   199us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:48.347677 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.347764 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:48.543035 (+195271us) raft_consensus.cc:1685] finished
0709 14:52:48.543058 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.543193 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30701,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":64,"raft.queue_time_us":21426,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10612,"thread_start_us":710985,"threads_started":1,"wal-append.queue_time_us":750195}
W20260709 14:52:48.555958  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59339.9us
I20260709 14:52:48.512239  2754 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:48.559298  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:48.559675  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:48.561497  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 807) took 5300 ms. Trace:
I20260709 14:52:48.562165  6239 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:48.561976  3537 rpcz_store.cc:276] 0709 14:52:43.261650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.232366 (+1970716us) service_pool.cc:224] Handling call
0709 14:52:48.447064 (+3214698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.447959 (+   895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.561250 (+113291us) spinlock_profiling.cc:243] Waited 3.16 s on lock 0x7b54003df9d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.561415 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7265076608}
I20260709 14:52:48.562871  6239 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:48.563846  6011 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:48.564013  6239 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.565083  6011 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:48.565470  6011 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:48.570966  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1211) took 3051 ms (client timeout 3251 ms). Trace:
W20260709 14:52:48.571558  2606 rpcz_store.cc:269] 0709 14:52:45.519851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.520001 (+   150us) service_pool.cc:224] Handling call
0709 14:52:48.570655 (+3050654us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400345b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:48.570871 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4801510272}
I20260709 14:52:48.572834  2682 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:48.512320  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.486608  6202 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.576042  2760 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:48.578159  6279 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
I20260709 14:52:48.464875  5766 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:48.509565  6185 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:48.527467  6059 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.506142  6168 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:48.534286  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.476995  2609 rpcz_store.cc:276] 0709 14:52:45.559435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.631847 (+ 72412us) service_pool.cc:224] Handling call
0709 14:52:45.632185 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:45.632438 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:47.331761 (+1699323us) log.cc:844] Serialized 14 byte log entry
0709 14:52:48.348061 (+1016300us) log.cc:844] Serialized 14 byte log entry
0709 14:52:48.354902 (+  6841us) log.cc:844] Serialized 14 byte log entry
0709 14:52:48.364126 (+  9224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:48.388193 (+ 24067us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:48.431668 (+ 43475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.448831 (+ 17163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:48.463317 (+ 14486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75545,"prepare.queue_time_us":1390175,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":4765,"raft.queue_time_us":288512,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":13758,"spinlock_wait_cycles":72974208,"thread_start_us":2584299,"threads_started":3,"wal-append.queue_time_us":570414}
W20260709 14:52:48.535053  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46474.3us
I20260709 14:52:48.472954  6182 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:48.537873  2577 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.094s (SENT)
I20260709 14:52:48.509526  2678 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:48.541503  6260 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:48.511027  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.521333  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1229) took 1359 ms. Trace:
I20260709 14:52:48.615172  2603 rpcz_store.cc:276] 0709 14:52:47.151074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.349105 (+1198031us) service_pool.cc:224] Handling call
0709 14:52:48.364533 (+ 15428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.505952 (+141419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.510425 (+  4473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:48.509871  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.582466  2758 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:52:48.587291  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78947.3us
I20260709 14:52:48.617684  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:48.618275  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.528959  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:48.597509  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:48.604241  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64719.4us
I20260709 14:52:48.620316  2757 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:48.582973  6244 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:48.624693  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.628891  5766 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:48.591099  6126 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:48.631918  6126 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:48.635262  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 97423us
W20260709 14:52:48.636093  2577 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.221s (SENT)
I20260709 14:52:48.636535  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:48.636878  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.639175  2686 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:48.639991  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111207us
I20260709 14:52:48.640244  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1110) took 1364 ms. Trace:
I20260709 14:52:48.640987  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:48.637344  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 8, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:52:48.642418  6207 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:48.643937  6207 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.644302  6207 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:48.643076  2433 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 7, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:48.642417  6202 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:48.566407  6011 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.647348  6202 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.649914  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.651196  2751 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:48.638478  6259 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:48.653937  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.652285  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.625954  2760 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:48.640708  2753 rpcz_store.cc:276] 0709 14:52:47.287890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:47.326309 (+ 38419us) service_pool.cc:224] Handling call
0709 14:52:47.327180 (+   871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.413828 (+1086648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:48.639974 (+226146us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540040ecd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:48.640180 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343496064}
I20260709 14:52:48.653199  2605 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:48.654635  2679 raft_consensus.cc:2376] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10. Re-sending same reply.
W20260709 14:52:48.673887  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.676061  2755 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:48.677701  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:48.679078  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55696us
I20260709 14:52:48.684487  2751 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 11
I20260709 14:52:48.575737  6079 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.617624  6203 raft_consensus.cc:2764] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:52:48.619033  6078 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:52:48.698971  2576 connection.cc:256] RPC call timeout handler was delayed by -1.3486701040000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
I20260709 14:52:48.699949  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.616415  6185 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:48.678474  2752 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:48.653430  6202 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:48.698644  6036 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:48.707005  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:48.709124  2682 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:48.702258  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:48.720795  2754 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:48.708720  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2865701040000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 14:52:48.722612  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1092701039999611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
I20260709 14:52:48.725267  2757 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:48.725544  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.726413  2605 raft_consensus.cc:2368] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:48.726519  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.727017  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.727384  2609 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.727828  3545 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.727996  2433 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:48.729660  6036 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:48.729739  2433 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 5, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:52:48.733685  2610 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:52:48.734422  2607 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:52:48.733460  2435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 165) took 30632 ms (client timeout 15000 ms). Trace:
I20260709 14:52:48.735564  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.736353  2686 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.750331  2433 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 6, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:48.752388  2609 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:48.752893  6079 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:48.735188  2435 rpcz_store.cc:269] 0709 14:52:18.100921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:18.117839 (+ 16918us) service_pool.cc:224] Handling call
0709 14:52:48.733160 (+30615321us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:52:35.755685 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:52:35.773311 (+ 17626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:52:35.773389 (+    78us) write_op.cc:435] Acquired schema lock
0709 14:52:35.773438 (+    49us) tablet.cc:662] Decoding operations
0709 14:52:36.201555 (+428117us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:52:36.266705 (+ 65150us) write_op.cc:641] Partition lock acquired for write op
0709 14:52:36.301715 (+ 35010us) tablet.cc:685] Acquiring locks for 15 operations
0709 14:52:36.656775 (+355060us) tablet.cc:701] Row locks acquired
0709 14:52:36.656841 (+    66us) write_op.cc:260] PREPARE: finished
0709 14:52:36.657125 (+   284us) write_op.cc:270] Start()
0709 14:52:36.657368 (+   243us) write_op.cc:276] Timestamp: P: 1783608756657034 usec, L: 0
0709 14:52:36.657407 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 14:52:36.668627 (+ 11220us) log.cc:844] Serialized 5969 byte log entry
0709 14:52:37.779279 (+1110652us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16b53c0ac 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b
0709 14:52:38.646461 (+867182us) op_driver.cc:464] REPLICATION: finished
0709 14:52:39.816705 (+1170244us) write_op.cc:301] APPLY: starting
0709 14:52:39.816846 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 14:52:39.817200 (+   354us) tablet.cc:1370] finished BulkCheckPresence
0709 14:52:39.817237 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:52:40.348463 (+531226us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:52:40.495553 (+147090us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 14:52:40.500364 (+  4811us) write_op.cc:312] APPLY: finished
0709 14:52:40.533687 (+ 33323us) log.cc:844] Serialized 106 byte log entry
0709 14:52:40.636874 (+103187us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:40.652711 (+ 15837us) write_op.cc:454] Released schema lock
0709 14:52:40.669173 (+ 16462us) write_op.cc:341] FINISH: Updating metrics
0709 14:52:40.775033 (+105860us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:40.783298 (+  8265us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9601,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":1058951,"apply.run_cpu_time_us":33465,"apply.run_wall_time_us":950929,"num_ops":15,"prepare.queue_time_us":487401,"prepare.run_cpu_time_us":75757,"prepare.run_wall_time_us":2263193,"raft.queue_time_us":303231,"raft.run_cpu_time_us":25215,"raft.run_wall_time_us":934525,"replication_time_us":1957979,"spinlock_wait_cycles":23874432,"thread_start_us":2715011,"threads_started":5,"wal-append.queue_time_us":1427933,"wal-append.run_cpu_time_us":65520,"wal-append.run_wall_time_us":2745853}]]}
I20260709 14:52:48.691886  6020 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:48.754388  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.011s (SENT)
I20260709 14:52:48.772162  2754 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:48.716095  5880 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:48.778612  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:48.778892  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:48.786229  2603 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.774021  2433 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:48.781749  2679 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:52:48.784071  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:48.788516  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.789650  2682 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:48.800056  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45711.2us
I20260709 14:52:48.788367  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.765936  6078 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:48.814945  3537 raft_consensus.cc:2368] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
W20260709 14:52:48.824271  2651 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.845s (SENT)
I20260709 14:52:48.722702  6097 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 14:52:48.831861  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.832986  2609 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:48.785226  5880 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:48.849702  2686 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:48.850083  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:48.858106  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.858865  3536 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:48.859246  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.860082  2681 raft_consensus.cc:2368] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9.
I20260709 14:52:48.860528  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.861110  6183 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:48.861635  6268 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.862726  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:48.862578  6207 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.863111  6182 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:48.864020  6182 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:48.864603  6182 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:48.865127  6207 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:48.865621  6231 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:48.815112  6016 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.866528  6185 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.866639  6231 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:48.867815  6231 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:48.868921  6231 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.870458  6231 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:48.871816  6059 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:48.872404  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.873382  3541 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:48.847927  6211 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:48.875514  6211 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:48.875965  6211 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:48.874554  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:48.877442  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:48.878216  3541 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:48.865160  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:48.879204  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 158648us
I20260709 14:52:48.880520  2682 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 14:52:48.882516  5877 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:48.883412  2679 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:48.884982  6227 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:48.885603  6227 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:48.886632  6227 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.889249  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:48.916188  2603 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:48.774950  6259 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 14:52:48.826401  6274 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:48.921928  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.922048  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.923163  2686 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:52:48.928066  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 125102us
I20260709 14:52:48.930590  3545 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:48.928344  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:48.853215  5880 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:48.943547  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.944506  6078 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:48.945097  6016 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:48.945600  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.936509  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:48.936092  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:48.947254  6182 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:48.947996  3547 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:48.948390  3538 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 14:52:48.938041  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55858.8us
I20260709 14:52:48.949151  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.949927  2723 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:48.928531  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.920887  6183 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:48.952070  2652 leader_election.cc:400] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:48.952477  2652 leader_election.cc:403] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:52:48.952308  6183 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:48.955204  6029 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:48.949975  3543 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:48.927069  6274 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:48.957765  2651 leader_election.cc:336] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.938s (SENT)
I20260709 14:52:48.936014  5877 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:48.961853  2575 leader_election.cc:400] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9.
I20260709 14:52:48.964624  2575 leader_election.cc:403] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 14:52:48.826472  6081 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:48.968577  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68324.3us
I20260709 14:52:48.969735  2686 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
W20260709 14:52:48.969873  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.973176  2433 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 7, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:52:48.976266  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 148892us
I20260709 14:52:48.979008  3545 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:48.920590  6036 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:48.983644  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47850.2us
I20260709 14:52:48.986100  3547 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 14:52:48.986487  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:48.962021  6274 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:48.996843  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.996801  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:48.999456  2751 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:49.001271  3538 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:52:49.003434  3542 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:49.004536  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46819.6us
I20260709 14:52:48.990635  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:49.006425  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:49.002678  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:49.012846  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.981153  5877 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:48.983093  6190 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:48.989096  6268 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:49.009403  2433 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:49.030738  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:49.051383  2577 leader_election.cc:336] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.632s (SENT)
I20260709 14:52:48.996752  6079 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 14:52:49.025203  6168 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:49.044584  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:49.068094  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:49.072280  6190 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:49.074522  6190 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:49.077615  3545 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:49.079465  3543 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:52:49.078610  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:49.080319  3541 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
I20260709 14:52:49.083490  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.084439  3538 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:49.087198  6268 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9.
W20260709 14:52:49.092772  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:49.035864  5793 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.737s	user 0.006s	sys 0.007s
W20260709 14:52:49.112744  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:48.998031  5718 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 14:52:49.139071  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:49.040297  6203 raft_consensus.cc:2764] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 14:52:49.193418  3547 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:49.066694  6168 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:49.110908  6096 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.006s
I20260709 14:52:49.196951  2433 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 5, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:49.179957  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:49.141769  2577 leader_election.cc:336] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.733s (SENT)
I20260709 14:52:49.132635  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:49.077487  5959 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 14:52:49.183959  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:49.127682  6202 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:49.213454  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.214514  3545 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:49.214684  2756 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:49.222201  2603 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:49.228799  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.079510  6190 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:49.229935  3543 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:49.230595  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 88897.6us
W20260709 14:52:49.238812  2577 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.745s (SENT)
I20260709 14:52:49.183084  6261 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:49.187676  6126 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:49.231644  2433 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:49.238951  3542 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:49.182693  6111 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:49.240067  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1252) took 1167 ms. Trace:
W20260709 14:52:49.236620  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 252089us
I20260709 14:52:49.253187  2604 rpcz_store.cc:276] 0709 14:52:48.073013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.481409 (+408396us) service_pool.cc:224] Handling call
0709 14:52:48.508680 (+ 27271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:48.515943 (+  7263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:48.889105 (+373162us) log.cc:844] Serialized 14 byte log entry
0709 14:52:49.239359 (+350254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:49.239455 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:49.239545 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:49.239656 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:49.239968 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5441,"prepare.queue_time_us":53921,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":5211,"thread_start_us":674271,"threads_started":2}
I20260709 14:52:49.260205  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.228190  6202 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:49.268003  3547 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:49.272696  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 631810us
I20260709 14:52:49.232046  5959 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:49.277745  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.280021  3543 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:49.248420  6190 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:49.283712  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 300277us
W20260709 14:52:49.285827  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:49.331548  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:49.284852  6260 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
W20260709 14:52:49.289773  6244 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.007s
I20260709 14:52:49.277207  6202 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:49.343745  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54979.9us
I20260709 14:52:49.339506  2433 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:52:49.336234  3542 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:49.274353  6126 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:49.288578  6029 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:49.271247  6111 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.333524  6081 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:49.356470  2433 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 2 to 4, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:49.261272  5877 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:49.367776  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84774.8us
I20260709 14:52:49.368353  6029 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.366803  2433 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 5, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:52:49.379705  6283 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 14:52:49.380647  6011 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:49.343837  6260 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:49.383215  3546 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:49.369100  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:49.407593  6126 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:49.410522  3538 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:49.412391  6308 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:49.414816  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 415195us
I20260709 14:52:49.364073  6287 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 14:52:49.421726  3542 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:49.422428  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53470.7us
I20260709 14:52:49.423862  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:49.425457  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:49.425709  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69640.3us
I20260709 14:52:49.426752  6097 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:49.427448  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:49.428568  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:49.434325  6231 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:49.435381  6231 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:49.438009  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:49.432564  6308 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:49.433342  2577 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.942s (SENT)
W20260709 14:52:49.406736  6272 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 14:52:49.482915  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44986.4us
I20260709 14:52:49.448922  6207 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:49.491367  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:49.455821  6097 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:49.485102  2684 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:52:49.384919  5880 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:49.512600  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 85287.4us
I20260709 14:52:49.494449  6207 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:49.507556  6097 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:49.533761  3537 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:49.449810  5959 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:49.509212  6260 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:49.575665  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:49.588168  6211 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.712s	user 0.016s	sys 0.000s
I20260709 14:52:49.588922  6211 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:49.589990  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 82686us
I20260709 14:52:49.591074  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:49.591507  6211 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:49.540783  6304 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:49.595269  6182 raft_consensus.cc:2764] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 14:52:49.596863  6203 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:49.547425  6097 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:49.546078  6207 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:49.616117  2577 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.185s (SENT)
W20260709 14:52:49.619918  6289 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
W20260709 14:52:49.633702  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.028s
W20260709 14:52:49.637722  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.007s	sys 0.028s
I20260709 14:52:49.634227  6203 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:49.634938  6296 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 14:52:49.636300  5880 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:49.645259  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 189037us
W20260709 14:52:49.650162  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86643.8us
W20260709 14:52:49.655870  6307 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:52:49.629770  6303 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:49.658047  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1244) took 1587 ms (client timeout 1791 ms). Trace:
I20260709 14:52:49.657714  6303 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:49.658530  2612 rpcz_store.cc:269] 0709 14:52:48.070213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.093758 (+ 23545us) service_pool.cc:224] Handling call
0709 14:52:49.657804 (+1564046us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54000ac154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:49.657976 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131779968}
I20260709 14:52:49.662062  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.662818  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.663602  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.666795  2754 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:49.666673  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:49.668964  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.669245  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.670811  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.671708  2752 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:49.671757  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77109.8us
I20260709 14:52:49.676256  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:49.675388  2651 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.119s (SENT)
I20260709 14:52:49.682189  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:49.702090  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:49.703925  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.704277  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.705014  2753 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:49.705040  2754 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:49.660040  6203 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.680543  6211 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:49.718474  6126 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 14:52:49.719895  6303 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:49.723448  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:49.726965  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45281.8us
I20260709 14:52:49.727586  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1246) took 1656 ms. Trace:
I20260709 14:52:49.728070  2611 rpcz_store.cc:276] 0709 14:52:48.070935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.467277 (+396342us) service_pool.cc:224] Handling call
0709 14:52:48.493582 (+ 26305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:48.520745 (+ 27163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:48.520781 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:48.588946 (+ 68165us) log.cc:844] Serialized 52 byte log entry
0709 14:52:49.135572 (+546626us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:49.135646 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:49.135749 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:49.727310 (+591561us) raft_consensus.cc:1685] finished
0709 14:52:49.727336 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:49.727515 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13024,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":101,"thread_start_us":480003,"threads_started":1,"wal-append.queue_time_us":731133}
I20260709 14:52:49.732427  2751 raft_consensus.cc:2376] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9. Re-sending same reply.
I20260709 14:52:49.732703  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.733739  2754 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:49.715124  6211 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:49.722581  6126 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:49.716617  6305 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:49.738476  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63171.4us
I20260709 14:52:49.743647  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.737355  6211 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:49.761636  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.762531  2754 raft_consensus.cc:2376] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11. Re-sending same reply.
W20260709 14:52:49.751824  6298 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
I20260709 14:52:49.764765  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.769269  2680 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:49.775508  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.779284  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.783809  6203 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:49.784731  6203 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:49.785642  6203 consensus_queue.cc:1243] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:49.788916  6062 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:49.789137  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.789538  6062 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:49.789707  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.789960  2685 raft_consensus.cc:2368] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:49.791994  2685 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:49.792418  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.793566  2682 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 14:52:49.760244  6294 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
I20260709 14:52:49.794924  6315 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 14:52:49.796185  6305 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:49.796705  6305 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:49.797189  6314 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 14:52:49.797717  6305 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.799417  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.799875  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.800601  2678 raft_consensus.cc:2368] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:52:49.800872  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.743937  6126 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:49.801637  2680 raft_consensus.cc:2368] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12.
I20260709 14:52:49.803349  6126 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:49.783927  6310 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 14:52:49.810214  6274 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.785s	user 0.014s	sys 0.000s
I20260709 14:52:49.811803  6274 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:49.812821  2577 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.399s (SENT)
I20260709 14:52:49.819340  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:49.794927  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1118) took 1748 ms. Trace:
I20260709 14:52:49.820384  2756 rpcz_store.cc:276] 0709 14:52:48.046534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.065391 (+ 18857us) service_pool.cc:224] Handling call
0709 14:52:49.794637 (+1729246us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:49.794802 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087184128}
I20260709 14:52:49.821080  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:49.814062  6316 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.126s	user 0.003s	sys 0.000s
I20260709 14:52:49.815500  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 628) took 1166 ms. Trace:
I20260709 14:52:49.827150  2758 rpcz_store.cc:276] 0709 14:52:48.648723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.659329 (+ 10606us) service_pool.cc:224] Handling call
0709 14:52:48.659944 (+   615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:48.779632 (+119688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:48.779679 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:48.798087 (+ 18408us) log.cc:844] Serialized 26 byte log entry
0709 14:52:49.268547 (+470460us) raft_consensus.cc:1645] Marking committed up to 2
0709 14:52:49.268590 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:49.268670 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:49.815175 (+546505us) raft_consensus.cc:1685] finished
0709 14:52:49.815199 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:49.815422 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14694,"raft.queue_time_us":20359,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":10685,"thread_start_us":413585,"threads_started":1,"wal-append.queue_time_us":659819}
I20260709 14:52:49.831244  6261 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:49.831648  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:49.833264  6317 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
I20260709 14:52:49.835985  3545 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:49.827165  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:49.841856  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.843053  2758 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:49.849025  2753 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 11
W20260709 14:52:49.855118  6190 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.007s
I20260709 14:52:49.886991  6190 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.888629  6207 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:49.890918  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:49.850935  6062 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.894289  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.856629  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.857612  2678 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
W20260709 14:52:49.897039  2651 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.510s (SENT)
I20260709 14:52:49.858098  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.859619  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.859863  6274 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:49.859951  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.860092  6305 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:49.861088  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.863065  2752 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:52:49.863128  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:49.868223  6202 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.006s
W20260709 14:52:49.904031  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52440.7us
I20260709 14:52:49.857493  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.904752  2760 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:49.906106  2685 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:49.876624  2724 connection.cc:256] RPC call timeout handler was delayed by -1.0652707699999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 14:52:49.877259  6097 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:49.877612  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:49.878484  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.879302  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.909060  3543 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:49.880162  6301 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 14:52:49.882272  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:49.884835  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:49.870592  2754 pending_rounds.cc:85] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:52:49.912554  2754 pending_rounds.cc:107] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 14:52:49.912914  2754 raft_consensus.cc:2889] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:49.895179  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:49.905062  2680 raft_consensus.cc:2368] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
W20260709 14:52:49.873565  6309 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 14:52:49.925742  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.926608  2678 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:49.926976  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:49.928174  6190 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:49.892699  6315 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:49.929704  6315 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:49.892725  6062 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:49.932466  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:49.921023  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:49.921793  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.922540  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.908977  6202 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:49.923430  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.924045  2723 leader_election.cc:400] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:49.929953  2681 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:49.914577  2724 leader_election.cc:400] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:49.934180  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:49.940511  2724 leader_election.cc:403] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 14:52:49.940776  2757 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 13
W20260709 14:52:49.920634  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 3.303s (SENT)
I20260709 14:52:49.936617  2752 raft_consensus.cc:2376] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11. Re-sending same reply.
W20260709 14:52:49.939932  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156951us
I20260709 14:52:49.942416  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:49.943243  3543 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:49.946471  3536 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:49.942135  5788 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:49.948396  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.958302  2680 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:49.952510  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:49.962778  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:49.964159  2679 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:49.973457  2685 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: []
I20260709 14:52:49.947491  6274 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:49.975386  6274 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:49.977846  6274 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:49.979080  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 67311.6us
I20260709 14:52:49.980635  2759 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:49.982139  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61686.1us
W20260709 14:52:49.982930  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0223553700000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 14:52:49.985020  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:49.988197  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:49.978411  6211 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:49.994082  6211 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:49.958503  6207 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:49.985198  6183 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.015s	sys 0.009s
W20260709 14:52:50.006124  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 95155.1us
I20260709 14:52:49.981557  6261 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:50.007840  6183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.015s	sys 0.010s
W20260709 14:52:50.017692  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46080.6us
I20260709 14:52:50.010129  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.012913  2753 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:50.010143  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.015285  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:49.988965  6323 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:50.023280  2752 raft_consensus.cc:2368] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:50.028148  2757 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:52:50.030260  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:50.042532  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.020416  6261 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:49.986424  5788 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.067404  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.068681  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.069823  2755 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.066444  3541 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.071394  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.072528  2760 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.075685  2753 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:50.097906  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:50.106413  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70155.1us
I20260709 14:52:50.108007  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:50.110424  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:50.121302  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48288.7us
W20260709 14:52:50.123525  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 225786us
W20260709 14:52:50.124817  2651 leader_election.cc:336] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.554s (SENT)
I20260709 14:52:50.109978  6323 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 14:52:50.072062  6261 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:50.032414  6315 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 14:52:50.134307  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 112935us
I20260709 14:52:50.031281  6211 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 14:52:50.135486  6074 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:50.137701  6324 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:50.138193  6074 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:50.138522  6323 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:50.139207  2723 leader_election.cc:400] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:52:50.139374  6323 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:50.139675  2723 leader_election.cc:403] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:52:50.140014  6323 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.141090  2723 leader_election.cc:400] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12.
I20260709 14:52:50.141474  2723 leader_election.cc:403] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 14:52:50.143085  6323 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:50.143615  6097 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:50.143920  6323 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:50.144526  6323 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:50.145814  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1040) took 4738 ms (client timeout 3248 ms). Trace:
I20260709 14:52:50.145584  6323 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:50.152740  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.155025  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:50.155081  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 966) took 1284 ms. Trace:
I20260709 14:52:50.155944  3537 rpcz_store.cc:276] 0709 14:52:48.870347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.870523 (+   176us) service_pool.cc:224] Handling call
0709 14:52:48.870948 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:49.541151 (+670203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:50.154749 (+613598us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:50.154981 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509011328}
W20260709 14:52:50.146327  2684 rpcz_store.cc:269] 0709 14:52:45.407830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:45.622706 (+214876us) service_pool.cc:224] Handling call
0709 14:52:50.145543 (+4522837us) spinlock_profiling.cc:243] Waited 3.82 s on lock 0x7b540024e5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:50.145700 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8777598336}
I20260709 14:52:50.120110  5959 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:50.163630  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.169512  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.151333  6336 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:52:50.136494  6315 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:50.176234  3541 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.175843  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.177625  3540 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:50.177217  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 214367us
I20260709 14:52:50.128497  6020 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:50.178855  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.183696  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:50.183883  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.898s (SENT)
I20260709 14:52:50.170043  6274 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:50.207422  3537 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.194504  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.195204  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.199391  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1260) took 1155 ms. Trace:
I20260709 14:52:50.209797  2603 rpcz_store.cc:276] 0709 14:52:49.043928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.078278 (+ 34350us) service_pool.cc:224] Handling call
0709 14:52:50.199138 (+1120860us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b54001ebc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:50.199284 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126163712}
I20260709 14:52:50.162581  6324 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12.
I20260709 14:52:50.202715  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.214902  2760 raft_consensus.cc:2368] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:50.220384  3538 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
W20260709 14:52:50.221216  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 190720us
W20260709 14:52:50.222851  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.991s (SENT)
W20260709 14:52:50.223609  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.125s (SENT)
I20260709 14:52:50.227202  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.228492  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.230469  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.231485  2755 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:50.235798  3540 raft_consensus.cc:2368] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
W20260709 14:52:50.236126  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 248436us
W20260709 14:52:50.235916  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 111163us
W20260709 14:52:50.239221  2651 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.647s (SENT)
I20260709 14:52:50.239581  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.169308  5877 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 14:52:50.241705  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.246330  2755 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.217123  6274 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:50.257252  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:50.259888  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.261154  2755 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:50.218108  6020 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:50.261039  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1056) took 1766 ms (client timeout 1915 ms). Trace:
W20260709 14:52:50.265060  2677 rpcz_store.cc:269] 0709 14:52:48.494919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.500494 (+  5575us) service_pool.cc:224] Handling call
0709 14:52:50.256247 (+1755753us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:50.260947 (+  4700us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905228544}
I20260709 14:52:50.230585  6074 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:50.244931  6183 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:50.269863  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.227789  6202 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:50.230780  6211 consensus_queue.cc:1243] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:50.271656  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.244490  5959 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:50.274331  2755 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:52:50.271517  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.276980  3537 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.260521  6274 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:50.278628  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:50.280670  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.290355  3537 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.293596  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:50.282939  2651 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.283s (SENT)
W20260709 14:52:50.287619  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103780us
I20260709 14:52:50.291972  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.300624  3537 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:50.305313  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:50.294668  2755 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8. Re-sending same reply.
I20260709 14:52:50.273701  6333 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:50.308586  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.309861  2755 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:50.331961  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.333083  2603 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11. Re-sending same reply.
W20260709 14:52:50.374053  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 102495us
I20260709 14:52:50.276654  6268 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:50.315636  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:50.376116  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70874.4us
I20260709 14:52:50.381496  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.387840  3537 raft_consensus.cc:2368] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:52:50.391386  6268 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:50.392661  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.392886  6268 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:50.393456  2604 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:52:50.394909  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.395323  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.395887  2607 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:50.397037  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:50.399667  6331 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 14:52:50.402535  6207 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:50.402946  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.402913  6333 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:50.405558  6333 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:50.307576  6260 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:50.406896  6260 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:50.407965  6260 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:50.410972  5788 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:50.410899  3537 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.412180  6111 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:50.382447  6338 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:50.417995  6318 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.730s	user 0.005s	sys 0.009s
I20260709 14:52:50.403312  6207 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:50.422693  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 165668us
I20260709 14:52:50.421581  2752 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:50.428516  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 974) took 1514 ms. Trace:
I20260709 14:52:50.418169  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.419283  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.429211  3546 rpcz_store.cc:276] 0709 14:52:48.873379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:48.945108 (+ 71729us) service_pool.cc:224] Handling call
0709 14:52:50.387494 (+1442386us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54004b2dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:50.387675 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004121856}
I20260709 14:52:50.431550  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.437173  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.437975  3538 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:50.438073  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:50.496753  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.498755  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.501047  3537 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:50.502885  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 219963us
W20260709 14:52:50.509009  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71007.7us
I20260709 14:52:50.510238  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.515259  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.516651  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.520804  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.521243  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.521914  2754 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:52:50.523702  3537 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.436269  5788 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.525235  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.523962  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:50.524718  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:50.528339  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106352us
I20260709 14:52:50.543262  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.544899  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:50.500391  6062 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:52:50.536329  5880 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:50.512116  6333 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:50.548228  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.548300  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.561213  3537 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.561329  2753 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.549357  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1124) took 1465 ms. Trace:
I20260709 14:52:50.562327  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.563436  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.561945  3547 rpcz_store.cc:276] 0709 14:52:49.084122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.284815 (+200693us) service_pool.cc:224] Handling call
0709 14:52:49.337493 (+ 52678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:49.366631 (+ 29138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:49.366671 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:49.445332 (+ 78661us) log.cc:844] Serialized 26 byte log entry
0709 14:52:49.613267 (+167935us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:49.641130 (+ 27863us) log.cc:844] Serialized 14 byte log entry
0709 14:52:49.653720 (+ 12590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:49.653968 (+   248us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:50.417393 (+763425us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:50.540054 (+122661us) raft_consensus.cc:1685] finished
0709 14:52:50.548984 (+  8930us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:50.549241 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10614,"prepare.queue_time_us":19683,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":61,"raft.queue_time_us":29607,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":42227,"spinlock_wait_cycles":21321984,"thread_start_us":118019,"threads_started":1,"wal-append.queue_time_us":204994}
I20260709 14:52:50.556483  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.557564  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:50.532127  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 231897us
I20260709 14:52:50.565503  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.566588  3538 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.564424  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.587723  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.589097  2603 raft_consensus.cc:2368] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:52:50.592072  3537 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.560734  6333 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:50.592996  6333 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:50.593593  3537 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:50.570523  5880 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:50.598167  5880 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:50.599539  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.550635  5788 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:50.601966  5788 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:50.602959  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.604524  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.605319  2753 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:50.606217  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.570489  6074 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 14:52:50.607376  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.608410  2609 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:50.607476  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.611157  2606 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:50.611433  3546 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:52:50.614529  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.615928  3547 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 14:52:50.617956  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 73697.1us
I20260709 14:52:50.619000  2753 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.619347  2723 leader_election.cc:400] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10.
I20260709 14:52:50.560616  6111 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.623238  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.623669  2755 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:50.624171  3537 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.628738  3538 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:52:50.621035  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:50.639026  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 3.271s (SENT)
I20260709 14:52:50.639714  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.638324  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.642652  2755 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:50.648696  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94048.4us
I20260709 14:52:50.651319  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.622112  6074 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:50.661432  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.674247  3538 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:50.677875  2606 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:50.674063  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.681063  2753 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.641111  6202 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:50.684427  2609 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:50.687347  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.688619  3547 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.690235  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:50.691385  2755 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:50.647116  6062 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.696512  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:50.703606  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76100.8us
I20260709 14:52:50.710197  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.711110  2603 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:50.713016  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1231) took 1055 ms. Trace:
I20260709 14:52:50.662706  6183 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:50.713505  2751 rpcz_store.cc:276] 0709 14:52:49.657712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.662515 (+  4803us) service_pool.cc:224] Handling call
0709 14:52:50.712789 (+1050274us) spinlock_profiling.cc:243] Waited 66 ms on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:50.712938 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151716992}
I20260709 14:52:50.676335  6244 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.678130  6074 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:50.716836  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.726279  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:50.686580  6202 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:50.640938  6111 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:50.744294  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1129) took 1302 ms. Trace:
I20260709 14:52:50.726949  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.737767  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.640506  6207 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.751955  2751 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:50.751868  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 239050us
I20260709 14:52:50.744837  3542 rpcz_store.cc:276] 0709 14:52:49.442098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.720206 (+1278108us) service_pool.cc:224] Handling call
0709 14:52:50.743692 (+ 23486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:50.743936 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:50.743962 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:50.744010 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:50.744049 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:50.744107 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:50.744228 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:50.791764  2751 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:52:50.745015  2755 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:50.799129  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.800365  3541 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.801705  2755 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:50.739771  6202 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:50.804438  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.813654  3536 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:50.805941  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:50.828048  2751 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
W20260709 14:52:50.831823  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:50.832098  2753 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:50.832252  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106114us
I20260709 14:52:50.716441  6062 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.838003  2755 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:50.839448  3547 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:50.797776  6305 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:50.728613  6244 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.803948  6323 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.847774  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:50.853240  6260 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.859668  6260 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:50.861291  6031 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:50.855755  6011 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:50.862995  6011 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:50.863657  6286 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.516s	user 0.003s	sys 0.010s
I20260709 14:52:50.865639  6011 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:50.868196  6111 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:50.767014  6183 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.086s
W20260709 14:52:50.879933  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46517.9us
W20260709 14:52:50.886582  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 225039us
I20260709 14:52:50.887744  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.888592  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:50.907274  2603 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:52:50.909319  2603 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:50.911810  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.914769  3536 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:50.916524  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.862164  6031 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:50.920008  2651 leader_election.cc:400] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:50.928150  3536 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:50.930531  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.934561  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:50.935024  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.935680  2609 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:50.919894  6305 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:50.884171  6323 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:50.941700  2726 leader_election.cc:400] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:52:50.852877  6074 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:50.933606  6020 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.002s
I20260709 14:52:50.954011  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.960619  3536 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.948293  2726 leader_election.cc:403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:52:50.949465  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.964041  2606 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:52:50.942870  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:50.971728  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 332718us
I20260709 14:52:50.974709  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.943514  6203 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:50.981323  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:50.981941  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.982612  3540 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:50.982661  2609 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:50.987174  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:50.987342  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:50.988341  2606 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:50.991394  2609 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:50.998297  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:50.980089  6305 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:52:51.017057  2606 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:50.985811  6344 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:50.962375  6020 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:51.020962  6062 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.024382  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:51.029253  6260 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:51.031566  6323 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.032507  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:51.046054  6261 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.874s	user 0.011s	sys 0.002s
I20260709 14:52:51.004699  6338 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.047271  6338 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:51.048380  6327 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.849s	user 0.003s	sys 0.009s
I20260709 14:52:51.049870  6338 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:51.051209  6031 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:51.052440  6350 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 14:52:51.052778  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1250) took 1191 ms. Trace:
I20260709 14:52:51.053191  2756 rpcz_store.cc:276] 0709 14:52:49.861679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.861863 (+   184us) service_pool.cc:224] Handling call
0709 14:52:49.862380 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:49.862850 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:49.862878 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:49.863537 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 14:52:50.125771 (+262234us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:50.125839 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:50.125962 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:51.047678 (+921716us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:51.052419 (+  4741us) raft_consensus.cc:1685] finished
0709 14:52:51.052447 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:51.052711 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28246,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":160,"spinlock_wait_cycles":34863488,"thread_start_us":261600,"threads_started":1,"wal-append.queue_time_us":288045}
I20260709 14:52:51.054934  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 824) took 1169 ms. Trace:
I20260709 14:52:51.055440  3546 rpcz_store.cc:276] 0709 14:52:49.885488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.526402 (+640914us) service_pool.cc:224] Handling call
0709 14:52:51.054635 (+528233us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b5400200dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:51.054805 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106752768}
W20260709 14:52:51.055976  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1121) took 1972 ms (client timeout 1795 ms). Trace:
W20260709 14:52:51.056511  3545 rpcz_store.cc:269] 0709 14:52:49.083309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.237642 (+154333us) service_pool.cc:224] Handling call
0709 14:52:51.055730 (+1818088us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54002baf54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:51.055896 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1270108416}
I20260709 14:52:51.058053  3541 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 14:52:51.058559  3541 pending_rounds.cc:85] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:52:51.058820  3541 pending_rounds.cc:107] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 14:52:51.060367  3541 raft_consensus.cc:2889] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:52:51.061414  6325 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.869s	user 0.002s	sys 0.006s
I20260709 14:52:51.064816  6020 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:51.065869  2611 raft_consensus.cc:2368] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9.
I20260709 14:52:51.066221  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1156) took 1051 ms. Trace:
I20260709 14:52:51.066645  2752 rpcz_store.cc:276] 0709 14:52:50.015182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.187271 (+172089us) service_pool.cc:224] Handling call
0709 14:52:51.065999 (+878728us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54000e6d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:51.066144 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444831104}
I20260709 14:52:51.068619  6244 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:51.069494  6349 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 14:52:51.072243  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74066.3us
I20260709 14:52:51.073664  2576 leader_election.cc:400] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:52:51.074165  2576 leader_election.cc:403] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:52:51.023548  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.078240  3540 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 14:52:51.018703  6345 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
W20260709 14:52:51.079726  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 305204us
I20260709 14:52:51.085995  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:51.086274  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:51.078327  6347 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.081187  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.098016  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.099973  6347 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.100553  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:51.101190  6355 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:52:51.101478  2609 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:51.107313  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:51.108212  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:51.113390  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:51.114114  2752 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:51.109741  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:51.072074  6353 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:51.116197  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.118362  2576 leader_election.cc:400] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:52:51.120780  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.121588  3545 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:51.113183  6352 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:52:51.119261  6353 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:51.124889  3540 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:51.126143  2756 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:51.126466  3536 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:51.125936  3543 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:51.046675  6261 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.130445  2606 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 14:52:51.128372  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:51.129701  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.125254  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.138962  2609 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:51.102674  6020 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:51.110553  6338 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:52:51.125551  6333 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.007s
I20260709 14:52:51.146724  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.159607  3538 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:51.090021  6244 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 14:52:51.167369  2612 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:51.130666  6323 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:51.095247  6031 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 14:52:51.141270  6346 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 14:52:51.174683  5877 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.047s	sys 0.127s
W20260709 14:52:51.175132  5877 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.047s	sys 0.127s
I20260709 14:52:51.175477  6338 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:51.174583  6182 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 14:52:51.181751  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:51.151400  6344 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:51.148699  6111 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:51.141819  6354 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:51.184504  6182 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:51.195508  6182 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:51.197067  6182 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:52:51.197293  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.167330  6011 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.127234  6353 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:51.199326  6011 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.200060  6353 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:51.200846  6261 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.183861  6343 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 14:52:51.201692  6261 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:51.192313  6344 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:51.202418  6344 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:51.203693  6031 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:51.190575  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.194342  6354 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.150967  6351 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.147531  6203 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 14:52:51.172888  6355 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:51.168625  6347 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.207402  3545 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:52:51.212774  2606 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 14:52:51.221839  2609 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 14:52:51.166602  6333 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:51.177624  6305 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 14:52:51.249111  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.211560  6355 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:51.274184  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:51.275250  2606 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:51.281693  3538 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 14:52:51.283494  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.286670  2607 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:51.263568  6062 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:51.341794  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:51.211149  6351 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:51.351356  2603 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:51.286453  6203 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:51.280473  6355 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:51.362931  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 272979us
I20260709 14:52:51.366978  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:51.371968  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.211393  6354 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:51.377408  2760 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.019s	sys 0.002s
W20260709 14:52:51.379390  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 448999us
W20260709 14:52:51.381071  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 255888us
I20260709 14:52:51.382020  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.382148  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:51.384094  2760 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.019s	sys 0.002s
I20260709 14:52:51.388769  2606 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:52:51.390729  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1153) took 1376 ms. Trace:
I20260709 14:52:51.391467  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1230) took 1734 ms. Trace:
I20260709 14:52:51.481154  2759 rpcz_store.cc:276] 0709 14:52:49.657331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.668569 (+ 11238us) service_pool.cc:224] Handling call
0709 14:52:51.391202 (+1722633us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:51.391406 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695622528}
I20260709 14:52:51.391585  2754 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:51.370042  6352 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:51.401379  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.406088  6354 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:51.350030  6062 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:51.362067  6203 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:51.456094  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.364089  6111 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
W20260709 14:52:51.463351  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 291156us
I20260709 14:52:51.390142  6305 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.414688  6344 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:51.466303  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 306608us
I20260709 14:52:51.496174  3545 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:51.483465  3541 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7. Re-sending same reply.
W20260709 14:52:51.521044  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72853.8us
I20260709 14:52:51.529978  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:51.531145  6363 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:51.531828  6363 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:51.532262  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.479058  2760 rpcz_store.cc:276] 0709 14:52:50.014358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.419560 (+405202us) service_pool.cc:224] Handling call
0709 14:52:50.419950 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:50.420423 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:50.420454 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:50.421013 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 14:52:50.696227 (+275214us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:51.384716 (+688489us) log.cc:844] Serialized 14 byte log entry
0709 14:52:51.390308 (+  5592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:51.390412 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:51.390457 (+    45us) raft_consensus.cc:1685] finished
0709 14:52:51.390473 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:51.390656 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5244,"prepare.queue_time_us":334007,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":4634,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5649,"thread_start_us":943302,"threads_started":2,"wal-append.queue_time_us":417915}
I20260709 14:52:51.465953  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:51.532806  6363 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:51.533686  2603 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:51.535648  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.536701  3541 raft_consensus.cc:2376] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11. Re-sending same reply.
I20260709 14:52:51.537333  6062 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:51.537922  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.542973  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.573057  3536 raft_consensus.cc:2376] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11. Re-sending same reply.
I20260709 14:52:51.535463  6203 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.526703  6305 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:51.526863  6274 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.220s	user 0.017s	sys 0.000s
I20260709 14:52:51.566324  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.591593  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:51.569070  2752 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:51.592465  3545 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:51.545811  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:51.593297  2603 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:51.539989  6347 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:51.540650  6360 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:52:51.570618  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 417490us
I20260709 14:52:51.551249  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.524209  6348 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:51.636236  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 267230us
W20260709 14:52:51.646998  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.016s	sys 0.050s
I20260709 14:52:51.535471  6352 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:51.647468  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.017s	sys 0.050s
I20260709 14:52:51.647871  6360 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:51.646672  6323 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:51.651151  2610 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:51.651216  2603 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:51.652808  2603 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:51.595960  2424 maintenance_manager.cc:419] P 3f6dfe8cfa964684b547b44a4b308ca2: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033392
I20260709 14:52:51.656339  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:51.657683  3536 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:51.663798  2603 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.664625  6351 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:51.665354  2603 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:51.665832  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114704us
I20260709 14:52:51.667223  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.616650  6356 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:51.668859  2686 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:52:51.669176  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:51.669781  2678 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:51.672906  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:51.680269  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.636384  6347 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:51.685926  2682 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:51.688143  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.688921  3547 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 14:52:51.688872  2686 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:51.690387  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.653082  6333 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:51.662515  6203 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:51.637916  6274 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:51.711808  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 329827us
I20260709 14:52:51.710042  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:51.716537  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.717319  2679 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:51.659665  6382 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:51.726722  2603 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 14:52:51.727723  2603 pending_rounds.cc:85] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:52:51.728178  2603 pending_rounds.cc:107] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 14:52:51.728636  2603 raft_consensus.cc:2889] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:51.730777  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:51.736274  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 240139us
I20260709 14:52:51.740799  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:51.744556  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.000s	sys 0.017s
W20260709 14:52:51.745471  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.000s	sys 0.017s
W20260709 14:52:51.747901  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.716163  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:51.645514  6362 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 14:52:51.693163  6347 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:51.755975  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:51.644474  6366 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 14:52:51.783648  2677 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:52:51.758948  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:51.785415  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 118351us
I20260709 14:52:51.786412  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.793751  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.644006  6348 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:51.708786  6305 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.788851  2608 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12.
I20260709 14:52:51.797159  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.804292  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:51.755838  6062 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:51.805888  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.812165  2684 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:51.812534  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133823us
I20260709 14:52:51.817902  3541 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:51.793000  6358 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:52:51.830000  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43679.3us
W20260709 14:52:51.824819  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103287us
I20260709 14:52:51.831185  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.825911  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:51.832999  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.834215  2685 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:51.840057  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:51.840603  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81816.9us
I20260709 14:52:51.842175  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.848433  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:51.814291  6385 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:51.858400  6344 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.861253  6367 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:51.864315  3536 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:51.835470  6097 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:51.865454  6097 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:51.866819  6097 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:51.868564  6097 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:51.872942  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.843209  6351 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:52:51.888904  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48721.5us
I20260709 14:52:51.846536  6333 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:51.816946  6062 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:51.896195  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47913us
I20260709 14:52:51.897848  2678 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:52:51.903527  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:52:51.906250  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73313.5us
I20260709 14:52:51.907631  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:51.909406  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.850593  6392 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 14:52:51.883658  6386 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.892534  6354 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:51.910719  6364 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
W20260709 14:52:51.927274  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 85253.7us
I20260709 14:52:51.927340  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:51.933662  3542 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:51.913520  6305 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:52:51.911449  6396 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:51.937172  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:51.942350  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:51.918040  6385 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:51.944664  6274 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:51.897733  6358 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 14:52:51.942986  5877 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 14:52:51.943342  6398 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:52:51.955387  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60117.8us
I20260709 14:52:51.932652  6372 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:51.969023  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1258) took 1363 ms. Trace:
W20260709 14:52:51.974772  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150025us
I20260709 14:52:51.983390  2605 rpcz_store.cc:276] 0709 14:52:50.605093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.617206 (+ 12113us) service_pool.cc:224] Handling call
0709 14:52:50.647694 (+ 30488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:50.910247 (+262553us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:51.267984 (+357737us) log.cc:844] Serialized 14 byte log entry
0709 14:52:51.785223 (+517239us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:51.800082 (+ 14859us) log.cc:844] Serialized 26 byte log entry
0709 14:52:51.800272 (+   190us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:51.800338 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:51.800451 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:51.968599 (+168148us) raft_consensus.cc:1685] finished
0709 14:52:51.968634 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:51.968910 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143391,"prepare.queue_time_us":611275,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":474928,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":19919,"spinlock_wait_cycles":12413696,"thread_start_us":871463,"threads_started":3,"wal-append.queue_time_us":515012}
I20260709 14:52:51.976986  6388 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
W20260709 14:52:52.003752  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 162692us
I20260709 14:52:52.006479  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:52.012925  2726 leader_election.cc:336] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.855s (SENT)
I20260709 14:52:52.013849  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:51.933828  6386 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.961467  6376 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.020689  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:52.021790  6261 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:52.028541  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.022397  6261 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:52.032140  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.032624  2685 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 14:52:52.033759  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.035140  6348 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:52.038411  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1254) took 1435 ms. Trace:
I20260709 14:52:52.039104  2604 rpcz_store.cc:276] 0709 14:52:50.603356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.622449 (+ 19093us) service_pool.cc:224] Handling call
0709 14:52:50.646359 (+ 23910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:50.939920 (+293561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:50.939975 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:50.954619 (+ 14644us) log.cc:844] Serialized 26 byte log entry
0709 14:52:51.277939 (+323320us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:51.564951 (+287012us) log.cc:844] Serialized 14 byte log entry
0709 14:52:51.588996 (+ 24045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:51.589152 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:52.038088 (+448936us) raft_consensus.cc:1685] finished
0709 14:52:52.038118 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:52.038303 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31702,"prepare.queue_time_us":261209,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":433634,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":60548,"thread_start_us":824059,"threads_started":3,"wal-append.queue_time_us":496868}
I20260709 14:52:52.042901  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.045161  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.057616  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.032249  6261 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:52.108026  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:51.995693  6359 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 14:52:51.952154  6396 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.026438  6333 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 14:52:52.114423  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:52.113756  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 126442us
I20260709 14:52:52.127017  2605 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:52.043632  6182 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:52.057113  6244 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:52.046720  6207 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 14:52:52.062089  6062 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:52.047153  6372 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:52.115483  6202 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.238s	user 0.018s	sys 0.014s
I20260709 14:52:52.063918  6031 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:52.048161  6408 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:52.061570  6348 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:52.168819  6348 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:52.169446  6348 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:52.169500  6375 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.170308  6375 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.173046  6375 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:52.173794  6405 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 14:52:52.149410  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:52.175282  6396 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:52.176774  6403 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:52:52.177487  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:52.178174  2752 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:52.183885  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.049363  6305 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:52.125655  6358 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:52.161726  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 148719us
I20260709 14:52:52.026417  6376 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.168087  6408 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:52.110667  6360 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 14:52:52.214686  6413 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:52.182715  6182 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:52.176839  6396 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:52.186131  3411 connection.cc:256] RPC call timeout handler was delayed by -1.6160865929999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 14:52:52.240494  3411 connection.cc:256] RPC call timeout handler was delayed by -1.5909351509999397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 14:52:52.241745  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.242496  2605 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:52:52.223315  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 95559.5us
I20260709 14:52:52.246536  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:52.229545  2726 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.052s (SENT)
I20260709 14:52:52.247897  2757 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:52.215816  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 158287us
I20260709 14:52:52.234583  6409 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.237197  6413 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.216979  2752 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:52.248184  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.234738  6074 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:52.248962  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.241745  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.264374  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:52.272435  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:52.274085  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:52.275223  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45723.2us
W20260709 14:52:52.276271  2726 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.060s (SENT)
I20260709 14:52:52.277279  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:52.278345  2756 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:52.279644  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:52.167433  6202 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.291433  2752 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:52.229317  6408 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:52.299814  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50961.8us
W20260709 14:52:52.263231  6355 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.889s	user 0.014s	sys 0.004s
I20260709 14:52:52.298457  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.240582  6182 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:52.300979  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.305333  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.306380  6416 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.307065  6416 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.306684  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.310801  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.191386  6372 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.309441  6062 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.316642  6062 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.318635  2758 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:52.293015  6333 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:52.324987  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.329180  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.329708  3546 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:52.300680  6418 raft_consensus.cc:476] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Not starting pre-election -- already a leader
I20260709 14:52:52.333247  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.333958  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1143) took 1213 ms. Trace:
I20260709 14:52:52.309209  6182 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.348562  3547 rpcz_store.cc:276] 0709 14:52:51.120770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.196946 (+ 76176us) service_pool.cc:224] Handling call
0709 14:52:52.333714 (+1136768us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.333896 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128240384}
I20260709 14:52:52.353705  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.334065  2605 raft_consensus.cc:2376] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13. Re-sending same reply.
I20260709 14:52:52.337090  2756 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:52.330896  6031 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:52.314294  6355 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:52.339459  6421 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:52:52.347102  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1157) took 2331 ms. Trace:
W20260709 14:52:52.328482  6347 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.011s
I20260709 14:52:52.328917  6207 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:52.227605  6358 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:52.368333  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:52.367713  2604 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:52.366168  2754 rpcz_store.cc:276] 0709 14:52:50.015438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.544528 (+529090us) service_pool.cc:224] Handling call
0709 14:52:51.391368 (+846840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:51.497652 (+106284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:52.346713 (+849061us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54000d0a50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.346995 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1914993408}
W20260709 14:52:52.379943  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46813.2us
I20260709 14:52:52.379943  2606 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:52.361435  6396 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:52.262323  6409 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.348621  6333 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:52.398185  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.404973  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.406046  2605 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:52:52.375332  6207 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:52.410940  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:52.414718  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113833us
I20260709 14:52:52.415642  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:52.398751  6421 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:52.325784  6305 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.398234  6418 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:52:52.390095  6386 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:52.377017  6347 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.423013  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.411871  6168 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.412169  6207 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:52.436857  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1245) took 2057 ms (client timeout 1855 ms). Trace:
W20260709 14:52:52.437641  2611 rpcz_store.cc:269] 0709 14:52:50.379825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.396692 (+ 16867us) service_pool.cc:224] Handling call
0709 14:52:52.436562 (+2039870us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.436761 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535883648}
I20260709 14:52:52.450683  2603 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:52.454171  2611 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:52.456537  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.453341  6418 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 14:52:52.505682  6418 consensus_queue.cc:1243] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 14:52:52.510047  6203 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.793s	user 0.017s	sys 0.001s
W20260709 14:52:52.511186  6422 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 14:52:52.512554  2605 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:52.515333  6202 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.400972  6428 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 14:52:52.520689  6347 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:52.452360  6020 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.298s	user 0.017s	sys 0.002s
W20260709 14:52:52.529711  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:52.531672  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.535132  6424 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:52.441630  6426 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 14:52:52.555207  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 278971us
W20260709 14:52:52.559383  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146878us
I20260709 14:52:52.563916  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:52.556080  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 140529us
I20260709 14:52:52.566311  2756 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:52.569391  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:52.566394  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:52.571081  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:52.573570  3536 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 14:52:52.576859  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1251) took 2055 ms (client timeout 1794 ms). Trace:
I20260709 14:52:52.577977  2605 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 14:52:52.579149  2612 rpcz_store.cc:269] 0709 14:52:50.521329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.521777 (+   448us) service_pool.cc:224] Handling call
0709 14:52:52.576590 (+2054813us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b5400480654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.576767 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384917376}
I20260709 14:52:52.445029  6168 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.593894  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.588795  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:52.544693  6414 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:52.595253  3540 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:52.596057  2756 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:52.597595  2751 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:52:52.603410  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.604609  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.605227  3536 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:52.606127  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.606603  3536 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:52.607432  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.607913  3536 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:52.608529  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.453538  6421 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 14:52:52.572650  6020 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:52.501093  6433 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 14:52:52.560026  6333 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.613440  6413 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.660300  6376 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:52.661896  6097 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.003s
I20260709 14:52:52.536455  6348 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.602725  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:52.615059  2612 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:52.616374  2751 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:52.616931  6168 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.617633  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:52.609539  5959 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 14:52:52.623368  6438 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:52.627171  6425 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:52.631784  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:52.632961  3537 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 14:52:52.617486  6011 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:52.634956  6182 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.635622  6416 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:52.639578  6355 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:52.646287  6354 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.000s
I20260709 14:52:52.652318  6428 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:52.626478  2603 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:52:52.674878  3537 pending_rounds.cc:85] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 2
W20260709 14:52:52.682542  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50849.5us
I20260709 14:52:52.675510  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:52.667573  6422 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 14:52:52.692764  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112491us
I20260709 14:52:52.696261  3537 pending_rounds.cc:107] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 14:52:52.715967  3537 raft_consensus.cc:2889] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:52.716002  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.666558  6020 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:52.673189  6438 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:52.686385  5959 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:52.719460  6375 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:52.719509  6424 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:52.672643  6421 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 14:52:52.609447  6414 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:52.708439  6354 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:52.732172  6182 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.732704  6366 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:52:52.733206  6366 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:52.730446  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:52.732337  6261 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.610s	user 0.013s	sys 0.001s
I20260709 14:52:52.735172  2603 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 6. Current term is 7. Ops: []
W20260709 14:52:52.735399  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59994.2us
I20260709 14:52:52.737993  6421 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:52.737587  6438 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.739071  6440 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 14:52:52.744302  6400 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.619s	user 0.005s	sys 0.000s
I20260709 14:52:52.744459  6348 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:52.743886  6424 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:52.745774  3412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.775s (SENT) [suppressed 84 similar messages]
I20260709 14:52:52.718652  6020 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:52.761480  6020 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:52.752017  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.728242  6375 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:52.757148  2612 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:52.762136  6438 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:52.763314  6433 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:52.754920  6421 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.773790  6440 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:52.773962  6421 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.774526  6440 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:52.775031  6440 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:52.764840  5959 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:52.748512  6261 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.767589  6375 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.780223  5959 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:52.788733  5959 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.792021  6414 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.754793  6426 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:52.734155  6366 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:52.803611  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1250) took 2294 ms (client timeout 2052 ms). Trace:
W20260709 14:52:52.804178  2607 rpcz_store.cc:269] 0709 14:52:50.509417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.520857 (+ 11440us) service_pool.cc:224] Handling call
0709 14:52:52.803311 (+2282454us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b540016d5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.803521 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757572352}
I20260709 14:52:52.806252  2605 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
W20260709 14:52:52.812547  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 66822.5us
I20260709 14:52:52.814107  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.818864  2607 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:52.822872  2751 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 14:52:52.785005  6358 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:52.830371  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.787065  6333 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 14:52:52.845664  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1064) took 1180 ms. Trace:
I20260709 14:52:52.832127  6366 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:52.846283  2683 rpcz_store.cc:276] 0709 14:52:51.665404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.666608 (+  1204us) service_pool.cc:224] Handling call
0709 14:52:51.667081 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:51.766484 (+ 99403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:51.766530 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:51.790271 (+ 23741us) log.cc:844] Serialized 52 byte log entry
0709 14:52:51.915093 (+124822us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:51.926648 (+ 11555us) log.cc:844] Serialized 14 byte log entry
0709 14:52:51.960101 (+ 33453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:51.960207 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:52.743381 (+783174us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:52.845274 (+101893us) raft_consensus.cc:1685] finished
0709 14:52:52.845406 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:52.845588 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73997,"prepare.queue_time_us":32542,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":181980,"raft.run_cpu_time_us":572,"raft.run_wall_time_us":28662,"spinlock_wait_cycles":200423552,"thread_start_us":179201,"threads_started":2,"wal-append.queue_time_us":198990}
I20260709 14:52:52.854959  2609 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:52.846666  5959 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:52.858400  2608 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1247) took 2477 ms (client timeout 2060 ms). Trace:
W20260709 14:52:52.858979  2608 rpcz_store.cc:269] 0709 14:52:50.380699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.381040 (+   341us) service_pool.cc:224] Handling call
0709 14:52:52.858145 (+2477105us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.858318 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3222105984}
I20260709 14:52:52.860112  5959 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.860309  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.861122  2607 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:52.861948  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.862674  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:52.863104  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.863440  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:52.864037  2607 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 14:52:52.866026  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.870522  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.870556  6426 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.871469  2608 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:52.873313  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.876034  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.877784  6375 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:52.878259  6407 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.664s	user 0.000s	sys 0.006s
I20260709 14:52:52.878916  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.879002  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 615) took 1288 ms. Trace:
W20260709 14:52:52.879281  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 991) took 1759 ms (client timeout 2034 ms). Trace:
I20260709 14:52:52.881601  6428 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:52.882628  2608 raft_consensus.cc:2376] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13. Re-sending same reply.
W20260709 14:52:52.882668  3541 rpcz_store.cc:269] 0709 14:52:51.119561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.537548 (+417987us) service_pool.cc:224] Handling call
0709 14:52:52.879124 (+1341576us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.879221 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603254016}
I20260709 14:52:52.882588  2606 rpcz_store.cc:276] 0709 14:52:51.591074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.591260 (+   186us) service_pool.cc:224] Handling call
0709 14:52:52.878759 (+1287499us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400371c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:52.878936 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1810471168}
I20260709 14:52:52.886467  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.888820  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.895998  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.897408  2606 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:52.880870  6421 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.903581  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.905809  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.907265  2606 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 14:52:52.900735  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.851091  6207 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.900619  6433 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:52.912241  6433 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:52.912678  6433 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.884320  6428 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:52.916345  6428 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:52.918079  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:52.910508  6348 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:52.921736  6207 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:52.929504  6183 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:52.886409  6440 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.935607  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.936244  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 994) took 1030 ms. Trace:
I20260709 14:52:52.937433  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.936691  3542 rpcz_store.cc:276] 0709 14:52:51.906092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.936283 (+ 30191us) service_pool.cc:224] Handling call
0709 14:52:51.939343 (+  3060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:51.939746 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:51.939781 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:51.949748 (+  9967us) log.cc:844] Serialized 26 byte log entry
0709 14:52:52.050495 (+100747us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:52.050572 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:52.050682 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:52.877640 (+826958us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:52.931141 (+ 53501us) raft_consensus.cc:1685] finished
0709 14:52:52.931160 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:52.936161 (+  5001us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3059,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1846,"spinlock_wait_cycles":41122048,"thread_start_us":90528,"threads_started":1,"wal-append.queue_time_us":264310}
I20260709 14:52:52.938370  6207 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:52:52.938948  6261 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:52.939183  6348 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:52.939827  6348 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:52.939929  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:52.940254  6348 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:52.941102  6438 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:52.942720  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:52.939002  6207 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:52:52.944043  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.03648e+06us
I20260709 14:52:52.944873  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:52.944782  6207 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:52.948719  6111 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:52.946146  6392 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:52:52.948741  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:52.946573  6383 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:52.949349  6111 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.950148  6383 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:52.943671  6438 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:52:52.954524  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.955250  3542 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:52.955425  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:52.952066  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:52.930214  6183 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:52.957073  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.958384  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:52.960273  2683 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
W20260709 14:52:52.960235  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.02649e+06us
I20260709 14:52:52.962905  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:52.964138  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:52.964747  2652 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.791s (SENT)
I20260709 14:52:52.965240  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 630) took 1268 ms. Trace:
I20260709 14:52:52.946125  2651 leader_election.cc:400] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:52:52.953720  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:52.968397  2608 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:52.958400  3541 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:52.971946  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.972393  6399 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:52.974114  6399 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 14:52:52.975818  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.969604  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:52.976024  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:52.981374  2681 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:52.981694  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:52.983846  2678 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:52:52.958069  6450 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 14:52:52.965670  2686 rpcz_store.cc:276] 0709 14:52:51.696928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.697135 (+   207us) service_pool.cc:224] Handling call
0709 14:52:51.757615 (+ 60480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:51.757948 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:51.931897 (+173949us) log.cc:844] Serialized 14 byte log entry
0709 14:52:52.964860 (+1032963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:52.964926 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:52.964988 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:52.965058 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:52.965180 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":927102,"prepare.queue_time_us":102042,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"thread_start_us":247365,"threads_started":2,"wal-append.queue_time_us":303779}
I20260709 14:52:52.993607  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:52.994204  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:52.997437  3540 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:52.992216  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:52.994545  6351 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:52.999761  6351 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:52.999897  6408 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.687s	user 0.015s	sys 0.000s
I20260709 14:52:53.000240  6351 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:53.003468  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:53.005950  2608 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:53.006322  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:53.008975  6426 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.015625  6445 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:53.015969  6426 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.017006  6244 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:53.017498  6244 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:53.018877  6074 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 14:52:53.019579  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 616) took 1399 ms. Trace:
I20260709 14:52:53.020030  2610 rpcz_store.cc:276] 0709 14:52:51.619930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.620176 (+   246us) service_pool.cc:224] Handling call
0709 14:52:51.644007 (+ 23831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:51.652324 (+  8317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.019305 (+1366981us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.019514 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10738432}
I20260709 14:52:53.021713  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:53.025128  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.026206  2757 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:53.027865  3536 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:53.028694  2608 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:52:53.030526  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.036285  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.040429  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.042562  6426 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.043610  5959 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:52:53.044799  6448 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 14:52:53.045462  6424 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:53.057740  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:53.058641  2608 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
I20260709 14:52:53.047431  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:53.059312  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.029191  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:53.061215  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91713.9us
I20260709 14:52:53.063292  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.065231  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:53.066239  2752 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:53.072772  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.081980  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.008525  6367 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:53.089083  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.000422  6408 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:52.951258  6383 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:53.047237  6274 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:52:53.107520  6111 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:53.018443  6244 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:51.201416   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:52:53.098285  2752 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:52:53.133659  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 104583us
W20260709 14:52:53.132364  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 140288us
I20260709 14:52:53.081257  6343 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:53.137463  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:53.119112  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55890us
I20260709 14:52:53.090243  6393 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:53.088917  6426 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:53.141947  6426 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:53.142490  6426 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:53.089376  6367 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:52:53.147478  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69744.3us
I20260709 14:52:53.140179  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.151706  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:53.126689  6363 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:52:53.161518  2607 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
W20260709 14:52:53.173950  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.214794  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1275) took 1314 ms. Trace:
I20260709 14:52:53.157032  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:53.147162  6074 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:53.209570  2611 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:52:53.029711  6183 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:53.125470  6382 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:53.225175  2606 rpcz_store.cc:276] 0709 14:52:51.900213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.162715 (+1262502us) service_pool.cc:224] Handling call
0709 14:52:53.163257 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.214238 (+ 50981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:53.214292 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:53.214360 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:53.214417 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.214508 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:53.214710 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.228168  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1279) took 1258 ms. Trace:
I20260709 14:52:53.239588  2611 rpcz_store.cc:276] 0709 14:52:51.970208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.163949 (+1193741us) service_pool.cc:224] Handling call
0709 14:52:53.209126 (+ 45177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.227679 (+ 18553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.228075 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.212843  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1273) took 1303 ms. Trace:
I20260709 14:52:53.240919  2603 rpcz_store.cc:276] 0709 14:52:51.862440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.718097 (+855657us) service_pool.cc:224] Handling call
0709 14:52:52.729713 (+ 11616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.752854 (+ 23141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.164947 (+412093us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.165309 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12065152}
I20260709 14:52:53.117511  6011 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
W20260709 14:52:53.153920  6419 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.937s	user 0.004s	sys 0.000s
I20260709 14:52:53.240420  2606 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 14:52:53.232347  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1274) took 1369 ms. Trace:
I20260709 14:52:53.262903  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1261) took 1138 ms. Trace:
I20260709 14:52:53.263401  2606 rpcz_store.cc:276] 0709 14:52:52.124399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.239612 (+1115213us) service_pool.cc:224] Handling call
0709 14:52:53.240079 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:53.262504 (+ 22425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.262813 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.262573  2607 rpcz_store.cc:276] 0709 14:52:51.862733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.160589 (+1297856us) service_pool.cc:224] Handling call
0709 14:52:53.161113 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.231732 (+ 70619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.232101 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.226379  6244 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:53.270102  2606 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:53.276609  2603 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:53.277362  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1263) took 1148 ms. Trace:
I20260709 14:52:53.277724  2603 rpcz_store.cc:276] 0709 14:52:52.128850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.242287 (+1113437us) service_pool.cc:224] Handling call
0709 14:52:53.276410 (+ 34123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.277174 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.277316 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.279325  2611 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:53.280035  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1262) took 1155 ms. Trace:
I20260709 14:52:53.280319  2611 rpcz_store.cc:276] 0709 14:52:52.128365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.240696 (+1112331us) service_pool.cc:224] Handling call
0709 14:52:53.279093 (+ 38397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:53.279775 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.279983 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:53.275525  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 118572us
I20260709 14:52:53.277181  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1264) took 1148 ms. Trace:
I20260709 14:52:53.169932  6375 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.137348  6020 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 14:52:53.282632  2606 rpcz_store.cc:276] 0709 14:52:52.129191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.264534 (+1135343us) service_pool.cc:224] Handling call
0709 14:52:53.264900 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.276772 (+ 11872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.277101 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:53.227397  6074 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:53.284580  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 134828us
I20260709 14:52:53.285595  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:52:53.281916  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113251us
W20260709 14:52:53.273363  6405 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 14:52:53.299177  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.257407  6362 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:53.307021  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.309892  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:53.151208  6433 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:53.276763  6111 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:53.307601  6351 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.329201  2604 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:53.329655  2603 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 14:52:53.283572  6183 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:53.332907  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.333957  2685 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:53.334043  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.334667  2681 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:53.294181  6074 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:53.343043  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:53.340510  2677 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:52:53.290457  6440 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:53.344843  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.336074  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:53.343043  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:53.404089  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62728.2us
I20260709 14:52:53.404225  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.405465  2752 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:53.381982  6428 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.399247  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.406591  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:53.328938  6359 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:53.355259  6440 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:53.300338  6438 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:52:53.356336  6433 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:53.412626  2757 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:53.412995  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.328753  6455 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:53.412681  2754 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:52:53.417186  2754 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:53.360265  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1000) took 1110 ms. Trace:
I20260709 14:52:53.353545  6421 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:53.311018  6375 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:53.363202  6371 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.233s
W20260709 14:52:53.450968  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.453542  3542 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:53.329241  6207 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:53.369663  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:53.369928  6409 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.372159  6351 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.377166  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 610) took 2093 ms. Trace:
I20260709 14:52:53.298746  6367 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.379925  6353 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:52:53.381238  6062 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:53.383751  2610 raft_consensus.cc:2376] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9. Re-sending same reply.
I20260709 14:52:53.384531  6031 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:53.388247  6412 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.390019  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1267) took 1145 ms. Trace:
W20260709 14:52:53.390457  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 824129us
I20260709 14:52:53.391144  6305 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:53.391415  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1265) took 1594 ms. Trace:
I20260709 14:52:53.496894  2605 rpcz_store.cc:276] 0709 14:52:51.796911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.578927 (+782016us) service_pool.cc:224] Handling call
0709 14:52:52.805921 (+226994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.806962 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.391178 (+584216us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540016d5d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:53.391327 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469775232}
W20260709 14:52:53.391678  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 759092us
I20260709 14:52:53.392192  6408 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:53.290118  6382 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 14:52:53.396525  3540 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:52:53.397462  6415 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.397987  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1282) took 1358 ms. Trace:
I20260709 14:52:53.500988  6415 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:53.390748  6372 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.388311  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.496353  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.407608  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.503171  2759 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:53.503891  6415 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:53.361769  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1270) took 1523 ms. Trace:
I20260709 14:52:53.504946  2612 rpcz_store.cc:276] 0709 14:52:51.838247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.667489 (+829242us) service_pool.cc:224] Handling call
0709 14:52:52.684392 (+ 16903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.769049 (+ 84657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.361507 (+592458us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.361701 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166799104}
I20260709 14:52:53.414389  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.505714  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:53.506450  2757 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:53.417912  3545 rpcz_store.cc:276] 0709 14:52:52.249841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.250052 (+   211us) service_pool.cc:224] Handling call
0709 14:52:52.274589 (+ 24537us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:52:52.330530 (+ 55941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:52.330566 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:52:52.331447 (+   881us) log.cc:844] Serialized 78 byte log entry
0709 14:52:52.780774 (+449327us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b180046bd00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:52:52.854809 (+ 74035us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:52.856182 (+  1373us) log.cc:844] Serialized 14 byte log entry
0709 14:52:52.856430 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:52.856515 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:53.359878 (+503363us) raft_consensus.cc:1685] finished
0709 14:52:53.360007 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:53.360195 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":802,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":91274,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":15396,"spinlock_wait_cycles":33201024,"thread_start_us":106105,"threads_started":2,"wal-append.queue_time_us":553483}
I20260709 14:52:53.401569  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:53.422078  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.426855  2760 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:53.495811  2753 rpcz_store.cc:276] 0709 14:52:52.244264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.249067 (+  4803us) service_pool.cc:224] Handling call
0709 14:52:52.276425 (+ 27358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.301452 (+ 25027us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:52:52.320147 (+ 18695us) log.cc:844] Serialized 14 byte log entry
0709 14:52:53.388900 (+1068753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:53.388969 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:53.389028 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.389733 (+   705us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.389782 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:53.389940 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":990360,"prepare.queue_time_us":25415,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":970263,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":134,"spinlock_wait_cycles":56664832,"thread_start_us":98380,"threads_started":1,"wal-append.queue_time_us":181119}
I20260709 14:52:53.413765  6203 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.435499  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.439639  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.441437  6461 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.441752  6462 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:53.388576  6074 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:53.442711  6392 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:53.442756  6460 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:52:53.445124  6348 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.005s
I20260709 14:52:53.408957  6354 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 14:52:53.445822  6399 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:52:53.448518  6428 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.421950  6111 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 14:52:53.315225  6393 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:53.487865  2609 rpcz_store.cc:276] 0709 14:52:51.283871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.303009 (+ 19138us) service_pool.cc:224] Handling call
0709 14:52:52.577576 (+1274567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.855521 (+277945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.157015 (+301494us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400480654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.376868 (+219853us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400480650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:53.377078 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3567219200}
I20260709 14:52:53.489122  2754 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
W20260709 14:52:53.489647  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.490796  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.421407  6455 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:53.504536  6415 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:53.501526  2604 rpcz_store.cc:276] 0709 14:52:52.039904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.211626 (+1171722us) service_pool.cc:224] Handling call
0709 14:52:53.258708 (+ 47082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:53.343234 (+ 84526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.397706 (+ 54472us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:53.397916 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161481984}
I20260709 14:52:53.511715  6363 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 14:52:53.512372  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:53.512844  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 90842.6us
I20260709 14:52:53.511782  2684 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:53.511106  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.457427  6375 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:53.501384  6305 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:53.519155  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.494385  6353 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:53.497912  6031 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:53.535830  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 134353us
I20260709 14:52:53.592252  2678 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:53.681855  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.557559  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.674688  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:53.488330  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 332348us
I20260709 14:52:53.597364  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.683902  3540 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:53.517823  6372 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:53.686050  3536 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:53.686196  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 167108us
W20260709 14:52:53.686764  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 535084us
W20260709 14:52:53.682050  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:53.687482  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 282098us
W20260709 14:52:53.687994  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 175773us
W20260709 14:52:53.684823  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 375021us
I20260709 14:52:53.691852  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.686877  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.692310  3542 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:53.712023  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:53.732869  2679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.014s	sys 0.008s
W20260709 14:52:53.748904  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.733975  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:53.761416  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.769626  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:53.769637  3546 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:53.770161  2679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.022s	sys 0.008s
I20260709 14:52:53.771518  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 637) took 1940 ms. Trace:
W20260709 14:52:53.687022  6426 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.002s
I20260709 14:52:53.583906  6408 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:53.508141  6412 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:53.787645  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:52:53.787916  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106164us
I20260709 14:52:53.643011  6392 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:52:53.685782  6366 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.835s	user 0.012s	sys 0.002s
I20260709 14:52:53.777805  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:53.799081  2682 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:53.685586  6455 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:53.772151  2679 rpcz_store.cc:276] 0709 14:52:51.831115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:51.831468 (+   353us) service_pool.cc:224] Handling call
0709 14:52:51.847279 (+ 15811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:51.887089 (+ 39810us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:52:51.905530 (+ 18441us) log.cc:844] Serialized 14 byte log entry
0709 14:52:52.018474 (+112944us) log.cc:844] Serialized 14 byte log entry
0709 14:52:53.138022 (+1119548us) log.cc:844] Serialized 14 byte log entry
0709 14:52:53.138196 (+   174us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:53.140400 (+  2204us) log.cc:844] Serialized 52 byte log entry
0709 14:52:53.140590 (+   190us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:53.140813 (+   223us) log.cc:844] Serialized 14 byte log entry
0709 14:52:53.771026 (+630213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.771112 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:53.771158 (+    46us) raft_consensus.cc:1685] finished
0709 14:52:53.771172 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:53.771453 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1088855,"prepare.queue_time_us":539375,"prepare.run_cpu_time_us":199,"prepare.run_wall_time_us":5610,"raft.queue_time_us":2104246,"raft.run_cpu_time_us":783,"raft.run_wall_time_us":17812,"thread_start_us":839874,"threads_started":3,"wal-append.queue_time_us":109001}
I20260709 14:52:53.777976  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:53.668216  6381 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:52:53.613977  6456 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 14:52:53.789680  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.687352  6367 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:53.692873  6416 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.794091  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.524703  6203 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:53.729703  6062 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:53.695950  6352 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 14:52:53.802121  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:53.829831  3542 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:53.831297  3547 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:53.831751  3547 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:53.784492  6261 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:52:53.814076  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:53.814908  2610 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 14:52:53.815397  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45850.2us
I20260709 14:52:53.820582  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:53.821722  6471 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:53.821830  6416 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:53.784287  6426 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:53.834585  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:53.693266  6347 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 14:52:53.798388  6392 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:53.782771  6408 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:53.823279  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:53.816488  6354 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:53.824108  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:53.777562  6433 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:52:53.823499  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:53.826452  2756 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:52:53.769121  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 82352.6us
I20260709 14:52:53.694528  6348 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.825618  6412 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.970611  3540 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:53.835829  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46267.9us
I20260709 14:52:54.018802  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:53.822420  6074 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:53.805346  6388 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:53.777951  6428 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 14:52:53.929895  2604 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:53.826975  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:54.046504  2679 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:53.924489  2752 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:53.529938  6461 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:53.915987  3547 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:53.835179  6182 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:53.832883  6323 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:52:53.929458  6445 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:53.818405  6381 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:54.109458  6381 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.923745  6344 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.143s
I20260709 14:52:53.929044  6392 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:53.972831  6369 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.266s
I20260709 14:52:53.979936  6031 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:53.980402  2680 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:53.982101  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:53.983021  6348 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:53.910072  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:52:53.967796  6305 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:53.829948  6415 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:53.990442  6467 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:53.827039  6399 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 14:52:54.005901  6359 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.007297  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:54.141014  3545 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:54.013252  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:54.014322  6351 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:54.120337  6323 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:54.023979  3542 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:54.024634  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:54.147714  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123216us
W20260709 14:52:54.032284  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 211778us
I20260709 14:52:53.907377  6338 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:54.056981  3547 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:53.936813  6408 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:53.944723  2682 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:52:54.171600  6466 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.171831  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:54.172432  6466 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:53.999002  6393 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:54.173586  6392 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:54.175238  6463 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.093931  6462 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:52:54.175868  6462 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:54.175069  2684 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:53.962653  6471 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:54.177280  6471 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:54.176904  6462 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:54.177747  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:54.178455  6462 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:54.107345  6476 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 14:52:54.108140  6203 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:54.178759  6409 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:54.179476  6203 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:52:54.179788  6183 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.837s	user 0.008s	sys 0.007s
I20260709 14:52:54.180037  6409 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:54.119973  6445 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:54.180500  6409 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:54.112253  2686 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:53.822784  6374 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:54.116812  2604 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:54.044446  6207 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.120915  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 647) took 1158 ms. Trace:
I20260709 14:52:54.033499  6371 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:53.915241  6261 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:54.045096  6388 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:54.013441  6426 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:54.126622  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:54.133118  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 223554us
I20260709 14:52:54.219761  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:52:54.133662  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 366273us
I20260709 14:52:54.111034  6381 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:54.060185  6412 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:54.137795  3540 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:54.143637  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:54.145579  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:54.005895  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:54.153165  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:54.175048  6414 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:54.106400  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 630) took 1345 ms. Trace:
I20260709 14:52:54.250520  2607 rpcz_store.cc:276] 0709 14:52:52.760885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.267751 (+506866us) service_pool.cc:224] Handling call
0709 14:52:53.268318 (+   567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:53.297767 (+ 29449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:53.297805 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:53.298388 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 14:52:53.497924 (+199536us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:53.497972 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.498053 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:54.106082 (+608029us) raft_consensus.cc:1685] finished
0709 14:52:54.106112 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:54.106316 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42384,"raft.queue_time_us":39115,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":13469,"thread_start_us":183827,"threads_started":1,"wal-append.queue_time_us":790152}
I20260709 14:52:54.078155  6182 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:54.108386  6461 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:53.971282  6354 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:53.930236  6375 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:54.195169  3546 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:52:54.197594  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 426112us
I20260709 14:52:54.198761  2679 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:52:54.144896  6411 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:54.201241  2683 rpcz_store.cc:276] 0709 14:52:52.962015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.962929 (+   914us) service_pool.cc:224] Handling call
0709 14:52:52.970992 (+  8063us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:53.009120 (+ 38128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:53.009166 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:53.019440 (+ 10274us) log.cc:844] Serialized 52 byte log entry
0709 14:52:53.273827 (+254387us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:53.273893 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.274004 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:54.120467 (+846463us) raft_consensus.cc:1685] finished
0709 14:52:54.120503 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:54.120772 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82373,"raft.queue_time_us":38505,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3379,"spinlock_wait_cycles":53376,"thread_start_us":253867,"threads_started":1,"wal-append.queue_time_us":388912}
I20260709 14:52:54.144397  6062 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.181372  6413 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:54.201627  6351 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:54.243840  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:54.154327  6359 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:54.175902  6463 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.207335  6207 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.248760  6203 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:54.303370  6392 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:54.291033  6411 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:54.294430  6413 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:54.303416  6182 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:54.243698  6466 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:54.343940  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:54.338794  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 95052.2us
I20260709 14:52:54.215814  6388 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:54.324738  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 283424us
I20260709 14:52:54.255894  6429 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 14:52:54.285071  6426 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.307384  6354 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:54.252683  6412 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:54.212308  6371 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.355701  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:54.362933  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:54.326584  6203 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:54.215265  6415 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:54.361342  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:54.364871  2607 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:54.392947  2608 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [9.6-12.7]
I20260709 14:52:54.398609  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:54.340272  6411 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:54.339934  6413 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:54.396674  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:54.411896  2611 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 14:52:54.412703  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.413786  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:54.398975  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 176667us
I20260709 14:52:54.312034  6062 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:54.406366  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:54.407507  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:54.415668  2606 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:54.284317  6409 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:54.417630  6409 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:54.358033  6362 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:54.425284  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:54.348753  6433 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:54.428398  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:54.429323  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1160) took 1091 ms. Trace:
I20260709 14:52:54.438371  2686 rpcz_store.cc:276] 0709 14:52:53.338285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.587518 (+249233us) service_pool.cc:224] Handling call
0709 14:52:54.429040 (+841522us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540022d154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:54.429244 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984180992}
I20260709 14:52:54.333262  6399 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 14:52:54.373193  6182 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:54.446470  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:54.395995  6448 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 14:52:54.449505  2757 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:54.401526  6203 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:54.463358  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 652) took 1089 ms. Trace:
I20260709 14:52:54.463632  6444 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:54.464396  6444 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:54.465495  6444 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.463953  2682 rpcz_store.cc:276] 0709 14:52:53.374275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.173296 (+799021us) service_pool.cc:224] Handling call
0709 14:52:54.173719 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:54.182947 (+  9228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:54.182986 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:54.204530 (+ 21544us) log.cc:844] Serialized 26 byte log entry
0709 14:52:54.401817 (+197287us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:54.402202 (+   385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:54.402295 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:54.463060 (+ 60765us) raft_consensus.cc:1685] finished
0709 14:52:54.463090 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:54.463279 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8674,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14448,"thread_start_us":102710,"threads_started":1,"wal-append.queue_time_us":205236}
I20260709 14:52:54.395522  6412 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:54.475332  2757 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:54.402129  6393 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:54.395643  6429 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:52:54.447144  6478 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:54.438727  6461 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:52:54.458901  6362 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:54.459638  6480 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:54.459990  6475 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 14:52:54.477697  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:54.510097  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.394260  6426 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.443900  6354 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:54.436612  6011 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:54.328787  6323 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:54.511039  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.457103  6031 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:52:54.497043  3547 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 14:52:54.485494  6467 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.388008  6468 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.101s
W20260709 14:52:54.505728  6244 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.213s	user 0.013s	sys 0.004s
I20260709 14:52:54.555246  6011 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:54.554905  6111 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:54.559679  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1168) took 1146 ms. Trace:
I20260709 14:52:54.560261  6261 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:54.454236  6351 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.577035  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:54.555512  3547 pending_rounds.cc:85] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:52:54.580986  3547 pending_rounds.cc:107] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 14:52:54.581355  3547 raft_consensus.cc:2889] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:54.560133  6359 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:54.502962  6478 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:54.598426  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:54.966179  3538 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:54.563786  6467 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:54.800886  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.579347  2759 rpcz_store.cc:276] 0709 14:52:53.412910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.504040 (+ 91130us) service_pool.cc:224] Handling call
0709 14:52:53.504476 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:53.518798 (+ 14322us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:53.518853 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:53.519496 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 14:52:53.924429 (+404933us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:53.924478 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.924555 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:54.559417 (+634862us) raft_consensus.cc:1685] finished
0709 14:52:54.559444 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:54.559608 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13798,"raft.queue_time_us":15988,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":404471,"threads_started":1,"wal-append.queue_time_us":563002}
W20260709 14:52:55.068670  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:55.069051  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.040s
W20260709 14:52:55.086953  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.005s	sys 0.040s
W20260709 14:52:55.075766  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.002s	sys 0.012s
W20260709 14:52:55.087387  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 635755us
W20260709 14:52:55.087589  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.002s	sys 0.013s
I20260709 14:52:54.951609  6493 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:54.986918  6408 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.818s	user 0.015s	sys 0.005s
W20260709 14:52:54.872191  6481 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
W20260709 14:52:55.035576  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.027s	sys 0.020s
W20260709 14:52:54.780020  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:55.148327  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.028s	sys 0.021s
W20260709 14:52:55.148707  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.953434  6333 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.093173  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:54.657200  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73004.3us
I20260709 14:52:54.922968  6463 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:54.597450  6468 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.110219  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.015s	sys 0.015s
W20260709 14:52:55.157702  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.016s	sys 0.015s
I20260709 14:52:54.809854  6183 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:54.875952  6498 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.875448  6338 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.662761  6111 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:54.600373  6031 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:54.597419  6343 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.334s	user 0.002s	sys 0.010s
I20260709 14:52:54.552234  6480 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:54.599556  6445 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:54.647662  6074 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:55.157066  6244 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:54.583420  6477 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:54.580675  6433 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:55.150435  2611 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:52:55.121239  6455 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.287s	user 0.014s	sys 0.003s
I20260709 14:52:54.684639  6355 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.450s
I20260709 14:52:54.580226  6466 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.173573  6343 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.917s	user 0.006s	sys 0.023s
I20260709 14:52:55.161770  6372 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:55.187448  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:54.952960  6362 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.183583  6445 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.144004  6408 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.188525  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 658) took 1413 ms. Trace:
W20260709 14:52:55.188597  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.009s
W20260709 14:52:55.228976  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.015s	sys 0.010s
I20260709 14:52:55.120343  6493 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.230420  6466 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:55.231352  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:55.168500  6183 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.189834  6074 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.197332  6477 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:55.236863  6074 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:55.238119  6483 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 14:52:55.166647  6333 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:55.198530  2652 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.930s (SENT)
I20260709 14:52:55.228756  2683 rpcz_store.cc:276] 0709 14:52:53.774800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.295792 (+520992us) service_pool.cc:224] Handling call
0709 14:52:54.311130 (+ 15338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:54.311411 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:54.784627 (+473216us) log.cc:844] Serialized 14 byte log entry
0709 14:52:55.187957 (+403330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:55.188044 (+    87us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:55.188126 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:55.188228 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:55.188438 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":188,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":10329,"thread_start_us":875386,"threads_started":2}
I20260709 14:52:55.247733  2607 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:55.249114  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:55.181538  6338 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.206445  6478 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.236672  6183 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:55.190181  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.021s	sys 0.024s
W20260709 14:52:55.254030  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.022s	sys 0.024s
I20260709 14:52:55.209028  6343 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:55.295523  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:55.301575  2759 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:55.301748  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:55.233763  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:55.204473  6371 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.250000  3541 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:55.167151  6468 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.263538  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:55.311148  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.008s	sys 0.015s
I20260709 14:52:55.241904  6062 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.231472  6493 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:55.262631  6490 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.436s	user 0.000s	sys 0.002s
W20260709 14:52:55.307840  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 219565us
I20260709 14:52:55.307758  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1249) took 5441 ms. Trace:
I20260709 14:52:55.242184  6467 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.310570  2759 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:52:55.203795  6244 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:55.311905  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.008s	sys 0.015s
W20260709 14:52:55.300884  6367 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.393s	user 0.009s	sys 0.008s
I20260709 14:52:55.233661  6372 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:55.173933  6498 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.294672  6502 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.307096  6388 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.911s	user 0.015s	sys 0.000s
I20260709 14:52:55.319548  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:55.321636  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:55.318332  2758 rpcz_store.cc:276] 0709 14:52:49.861264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:49.863265 (+  2001us) service_pool.cc:224] Handling call
0709 14:52:50.714477 (+851212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:52.319355 (+1604878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:53.395121 (+1075766us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:55.302046 (+1906925us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54000e6850. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:55.302255 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5645193344}
I20260709 14:52:55.340480  2759 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:55.237777  6477 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.341068  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:55.334009  6467 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:55.341507  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:55.342234  6333 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.341995  6467 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:55.344465  6498 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.344585  6467 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:55.332201  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:55.345919  2757 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:55.316514  6471 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.139s	user 0.009s	sys 0.006s
I20260709 14:52:55.241675  6445 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.270994  6031 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:55.313251  6483 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 14:52:55.217702  6455 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:55.354588  6468 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.346705  6471 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.351791  6505 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:55.352228  6499 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 14:52:55.342267  2759 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:52:55.366333  6381 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.116s	user 0.000s	sys 0.014s
W20260709 14:52:55.366689  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47251.7us
W20260709 14:52:55.367300  6503 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 14:52:55.368172  6471 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.371759  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:55.367326  6498 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:52:55.368921  6505 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:55.372716  6426 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.372977  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:55.373713  2680 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:55.352797  6445 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:55.378046  6411 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.933s	user 0.010s	sys 0.001s
W20260709 14:52:55.372751  6462 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.194s	user 0.011s	sys 0.008s
I20260709 14:52:55.331943  6062 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:55.356288  6455 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:55.391561  6343 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.396097  6510 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:52:55.357172  6388 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.388131  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:55.382989  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52617.2us
W20260709 14:52:55.397957  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1161) took 1999 ms (client timeout 1868 ms). Trace:
W20260709 14:52:55.379797  6504 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 14:52:55.300217  6111 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:55.402110  6203 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.941s	user 0.013s	sys 0.001s
W20260709 14:52:55.404538  6497 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 14:52:55.404224  6062 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.405788  6498 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.419066  2756 rpcz_store.cc:269] 0709 14:52:53.398237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.412744 (+ 14507us) service_pool.cc:224] Handling call
0709 14:52:55.397706 (+1984962us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:55.397880 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948683520}
I20260709 14:52:55.417522  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:55.394534  6468 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 14:52:55.429471  2759 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
W20260709 14:52:55.430796  6409 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.012s	user 0.009s	sys 0.009s
I20260709 14:52:55.410106  6323 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:55.420858  6462 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:55.437217  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:55.439833  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:55.424453  6514 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.349220  6502 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:55.437049  6362 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:55.444060  6514 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:55.455406  6480 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:55.428095  6203 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.445225  6455 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:52:55.457350  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:55.441116  6462 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:55.468554  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1288) took 1107 ms. Trace:
I20260709 14:52:55.469929  2609 rpcz_store.cc:276] 0709 14:52:54.360962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.363996 (+  3034us) service_pool.cc:224] Handling call
0709 14:52:54.364490 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:54.400212 (+ 35722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:54.400253 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:54.423846 (+ 23593us) log.cc:844] Serialized 26 byte log entry
0709 14:52:55.158350 (+734504us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:55.158422 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:55.158535 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:55.468285 (+309750us) raft_consensus.cc:1685] finished
0709 14:52:55.468309 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:55.468479 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27070,"raft.queue_time_us":27276,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":12665,"thread_start_us":655447,"threads_started":1,"wal-append.queue_time_us":652253}
I20260709 14:52:55.456092  6062 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.458364  6338 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.409675  6411 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.437417  6498 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:55.468938  2605 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:55.457985  6362 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:55.551673  2760 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:55.692687  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 258215us
I20260709 14:52:55.478806  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1286) took 1118 ms. Trace:
W20260709 14:52:55.879174  2651 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.958s (SENT)
W20260709 14:52:55.755467  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:55.879513  2604 rpcz_store.cc:276] 0709 14:52:54.360333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.362945 (+  2612us) service_pool.cc:224] Handling call
0709 14:52:54.363443 (+   498us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:54.376399 (+ 12956us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:52:54.376452 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:54.400103 (+ 23651us) log.cc:844] Serialized 52 byte log entry
0709 14:52:55.148858 (+748755us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:55.148910 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:55.149003 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:55.478135 (+329132us) raft_consensus.cc:1685] finished
0709 14:52:55.478170 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:55.478615 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24988,"raft.queue_time_us":27676,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":12284,"thread_start_us":659035,"threads_started":1,"wal-append.queue_time_us":712743}
W20260709 14:52:55.480180  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:55.802234  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1163) took 2400 ms (client timeout 1958 ms). Trace:
I20260709 14:52:55.881774  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:55.768590  3546 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:52:55.631150  6358 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.473749  6462 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:55.883628  6358 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:55.657037  6518 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.205s	user 0.000s	sys 0.002s
I20260709 14:52:55.547375  6480 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.890204  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:55.890569  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:55.891029  2678 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:55.542197  2758 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:52:55.680617  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:55.891925  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:55.881880  2752 rpcz_store.cc:269] 0709 14:52:53.402114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.406255 (+  4141us) service_pool.cc:224] Handling call
0709 14:52:55.801915 (+2395660us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:55.802107 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185864576}
I20260709 14:52:55.892913  2679 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
W20260709 14:52:55.882424  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:55.895212  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:55.895212  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:55.897276  2752 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:55.898478  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:55.899164  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:55.571486  6381 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.687494  6393 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.201s	user 0.013s	sys 0.001s
I20260709 14:52:55.900315  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:55.907258  2685 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:55.900691  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:55.564476  6182 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.089s	user 0.013s	sys 0.000s
W20260709 14:52:55.681622  6513 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 14:52:55.469048  6244 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.468550  6455 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:55.889290  6362 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:55.603725  6372 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:55.601523  6388 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.683215  6111 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.657348  6510 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:55.909113  2686 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:52:55.800822  6011 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:55.921352  6438 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.905664  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:55.690292  6392 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:55.556550  6444 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.564776  6398 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.902103  6480 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:55.888008  6467 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:55.895975  6354 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.340s	user 0.007s	sys 0.007s
I20260709 14:52:55.473214  6408 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.896222  6445 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.924769  2685 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:55.936605  2608 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:55.917613  6393 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:52:55.590392  6468 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:52:55.941745  6412 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.438s	user 0.007s	sys 0.008s
W20260709 14:52:55.960276  6522 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 14:52:56.052768  2681 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:52:55.913892  6381 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.011173  6398 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.056917  6381 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.057070  6398 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.029966  6444 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.059842  6444 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.010792  6502 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:55.996630  6358 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.061395  6468 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.064937  6411 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:56.066150  6489 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.211s	user 0.003s	sys 0.001s
I20260709 14:52:56.001252  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1282) took 2603 ms. Trace:
I20260709 14:52:55.569536  6415 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:56.040211  6366 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:55.961253  6111 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:55.966598  6510 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.995286  6498 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.050634  6412 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:55.993245  6392 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.074534  6502 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.075209  6372 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:56.076001  6031 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:52:56.076443  6344 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.076951  6358 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:56.081700  6444 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:55.990542  6438 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.111763  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1287) took 2095 ms. Trace:
I20260709 14:52:56.251302  2751 rpcz_store.cc:276] 0709 14:52:54.016585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.016840 (+   255us) service_pool.cc:224] Handling call
0709 14:52:54.081073 (+ 64233us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:54.113386 (+ 32313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:54.113422 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:54.114004 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 14:52:54.398981 (+284977us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:54.399045 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:54.403344 (+  4299us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:56.065454 (+1662110us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:56.111353 (+ 45899us) raft_consensus.cc:1685] finished
0709 14:52:56.111378 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:56.111673 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6656,"raft.queue_time_us":5305,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":13504,"spinlock_wait_cycles":9148288,"thread_start_us":258147,"threads_started":1,"wal-append.queue_time_us":570229}
I20260709 14:52:56.125355  3536 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:56.121068  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:56.121311  3411 connection.cc:256] RPC call timeout handler was delayed by -2.272470285000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 14:52:56.044236  2678 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:56.107030  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.114087  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.068794  2755 rpcz_store.cc:276] 0709 14:52:53.397336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.411535 (+ 14199us) service_pool.cc:224] Handling call
0709 14:52:55.398406 (+1986871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:55.408319 (+  9913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:55.408353 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:55.408950 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 14:52:55.567160 (+158210us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:52:55.685806 (+118646us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:55.685858 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:55.685965 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:55.998492 (+312527us) raft_consensus.cc:1685] finished
0709 14:52:55.998530 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:56.000877 (+  2347us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54003363d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:56.001097 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9482,"raft.queue_time_us":12785,"raft.run_cpu_time_us":2256,"raft.run_wall_time_us":145155,"spinlock_wait_cycles":4518065408,"thread_start_us":276456,"threads_started":1,"wal-append.queue_time_us":546083}
I20260709 14:52:56.133590  3541 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:56.248390  6398 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 14:52:56.256024  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.256352  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.256997  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:56.258512  3536 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:56.135646  3542 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 14:52:56.258258  6353 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.259704  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.260402  6356 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:56.260742  3542 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:56.261777  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.263177  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.269124  2751 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:52:56.201795  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:56.223701  6388 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:56.187748  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 282225us
I20260709 14:52:56.070374  6480 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.188869  6363 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:56.088629  6415 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:56.096378  6416 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.040760  6467 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.036382  6455 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.144495  6421 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.239676  3540 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
W20260709 14:52:56.110913  6491 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.564s	user 0.007s	sys 0.010s
I20260709 14:52:56.296182  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.140432  6426 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:56.107165  6413 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.653s	user 0.010s	sys 0.006s
I20260709 14:52:56.144187  6433 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 14:52:56.298820  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:56.188252  2679 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:56.147547  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.299865  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:56.300226  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 152831us
I20260709 14:52:56.240942  6471 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:56.241976  6392 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.145041  6352 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 14:52:56.245291  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.305806  2606 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:56.248672  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.260824  6356 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:56.140188  3543 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:52:56.155918  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.263636  6203 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:56.264309  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 143309us
I20260709 14:52:56.321990  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.264854  6505 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:56.265174  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:56.265501  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.266294  3541 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 14:52:56.267796  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1170) took 2130 ms (client timeout 2113 ms). Trace:
W20260709 14:52:56.329237  2760 rpcz_store.cc:269] 0709 14:52:54.137147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.146768 (+  9621us) service_pool.cc:224] Handling call
0709 14:52:56.267564 (+2120796us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540040e7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:56.267717 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2999388416}
I20260709 14:52:56.042107  6445 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.272526  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:56.151155  6502 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.277660  6514 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.278719  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:56.276654  6381 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 14:52:56.287788  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1163) took 2881 ms. Trace:
I20260709 14:52:56.375746  2680 rpcz_store.cc:276] 0709 14:52:53.406855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.145243 (+738388us) service_pool.cc:224] Handling call
0709 14:52:56.287392 (+2142149us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:56.287657 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2584169216}
I20260709 14:52:56.158613  6372 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.223636  3538 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:52:56.283463  6388 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.215487  6344 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.429771  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1294) took 1235 ms. Trace:
I20260709 14:52:56.431396  2608 rpcz_store.cc:276] 0709 14:52:55.194009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.194229 (+   220us) service_pool.cc:224] Handling call
0709 14:52:55.201536 (+  7307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.031547 (+830011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.429357 (+397810us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54001ebc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:56.429672 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686716800}
W20260709 14:52:56.299460  6414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.997s	user 0.027s	sys 0.053s
I20260709 14:52:56.303720  6421 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.322760  3547 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 14:52:56.433977  6414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.138s	user 0.027s	sys 0.053s
I20260709 14:52:56.434679  6305 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:56.435904  6421 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.436616  6421 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.436509  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.437405  3547 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:56.281693  6363 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:56.439077  6363 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:56.336326  2751 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:56.440867  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.442212  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.298188  6480 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:52:56.338928  2679 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:52:56.254153  6358 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.298389  6399 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.295564  6478 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:56.369659  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:56.370026  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91420.3us
I20260709 14:52:56.145401  6366 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.370927  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.306145  6413 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.319267  6408 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:56.375010  3536 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:56.466195  2760 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:52:56.200481  6031 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.386063  3541 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:56.389197  6372 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.390818  6468 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.392086  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.392951  6202 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 14:52:56.393417  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1168) took 2985 ms (client timeout 2031 ms). Trace:
W20260709 14:52:56.394418  6523 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.700s	user 0.010s	sys 0.010s
W20260709 14:52:56.455899  6375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.013s	sys 0.035s
I20260709 14:52:56.400513  6471 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:56.389268  6445 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.398679  6466 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:56.404397  6529 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 14:52:56.410614  6438 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.415676  6467 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:52:56.416097  6525 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.855s	user 0.001s	sys 0.000s
I20260709 14:52:56.424104  6412 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.425457  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.427532  6502 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.431926  6344 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.304682  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.612040  3542 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 7.
I20260709 14:52:56.613371  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.380699  6505 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:56.432943  6433 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:56.320165  6426 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.322705  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:56.438580  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:56.620244  3411 connection.cc:256] RPC call timeout handler was delayed by -1.9925702850000562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2194s for a response
I20260709 14:52:56.462745  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:52:56.621121  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 158499us
I20260709 14:52:56.622339  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:56.455379  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 181866us
I20260709 14:52:56.399277  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.470723  2754 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:56.558156  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.557608  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:56.511319  2684 rpcz_store.cc:269] 0709 14:52:53.408362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:53.691510 (+283148us) service_pool.cc:224] Handling call
0709 14:52:56.393069 (+2701559us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:56.393302 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016715648}
I20260709 14:52:56.386314  6415 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.600189  6534 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:56.508656  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 186754us
I20260709 14:52:56.570197  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.467326  6478 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.567750  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:56.467453  6362 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.000s
I20260709 14:52:56.575155  2756 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:56.470773  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.452268  6305 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:56.578336  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.429809  6510 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.655710  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:56.565019  6375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.755s	user 0.016s	sys 0.038s
I20260709 14:52:56.437237  6421 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.452286  6480 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:56.599807  2760 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:52:56.650164  3537 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:52:56.670902  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 660) took 1336 ms. Trace:
I20260709 14:52:56.671396  2760 rpcz_store.cc:276] 0709 14:52:55.334359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.575166 (+1240807us) service_pool.cc:224] Handling call
0709 14:52:56.579434 (+  4268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:56.670521 (+ 91087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.670801 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:56.470108  6408 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:56.494894  6462 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.005s
I20260709 14:52:56.566028  6461 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.574571  6202 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:56.572438  6471 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:56.632253  6412 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 14:52:56.452575  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 325549us
W20260709 14:52:56.323601  6351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.428s	user 0.038s	sys 0.085s
I20260709 14:52:56.599785  6445 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:56.591554  6468 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.559163  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:56.675906  6408 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:56.597854  6274 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.623606  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:56.705899  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:56.707516  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:56.708159  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:56.628002  3411 connection.cc:256] RPC call timeout handler was delayed by -1.9793352400000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
I20260709 14:52:56.467204  6399 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.623276  3538 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:56.624836  2683 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:56.602463  6372 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:56.570349  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:56.608556  6466 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.566406  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.632145  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:56.629724  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:56.644981  2686 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:56.658268  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.655714  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.618177  6505 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.630963  6062 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:56.667412  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:56.674466  2751 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:52:56.722908  3540 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:56.723429  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 665) took 1333 ms. Trace:
I20260709 14:52:56.651634  6415 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:56.677275  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:56.574124  6367 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.683300  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.645876  6534 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.633409  6382 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 14:52:56.673609  6440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.005s	sys 0.024s
I20260709 14:52:56.700268  2678 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:56.622807  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:56.707324  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:56.624502  2754 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 14:52:56.709053  3411 connection.cc:256] RPC call timeout handler was delayed by -1.6694702850001022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2553s for a response
W20260709 14:52:56.689879  6351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.818s	user 0.039s	sys 0.085s
I20260709 14:52:56.659543  6362 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.661303  6305 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.713359  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.725571  6062 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:56.720094  2681 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:56.696130  6182 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.723665  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:56.724852  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:56.706574  6468 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.687937  6461 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:56.659063  6416 consensus_queue.cc:1175] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c1b0e692e4294ee3b8694bbada174ba1" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 9 index: 5 } }
I20260709 14:52:56.701123  6445 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:56.679368  6462 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:56.731426  3536 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:56.736418  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:56.763085  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 133436us
I20260709 14:52:56.779366  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:56.779960  3411 connection.cc:256] RPC call timeout handler was delayed by -1.2062349929999527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 14:52:56.804863  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.805018  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.782534  2678 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:52:56.805531  2686 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:52:56.806175  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:56.783138  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 160403us
I20260709 14:52:56.807283  3538 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:52:56.725348  6505 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:52:56.784633  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71333.8us
I20260709 14:52:56.720184  3547 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:52:56.723874  2751 rpcz_store.cc:276] 0709 14:52:55.390150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.652628 (+1262478us) service_pool.cc:224] Handling call
0709 14:52:56.674211 (+ 21583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:56.723093 (+ 48882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.723355 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:56.788602  2683 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8. Re-sending same reply.
W20260709 14:52:56.777796  6440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.005s	sys 0.024s
I20260709 14:52:56.809530  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.735380  6540 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:52:56.715315  6274 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.810951  6540 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:56.811403  6540 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:56.797235  2684 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:56.815395  2683 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 14:52:56.816404  2684 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:56.798707  2681 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 14:52:56.817440  2681 pending_rounds.cc:85] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 5
I20260709 14:52:56.817709  2681 pending_rounds.cc:107] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 14:52:56.817936  2681 raft_consensus.cc:2889] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:56.799194  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:56.781418  6358 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.782720  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 662) took 1420 ms. Trace:
I20260709 14:52:56.819846  2754 rpcz_store.cc:276] 0709 14:52:55.362422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.623760 (+1261338us) service_pool.cc:224] Handling call
0709 14:52:56.624188 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:56.782426 (+158238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.782655 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:56.821506  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:56.823426  2751 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:56.823670  2754 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:56.824416  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 666) took 1429 ms. Trace:
I20260709 14:52:56.824687  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1179) took 1390 ms. Trace:
I20260709 14:52:56.824982  2751 rpcz_store.cc:276] 0709 14:52:55.395469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.810473 (+1415004us) service_pool.cc:224] Handling call
0709 14:52:56.822979 (+ 12506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.824071 (+  1092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.824340 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:56.825294  2754 rpcz_store.cc:276] 0709 14:52:55.434641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.822987 (+1388346us) service_pool.cc:224] Handling call
0709 14:52:56.823385 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.824310 (+   925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:56.824560 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:56.789685  6062 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:56.717980  6372 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.791082  6438 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.730888  6542 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 14:52:56.849684  6542 consensus_queue.cc:1243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:52:56.850272  6542 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:52:56.790596  6412 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:56.850965  6011 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:52:56.851437  6354 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:56.851982  6354 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:56.726037  6466 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.853220  6362 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.852954  6354 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.854127  6461 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.854712  6466 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.786415  6544 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:56.855362  6468 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.855791  6433 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:56.855569  6544 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.856274  6433 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:56.837046  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:56.740334  6367 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:56.871260  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.872625  2608 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 14:52:56.872541  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:56.716672  6480 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.795199  6413 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:56.844732  6545 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 14:52:56.887146  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:56.827565  6031 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.894557  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.895661  2608 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:56.845950  6207 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.848855  6534 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.800844  6388 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.859558  2678 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:56.865075  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.866343  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.867147  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.803143  6445 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:56.735780  6471 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.907311  6471 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:56.907522  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.909145  2609 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:56.874878  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.910552  2610 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:52:56.828668  6274 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.837929  6381 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:56.887077  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:56.887990  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.914564  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:56.858924  6532 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.252s	user 0.002s	sys 0.000s
I20260709 14:52:56.889060  2603 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:52:56.845757  6549 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 14:52:56.735380  6202 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.894076  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.795579  6182 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:56.896274  2751 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 14:52:56.899758  2607 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:56.892484  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.934026  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.851532  6412 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:56.899219  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:56.930972  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126488us
I20260709 14:52:56.898128  6426 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.863788  6540 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:56.910593  2681 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:56.871325  6438 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:56.918545  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104073us
I20260709 14:52:56.922969  2606 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 14:52:56.889892  6539 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:52:56.912612  6546 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 14:52:56.905692  6480 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:56.939467  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:56.901866  3536 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 14:52:56.920397  6381 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:56.948797  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:56.898907  6466 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 14:52:56.946425  6550 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 14:52:56.969053  6182 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:56.954797  3536 pending_rounds.cc:85] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:52:56.971194  3536 pending_rounds.cc:107] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 14:52:56.971678  3536 raft_consensus.cc:2889] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:56.973489  2683 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:56.914608  2605 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10. Re-sending same reply.
I20260709 14:52:56.956074  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:56.954429  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:56.962710  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:52:56.935868  6305 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:56.923954  6421 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.934228  6478 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:56.967360  6552 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 14:52:56.952983  6548 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:52:56.998476  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:56.949260  6440 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:56.950420  6202 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:56.951013  6535 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:56.954274  2612 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:52:56.964638  6381 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:57.021765  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.034241  6560 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:52:57.034791  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 89698.5us
I20260709 14:52:57.035393  6557 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:57.036309  6372 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:57.036160  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:57.000581  6399 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:57.037050  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:57.003178  6544 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:56.957693  6393 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 14:52:57.058606  2760 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:52:57.045532  6412 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:57.004001  6011 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
W20260709 14:52:57.268591  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:57.096390  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:57.557905  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 659) took 2228 ms. Trace:
W20260709 14:52:57.610483  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 654483us
W20260709 14:52:57.154765  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:57.499744  6505 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.005s
I20260709 14:52:57.009334  6207 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:57.522639  6388 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:57.616191  6466 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:57.485975  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:57.014199  6548 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:57.609905  2759 rpcz_store.cc:276] 0709 14:52:55.329009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.375100 (+ 46091us) service_pool.cc:224] Handling call
0709 14:52:55.375486 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:55.436542 (+ 61056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:57.557463 (+2120921us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:57.557799 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123709184}
I20260709 14:52:57.021574  6440 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:57.673396  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.687644  2760 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:57.038976  6535 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:57.696130  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.085687  6557 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:57.340421  6543 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:57.900311  6543 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:57.898581  2751 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.013s	sys 0.028s
W20260709 14:52:57.902292  2751 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.014s	sys 0.028s
I20260709 14:52:57.300969  6539 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:57.609957  6367 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:57.624817  6540 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:57.252605  6426 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:52:57.705243  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.904597  2760 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:57.906071  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.661233  6207 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:57.907150  6207 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:57.912699  2760 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:57.915477  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.695801  6440 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:57.917933  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.496675  6462 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:57.697901  6305 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:57.923616  6305 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:57.924510  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:57.925514  2760 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:52:57.744905  6564 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 14:52:57.686393  6466 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:57.927131  6426 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:57.929652  6466 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:57.930182  6426 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:57.692979  6548 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:57.716089  6461 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 14:52:57.931902  6461 consensus_queue.cc:1243] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 14:52:57.749337  6566 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 14:52:57.935158  6305 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:57.815991  6363 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.376s	user 0.016s	sys 0.001s
I20260709 14:52:57.701987  6388 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:57.939080  6363 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:52:57.809511  6408 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.093s	user 0.014s	sys 0.000s
I20260709 14:52:57.949209  6408 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:57.962692  6062 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.102s	user 0.011s	sys 0.004s
I20260709 14:52:57.989758  6062 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:57.670459  6505 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:57.993170  6505 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:57.999152  6543 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.005054  6408 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:57.869626  6480 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.912s	user 0.011s	sys 0.000s
I20260709 14:52:57.870409  6535 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:57.903307  6539 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.020851  6539 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:57.715174  6557 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:57.849500  6471 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.941s	user 0.009s	sys 0.005s
I20260709 14:52:57.954355  6363 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:57.956212  6388 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:57.856835  6381 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.004s
I20260709 14:52:58.032661  6381 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.036450  6381 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:57.980532  6202 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.927s	user 0.011s	sys 0.002s
I20260709 14:52:58.055863  6548 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:58.059376  6548 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:57.985664  6445 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.062s	user 0.015s	sys 0.000s
I20260709 14:52:57.986335  6426 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.063189  6426 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:57.864528  6415 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.092s	user 0.014s	sys 0.000s
W20260709 14:52:57.857759  6433 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.001s	user 0.016s	sys 0.000s
I20260709 14:52:58.065822  6433 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.069051  6433 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.072188  6426 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.074471  6445 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:58.077636  6445 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:58.165643  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.000s	sys 0.013s
I20260709 14:52:58.165961  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.167007  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1106) took 1131 ms. Trace:
I20260709 14:52:58.169171  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1308) took 1100 ms. Trace:
I20260709 14:52:58.171224  2684 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 14:52:58.173187  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 560824us
W20260709 14:52:58.168089  2681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.004s	sys 0.010s
W20260709 14:52:58.176224  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.176749  2680 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:57.105329  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:58.179610  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.006s	sys 0.000s
W20260709 14:52:58.180575  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.002s	sys 0.000s
W20260709 14:52:58.190547  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.882s (SENT) [suppressed 26 similar messages]
I20260709 14:52:58.191629  2606 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [9.6-12.7]
W20260709 14:52:58.207787  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:57.697922  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.210016  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1292) took 2150 ms. Trace:
I20260709 14:52:58.210750  2757 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:58.211126  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.213882  2753 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:58.246665  3538 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:58.002594  6207 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:52:58.455008  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 659) took 1399 ms (client timeout 1603 ms). Trace:
W20260709 14:52:58.477689  2612 rpcz_store.cc:269] 0709 14:52:57.055333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.056456 (+  1123us) service_pool.cc:224] Handling call
0709 14:52:58.454716 (+1398260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.088697  6557 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.480254  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:52:58.481432  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1310) took 1411 ms (client timeout 1633 ms). Trace:
I20260709 14:52:58.449222  2609 rpcz_store.cc:276] 0709 14:52:57.068718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.095916 (+ 27198us) service_pool.cc:224] Handling call
0709 14:52:58.168806 (+1072890us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:58.452732  2681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.004s	sys 0.010s
W20260709 14:52:58.483068  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.007s	sys 0.000s
I20260709 14:52:58.483634  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.484438  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1105) took 1485 ms. Trace:
I20260709 14:52:58.484889  2609 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:58.485699  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1311) took 1415 ms. Trace:
I20260709 14:52:58.485041  2681 rpcz_store.cc:276] 0709 14:52:56.998535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.998811 (+   276us) service_pool.cc:224] Handling call
0709 14:52:56.999350 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:56.999708 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:56.999746 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:57.009470 (+  9724us) log.cc:844] Serialized 26 byte log entry
0709 14:52:58.483629 (+1474159us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:58.483721 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.483909 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:58.484000 (+    91us) raft_consensus.cc:1685] finished
0709 14:52:58.484019 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.484306 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1429379,"threads_started":1,"wal-append.queue_time_us":843071}
I20260709 14:52:58.449056  2685 rpcz_store.cc:276] 0709 14:52:57.035236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.035812 (+   576us) service_pool.cc:224] Handling call
0709 14:52:58.166503 (+1130691us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:58.454452  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.003s	sys 0.000s
I20260709 14:52:58.456136  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1306) took 1388 ms. Trace:
I20260709 14:52:58.486346  2609 rpcz_store.cc:276] 0709 14:52:57.070728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.483149 (+1412421us) service_pool.cc:224] Handling call
0709 14:52:58.485607 (+  2458us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:58.457111  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.482004  6557 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:58.455233  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 264907us
W20260709 14:52:58.447813  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.000s	sys 0.014s
W20260709 14:52:58.448801  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1304) took 1382 ms (client timeout 1715 ms). Trace:
W20260709 14:52:58.454694  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.459354  2760 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:52:58.481870  2612 rpcz_store.cc:269] 0709 14:52:57.070437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.479640 (+1409203us) service_pool.cc:224] Handling call
0709 14:52:58.481315 (+  1675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.270560  6062 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:52:58.458386  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 674) took 1464 ms (client timeout 1923 ms). Trace:
I20260709 14:52:58.488297  2606 rpcz_store.cc:276] 0709 14:52:57.067411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.068078 (+   667us) service_pool.cc:224] Handling call
0709 14:52:58.178540 (+1110462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:58.455591 (+277051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.455997 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.361320  6544 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:52:58.491719  2603 rpcz_store.cc:269] 0709 14:52:57.066548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.068186 (+  1638us) service_pool.cc:224] Handling call
0709 14:52:58.448686 (+1380500us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:58.501752  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:58.497941  2759 rpcz_store.cc:269] 0709 14:52:56.993957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.697559 (+703602us) service_pool.cc:224] Handling call
0709 14:52:58.458067 (+760508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.509073  6544 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:58.511545  6363 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:52:58.513123  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.513890  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.515444  2685 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:52:58.459034  6480 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:58.517408  6480 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.457828  2751 rpcz_store.cc:276] 0709 14:52:56.059261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.941545 (+882284us) service_pool.cc:224] Handling call
0709 14:52:56.942099 (+   554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:56.942677 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:56.942721 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:56.943633 (+   912us) log.cc:844] Serialized 52 byte log entry
0709 14:52:57.007051 (+ 63418us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:52:57.902747 (+895696us) log.cc:844] Serialized 14 byte log entry
0709 14:52:57.903033 (+   286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:57.903118 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:58.209209 (+306091us) raft_consensus.cc:1685] finished
0709 14:52:58.209258 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.209667 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":901611,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":212,"raft.queue_time_us":7341,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":328,"thread_start_us":878146,"threads_started":2,"wal-append.queue_time_us":401611}
I20260709 14:52:58.520552  6415 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.521777  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1307) took 1454 ms. Trace:
I20260709 14:52:58.522300  6062 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:58.523283  6062 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:58.527076  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.528825  6388 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:58.529233  6207 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.529711  6388 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:58.531006  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1295) took 2470 ms. Trace:
I20260709 14:52:58.531818  6202 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.532007  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.532135  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 673) took 1552 ms. Trace:
I20260709 14:52:58.514237  2603 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:58.532946  6388 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.532717  6202 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.533581  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.532734  2760 rpcz_store.cc:276] 0709 14:52:56.979839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.926485 (+946646us) service_pool.cc:224] Handling call
0709 14:52:58.532037 (+605552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.536135  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.521780  6415 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:58.522315  2604 rpcz_store.cc:276] 0709 14:52:57.067769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.068314 (+   545us) service_pool.cc:224] Handling call
0709 14:52:57.101790 (+ 33476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:57.102092 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:57.102441 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 14:52:58.520970 (+1418529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.521103 (+   133us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:58.521227 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.521338 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.521664 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":329,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":4175,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":229,"thread_start_us":1388959,"threads_started":1,"wal-append.queue_time_us":773079}
I20260709 14:52:58.520594  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.531616  2759 rpcz_store.cc:276] 0709 14:52:56.060051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.523270 (+2463219us) service_pool.cc:224] Handling call
0709 14:52:58.523959 (+   689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.526277 (+  2318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:58.526441 (+   164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.526674 (+   233us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:58.527250 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.527353 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.530905 (+  3552us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":742,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":332}
I20260709 14:52:58.516402  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1109) took 1479 ms. Trace:
I20260709 14:52:58.547375  2685 rpcz_store.cc:276] 0709 14:52:57.036808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.513127 (+1476319us) service_pool.cc:224] Handling call
0709 14:52:58.516248 (+  3121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.452365  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:58.520553  6480 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:58.442803  6440 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.003s
W20260709 14:52:58.561806  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:58.564831  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112776us
I20260709 14:52:58.567448  6557 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:52:58.567448  6381 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:58.568717  6381 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:58.569152  6381 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:58.569080  6557 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:52:58.559989  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.575791  6544 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:58.592340  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.589732  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.597092  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:58.599524  2751 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:58.573266  6557 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:58.608526  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1313) took 1535 ms. Trace:
I20260709 14:52:58.611892  2609 rpcz_store.cc:276] 0709 14:52:57.071351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.507497 (+1436146us) service_pool.cc:224] Handling call
0709 14:52:58.606585 (+ 99088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.612386  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.611600  6062 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.615406  6462 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.616086  6468 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.616441  6462 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.616714  6363 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:58.616884  6468 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:58.617905  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51530.3us
I20260709 14:52:58.618637  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1016) took 2228 ms. Trace:
I20260709 14:52:58.619890  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:58.621203  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.605752  6393 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.627102  6202 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.627502  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.613983  2612 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:58.632454  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.633731  3547 rpcz_store.cc:276] 0709 14:52:56.382917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.482863 (+2099946us) service_pool.cc:224] Handling call
0709 14:52:58.573571 (+ 90708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.610951 (+ 37380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:58.610982 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.611065 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:58.611151 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.611234 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.611414 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:58.637790  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 175927us
I20260709 14:52:58.638319  2685 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:58.639034  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:58.639917  3547 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:52:58.639953  3412 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.572s (SENT)
I20260709 14:52:58.640991  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:58.643349  2609 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:52:58.643616  6207 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:58.644841  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.586282  6440 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:58.648119  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.610998  6356 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:58.656723  6541 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:58.630643  6363 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.684762  6455 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:58.634514  6393 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.691102  6393 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.668779  6356 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.671662  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 660) took 2206 ms. Trace:
I20260709 14:52:58.672611  2609 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:52:58.672792  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 672) took 2526 ms (client timeout 2195 ms). Trace:
I20260709 14:52:58.674023  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1107) took 1638 ms. Trace:
I20260709 14:52:58.676373  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.677258  6440 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:58.677512  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1303) took 2223 ms (client timeout 2793 ms). Trace:
I20260709 14:52:58.648147  6415 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.679632  6353 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:58.679826  2603 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:58.684981  6363 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.696978  6468 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.697808  2604 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:58.665839  6541 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:58.661703  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1289) took 2603 ms. Trace:
I20260709 14:52:58.666254  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.663724  2606 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:58.700671  2680 rpcz_store.cc:276] 0709 14:52:56.465458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.465607 (+   149us) service_pool.cc:224] Handling call
0709 14:52:56.466000 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.453314 (+1987314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.671112 (+217798us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:58.671523 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3932049280}
I20260709 14:52:58.709214  6438 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.698648  2759 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:52:58.690514  2605 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:58.701591  2684 rpcz_store.cc:276] 0709 14:52:57.035914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.036711 (+   797us) service_pool.cc:224] Handling call
0709 14:52:58.673739 (+1637028us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.673943 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1272433024}
W20260709 14:52:58.716012  2753 rpcz_store.cc:269] 0709 14:52:56.454670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.525603 (+ 70933us) service_pool.cc:224] Handling call
0709 14:52:58.677246 (+2151643us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.677444 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582904576}
W20260709 14:52:58.701160  2757 rpcz_store.cc:269] 0709 14:52:56.145914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.425013 (+279099us) service_pool.cc:224] Handling call
0709 14:52:58.672464 (+2247451us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.672680 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3692843776}
I20260709 14:52:58.729467  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.727522  2754 rpcz_store.cc:276] 0709 14:52:56.058459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.826711 (+768252us) service_pool.cc:224] Handling call
0709 14:52:58.659435 (+1832724us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:52:58.659891 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.660939 (+  1048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:58.660982 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.661085 (+   103us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:52:58.661144 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.661228 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.661437 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19859200}
I20260709 14:52:58.734556  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.736173  3547 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:58.732857  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1298) took 2654 ms. Trace:
W20260709 14:52:58.736769  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101445us
I20260709 14:52:58.737318  2759 rpcz_store.cc:276] 0709 14:52:56.076350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.549888 (+2473538us) service_pool.cc:224] Handling call
0709 14:52:58.612096 (+ 62208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.728947 (+116851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.730739 (+  1792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.737903  2612 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:52:58.754489  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.756134  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 656) took 1831 ms. Trace:
I20260709 14:52:58.738951  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:58.738543  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.738134  2607 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:58.763492  2606 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:58.738200  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 652) took 1859 ms. Trace:
I20260709 14:52:58.762629  2609 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:52:58.768682  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:58.771337  2753 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.771633  6578 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.747090  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:58.643661  6557 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.764228  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.772403  6578 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:52:58.776885  6535 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.008s
I20260709 14:52:58.777693  6578 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:58.777524  6535 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.779675  2680 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 14:52:58.780493  6531 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.502s	user 0.018s	sys 0.034s
I20260709 14:52:58.780995  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.781728  2680 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:58.782087  6369 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:58.782754  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:58.736078  6415 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.796602  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:58.796862  6415 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.797593  3542 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:58.782552  6369 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:52:58.799307  2757 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:58.757020  6541 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.801808  6541 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:58.785699  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.786230  2759 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.787235  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:58.787742  2754 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:58.791141  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.794082  6366 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:52:58.794657  2751 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:58.795118  6535 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.795934  6557 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:58.815124  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1305) took 1748 ms. Trace:
I20260709 14:52:58.767198  2611 rpcz_store.cc:276] 0709 14:52:56.865193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.865865 (+   672us) service_pool.cc:224] Handling call
0709 14:52:56.895104 (+ 29239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:56.895494 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:56.895522 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:56.905432 (+  9910us) log.cc:844] Serialized 26 byte log entry
0709 14:52:58.689002 (+1783570us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b1800037b40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1
0709 14:52:58.689458 (+   456us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:58.689522 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.689646 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:58.689744 (+    98us) raft_consensus.cc:1685] finished
0709 14:52:58.689756 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.724468 (+ 34712us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600,"raft.queue_time_us":2953,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12321,"spinlock_wait_cycles":10692992,"thread_start_us":12793,"threads_started":1,"wal-append.queue_time_us":40233,"wal-append.run_cpu_time_us":2562,"wal-append.run_wall_time_us":1016933}
I20260709 14:52:58.815740  2605 rpcz_store.cc:276] 0709 14:52:57.067006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.067967 (+   961us) service_pool.cc:224] Handling call
0709 14:52:58.690014 (+1622047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.736882 (+ 46868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.814815 (+ 77933us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400128250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.815044 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3601431296}
I20260709 14:52:58.816531  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.817252  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.817934  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.748965  6462 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.818704  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:52:58.769881  2651 connection.cc:256] RPC call timeout handler was delayed by -1.2553763660000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 14:52:58.807875  3547 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:52:58.738345  6111 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.809412  6381 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.760825  2608 rpcz_store.cc:276] 0709 14:52:56.868441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.911511 (+ 43070us) service_pool.cc:224] Handling call
0709 14:52:56.914651 (+  3140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.914939 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:56.923767 (+  8828us) log.cc:844] Serialized 14 byte log entry
0709 14:52:58.606785 (+1683018us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.629881 (+ 23096us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:58.665732 (+ 35851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.699623 (+ 33891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.699859 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16686,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":36251,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":219,"spinlock_wait_cycles":36736,"thread_start_us":286820,"threads_started":2,"wal-append.queue_time_us":835319}
I20260709 14:52:58.823947  2607 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:58.818727  2611 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:52:58.826179  2685 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:52:58.825685  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:58.831975  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.834365  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:58.835331  3547 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:58.837728  2681 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:52:58.838210  6480 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.807186  6369 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:58.838685  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:58.839186  3412 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.766s (SENT)
I20260709 14:52:58.839754  6369 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.839828  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:58.840372  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.841122  6535 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:52:58.841904  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.842959  2677 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 14:52:58.843384  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 876) took 4436 ms (client timeout 2998 ms). Trace:
I20260709 14:52:58.843959  6398 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:52:58.843892  3546 rpcz_store.cc:269] 0709 14:52:54.407017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.407165 (+   148us) service_pool.cc:224] Handling call
0709 14:52:58.842920 (+4435755us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54004b3cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.843125 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3110945664}
I20260709 14:52:58.844622  6398 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.845649  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:52:58.844417  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1154) took 2709 ms (client timeout 2244 ms). Trace:
W20260709 14:52:58.847191  2651 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.868s (SENT)
W20260709 14:52:58.846935  3537 rpcz_store.cc:269] 0709 14:52:56.134862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.135933 (+  1071us) service_pool.cc:224] Handling call
0709 14:52:58.844053 (+2708120us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.844208 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759740160}
I20260709 14:52:58.848037  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:58.848603  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.849548  3537 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:58.849964  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1164) took 2520 ms. Trace:
I20260709 14:52:58.850394  3541 rpcz_store.cc:276] 0709 14:52:56.329621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.329909 (+   288us) service_pool.cc:224] Handling call
0709 14:52:56.330363 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.469706 (+139343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.849456 (+2379750us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b5400550354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:58.849752 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127150464}
I20260709 14:52:58.851197  6353 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:52:58.851475  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 661) took 3512 ms (client timeout 1551 ms). Trace:
I20260709 14:52:58.852111  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:58.853278  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1178) took 3418 ms (client timeout 1726 ms). Trace:
I20260709 14:52:58.853992  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.848631  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1020) took 2464 ms. Trace:
I20260709 14:52:58.846489  3546 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:52:58.855334  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1180) took 2956 ms (client timeout 2229 ms). Trace:
I20260709 14:52:58.878051  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:58.879221  2754 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:58.879849  3541 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:58.881271  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.882095  3541 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:58.884117  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:58.884732  2603 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:58.886150  6574 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:58.888674  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:58.830956  2605 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:52:58.891249  2757 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:58.892385  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.893422  2757 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:52:58.894934  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.895387  6353 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:58.912204  6353 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.913332  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:58.877720  2679 rpcz_store.cc:269] 0709 14:52:55.898592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.898806 (+   214us) service_pool.cc:224] Handling call
0709 14:52:58.842755 (+2943949us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.855103 (+ 12348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658189056}
I20260709 14:52:58.930269  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.903296  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.905848  6385 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:58.922958  2757 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:52:58.907871  2652 leader_election.cc:336] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.715s (SENT)
W20260709 14:52:58.908120  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.908327  6398 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.951002  2759 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:58.850732  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.951666  6574 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:58.863807  2751 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:58.855341  3536 rpcz_store.cc:276] 0709 14:52:56.384422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.972998 (+588576us) service_pool.cc:224] Handling call
0709 14:52:58.846989 (+1873991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.847342 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:58.847372 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.847437 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:52:58.847467 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.847525 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.848259 (+   734us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54004b3cd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.848390 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4202514304}
I20260709 14:52:58.850899  2611 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:58.909722  6415 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:58.886307  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.915218  6353 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.836225  6381 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:52:58.871125  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:58.890489  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.907097  6399 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:52:58.946590  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:58.931665  6385 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:52:58.918186  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:58.961866  2685 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
W20260709 14:52:58.853878  2756 rpcz_store.cc:269] 0709 14:52:55.434381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.439490 (+  5109us) service_pool.cc:224] Handling call
0709 14:52:58.852692 (+3413202us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.853013 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2608885632}
I20260709 14:52:58.955308  2753 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:58.958743  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1309) took 1883 ms. Trace:
I20260709 14:52:58.959200  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:58.962590  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.964483  2611 rpcz_store.cc:276] 0709 14:52:57.070138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.850044 (+1779906us) service_pool.cc:224] Handling call
0709 14:52:58.850537 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.953206 (+102669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.953429 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.955061  2754 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:58.957875  3547 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:58.960568  2605 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:52:58.966236  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1296) took 2899 ms. Trace:
W20260709 14:52:58.851946  2758 rpcz_store.cc:269] 0709 14:52:55.339079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.340746 (+  1667us) service_pool.cc:224] Handling call
0709 14:52:58.851065 (+3510319us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b54002bf7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:58.851259 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227387136}
I20260709 14:52:58.951556  2679 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:58.966727  3542 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:58.956785  2604 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:58.968288  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:58.969056  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:58.970468  2751 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 8: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:58.922624  2431 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 5 to 11, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:52:58.971464  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:58.972241  2679 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.974359  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:58.978013  2758 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:52:58.838860  6480 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:58.980808  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:58.982187  6480 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:58.960417  6399 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:58.983845  6399 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:58.985425  6399 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:58.986121  2609 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:58.988150  6414 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:52:58.988735  6414 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:58.989974  2611 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:58.978958  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1182) took 2677 ms. Trace:
I20260709 14:52:58.969350  3537 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 6.
I20260709 14:52:58.991593  2758 rpcz_store.cc:276] 0709 14:52:56.301876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.976947 (+2675071us) service_pool.cc:224] Handling call
0709 14:52:58.977499 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.978553 (+  1054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.978871 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:58.991753  3541 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:59.000263  2752 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:59.001924  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:52:59.002678  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114244us
I20260709 14:52:59.003805  2754 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: []
I20260709 14:52:59.007568  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1181) took 2714 ms. Trace:
I20260709 14:52:59.007710  2685 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:59.008253  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:59.008665  2754 rpcz_store.cc:276] 0709 14:52:56.290538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.992836 (+2702298us) service_pool.cc:224] Handling call
0709 14:52:59.003649 (+ 10813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.004229 (+   580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.004396 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.016952  6381 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:59.018399  2752 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:59.019562  6385 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:52:59.021510  6385 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:59.022198  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 656) took 5247 ms. Trace:
I20260709 14:52:59.025252  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:59.028544  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.022655  2677 rpcz_store.cc:276] 0709 14:52:53.774318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:54.256184 (+481866us) service_pool.cc:224] Handling call
0709 14:52:54.256573 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.843397 (+4586824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.021786 (+178389us) spinlock_profiling.cc:243] Waited 4.59 s on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:59.022107 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10546068736}
W20260709 14:52:59.032923  3543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 866) took 6062 ms (client timeout 2078 ms). Trace:
W20260709 14:52:59.033561  3543 rpcz_store.cc:269] 0709 14:52:52.971486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:52.971628 (+   142us) service_pool.cc:224] Handling call
0709 14:52:59.032459 (+6060831us) spinlock_profiling.cc:243] Waited 3.09 s on lock 0x7b54004b41d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.032689 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7115039488}
I20260709 14:52:59.035866  2758 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:59.038535  2609 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:58.990008  6414 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:59.042145  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1302) took 2630 ms. Trace:
I20260709 14:52:59.042523  2758 rpcz_store.cc:276] 0709 14:52:56.411848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.995595 (+2583747us) service_pool.cc:224] Handling call
0709 14:52:59.026763 (+ 31168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.041903 (+ 15140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.042078 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.039264  2609 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:52:59.040926  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.044103  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.040779  2751 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 14:52:59.050424  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.006294  6480 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 14:52:59.055630  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1304) took 2595 ms. Trace:
I20260709 14:52:59.056104  2751 rpcz_store.cc:276] 0709 14:52:56.460155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.999003 (+2538848us) service_pool.cc:224] Handling call
0709 14:52:59.035388 (+ 36385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:59.055362 (+ 19974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.055571 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.057621  3545 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: []
W20260709 14:52:59.057926  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150107us
I20260709 14:52:59.060802  3540 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:59.061685  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1015) took 2679 ms. Trace:
I20260709 14:52:59.062043  3540 rpcz_store.cc:276] 0709 14:52:56.382501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.059829 (+2677328us) service_pool.cc:224] Handling call
0709 14:52:59.060378 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.061361 (+   983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.061599 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.065511  6414 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:59.068010  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.069211  2684 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.072535  2680 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:59.070508  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.075147  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:59.075739  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.083545  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.089787  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.091719  3536 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8.
I20260709 14:52:59.094430  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.096416  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.099081  3536 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:52:59.100354  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.103914  3536 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.104389  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.106336  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:59.107116  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.108815  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.109910  3537 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.111207  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.111410  2677 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:52:59.111935  3537 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:52:59.107429  2751 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:59.090956  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.112622  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.091074  3543 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:52:59.120007  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.120808  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.121264  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.123663  3546 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:52:59.114516  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.221937  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.128614  2684 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.216468  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.250789  3537 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:52:59.251139  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.218207  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:52:59.229593  2679 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:52:59.185277  2685 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:52:59.166066  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.222939  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.252157  2603 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 8. Re-sending same reply.
I20260709 14:52:59.253257  2751 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:59.252966  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.256181  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1187) took 1016 ms. Trace:
I20260709 14:52:59.248204  3540 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.258191  2679 rpcz_store.cc:276] 0709 14:52:58.237055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.971097 (+734042us) service_pool.cc:224] Handling call
0709 14:52:59.253612 (+282515us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.256846  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:52:59.171936  6392 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:59.264489  2684 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:52:59.265738  3543 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:59.124748  2609 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:59.268635  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.274768  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:59.275406  3537 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.276865  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1028) took 2617 ms. Trace:
I20260709 14:52:59.277396  3543 rpcz_store.cc:276] 0709 14:52:56.656078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.161604 (+2505526us) service_pool.cc:224] Handling call
0709 14:52:59.245855 (+ 84251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.272522 (+ 26667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.272831 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.278141  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.282369  2679 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.090986  6202 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.184s
I20260709 14:52:59.263300  6392 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:59.291402  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.325698  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.330461  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46802.9us
I20260709 14:52:59.329913  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:59.293350  3543 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:59.298957  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1029) took 2617 ms. Trace:
I20260709 14:52:59.352677  3536 rpcz_store.cc:276] 0709 14:52:56.671784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.256096 (+2584312us) service_pool.cc:224] Handling call
0709 14:52:59.284840 (+ 28744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.288857 (+  4017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:59.288897 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:59.288963 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:59.289005 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.289089 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.289255 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.356269  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.356726  2609 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:59.300276  3546 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:52:59.363867  3536 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:52:59.365671  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1300) took 3220 ms. Trace:
I20260709 14:52:59.367451  2684 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.379885  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1188) took 1142 ms. Trace:
I20260709 14:52:59.380414  2684 rpcz_store.cc:276] 0709 14:52:58.237758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.292175 (+1054417us) service_pool.cc:224] Handling call
0709 14:52:59.379769 (+ 87594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.381737  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.381753  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.382081  6394 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:59.366928  3543 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:52:59.371248  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.382891  3536 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.314216  3537 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 14:52:59.377507  2757 rpcz_store.cc:276] 0709 14:52:56.145050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.962019 (+2816969us) service_pool.cc:224] Handling call
0709 14:52:58.999140 (+ 37121us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:59.052005 (+ 52865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:59.052060 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:59.063361 (+ 11301us) log.cc:844] Serialized 52 byte log entry
0709 14:52:59.164309 (+100948us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:59.164371 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.164473 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:59.365304 (+200831us) raft_consensus.cc:1685] finished
0709 14:52:59.365329 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.365563 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4309,"raft.queue_time_us":16262,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19950,"thread_start_us":69974,"threads_started":1,"wal-append.queue_time_us":299390}
I20260709 14:52:59.382604  2684 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:52:59.393625  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.395954  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1168) took 2710 ms. Trace:
I20260709 14:52:59.396863  3546 rpcz_store.cc:276] 0709 14:52:56.685103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.260551 (+2575448us) service_pool.cc:224] Handling call
0709 14:52:59.281460 (+ 20909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.394374 (+112914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.395604 (+  1230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.401338  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:59.402884  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 183624us
I20260709 14:52:59.403604  3540 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 14:52:59.406541  2679 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.412432  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 674) took 1212 ms. Trace:
I20260709 14:52:59.406216  3536 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:59.413666  2679 rpcz_store.cc:276] 0709 14:52:58.199751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.350102 (+1150351us) service_pool.cc:224] Handling call
0709 14:52:59.412336 (+ 62234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.417279  3540 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 14:52:59.418533  2609 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 14:52:59.419386  2609 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:52:59.419713  2609 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 14:52:59.420065  2609 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:59.420312  6392 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:59.421868  6392 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:59.430401  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.382727  6394 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:59.436187  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.453090  3540 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:59.453833  2677 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 14:52:59.456883  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 114318us
W20260709 14:52:59.462129  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.462913  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.463500  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.457548  3409 connection.cc:256] RPC call timeout handler was delayed by -1.8147667519999686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 14:52:59.467823  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.467422  3409 connection.cc:256] RPC call timeout handler was delayed by -1.6896667519999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 14:52:59.469388  3409 connection.cc:256] RPC call timeout handler was delayed by -1.4135040989999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2431s for a response
W20260709 14:52:59.469679  3409 connection.cc:256] RPC call timeout handler was delayed by -1.2650210520000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
I20260709 14:52:59.477464  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:59.480199  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.483590  2575 connection.cc:256] RPC call timeout handler was delayed by -2.0004902429999447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30820000000000003s for a response
W20260709 14:52:59.484115  2575 connection.cc:256] RPC call timeout handler was delayed by -1.9201062300001013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 14:52:59.485338  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:59.487280  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:59.487957  2679 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:52:59.491160  6354 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:59.496691  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1180) took 3215 ms. Trace:
I20260709 14:52:59.497833  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1312) took 2426 ms. Trace:
I20260709 14:52:59.498353  2608 rpcz_store.cc:276] 0709 14:52:57.071063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.855550 (+1784487us) service_pool.cc:224] Handling call
0709 14:52:58.887504 (+ 31954us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:52:58.888191 (+   687us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:52:58.931167 (+ 42976us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.075731 (+144564us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.097201 (+ 21470us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:52:59.098031 (+   830us) log.cc:844] Serialized 52 byte log entry
0709 14:52:59.098231 (+   200us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:59.098274 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.098383 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:59.497527 (+399144us) raft_consensus.cc:1685] finished
0709 14:52:59.497550 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.497730 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26735,"prepare.queue_time_us":133219,"prepare.run_cpu_time_us":161,"prepare.run_wall_time_us":9378,"raft.queue_time_us":22285,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":10474,"spinlock_wait_cycles":31805696,"thread_start_us":149296,"threads_started":2,"wal-append.queue_time_us":488965}
I20260709 14:52:59.503656  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.497298  2753 rpcz_store.cc:276] 0709 14:52:56.281202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.992788 (+2711586us) service_pool.cc:224] Handling call
0709 14:52:59.001744 (+  8956us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 14:52:59.032755 (+ 31011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:59.033350 (+   595us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 14:52:59.064177 (+ 30827us) log.cc:844] Serialized 104 byte log entry
0709 14:52:59.089966 (+ 25789us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:52:59.090348 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.091900 (+  1552us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.495844 (+403944us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b18003116e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:59.496122 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.496225 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:59.496366 (+   141us) raft_consensus.cc:1685] finished
0709 14:52:59.496394 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.496589 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3251,"prepare.queue_time_us":1477,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":120,"raft.queue_time_us":419685,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":163,"spinlock_wait_cycles":120569344,"thread_start_us":25186,"threads_started":1,"wal-append.queue_time_us":247473}
W20260709 14:52:59.506294  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 415454us
I20260709 14:52:59.507500  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:59.508569  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.510375  6202 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:59.510636  6495 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:52:59.511089  6202 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:59.511400  6495 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:52:59.512602  6495 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:59.512625  6202 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:59.511142  6480 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:52:59.510707  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.514590  6414 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:52:59.530854  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 308998us
I20260709 14:52:59.531912  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1301) took 3264 ms. Trace:
W20260709 14:52:59.536834  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.531688  6414 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:59.538198  6414 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:59.538235  6074 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:52:59.538856  6074 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:59.539430  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.539917  6074 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:59.540750  6541 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:52:59.541262  6541 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:52:59.541733  6541 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:52:59.545288  3411 connection.cc:256] RPC call timeout handler was delayed by -1.4462275249999266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 14:52:59.545483  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:52:59.545890  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1359908049998921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 14:52:58.966727  2760 rpcz_store.cc:276] 0709 14:52:56.060305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.535886 (+2475581us) service_pool.cc:224] Handling call
0709 14:52:58.536549 (+   663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.541940 (+  5391us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:58.724523 (+182583us) log.cc:844] Serialized 14 byte log entry
0709 14:52:58.954077 (+229554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:58.954137 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:58.959664 (+  5527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:58.959763 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:58.959943 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":155067,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":78,"raft.queue_time_us":10378,"raft.run_wall_time_us":9405,"spinlock_wait_cycles":114120192,"thread_start_us":264370,"threads_started":2,"wal-append.queue_time_us":192515}
I20260709 14:52:59.546531  2757 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:59.537777  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.549086  2752 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:52:59.547297  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.551350  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.546718  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.536593  2759 rpcz_store.cc:276] 0709 14:52:56.267966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.992152 (+2724186us) service_pool.cc:224] Handling call
0709 14:52:59.007331 (+ 15179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.023057 (+ 15726us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:52:59.023509 (+   452us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.531363 (+507854us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:59.531465 (+   102us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:52:59.531554 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.531648 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.531831 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":301024,"prepare.queue_time_us":5764,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":5698,"raft.queue_time_us":304345,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"thread_start_us":205412,"threads_started":2,"wal-append.queue_time_us":422548}
I20260709 14:52:59.527441  6598 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:52:59.535020  6399 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:59.565776  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.569664  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.563868  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.547379  6480 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:52:59.572858  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:59.574229  2751 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:59.569235  6074 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:59.572683  6480 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:59.577709  6385 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:52:59.578327  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.594463  2751 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:52:59.579252  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:59.580150  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72736.4us
I20260709 14:52:59.581087  6551 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:52:59.582999  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:59.587700  2686 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:52:59.588341  6582 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.658s	user 0.006s	sys 0.000s
I20260709 14:52:59.589102  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1097) took 2938 ms. Trace:
I20260709 14:52:59.568270  6399 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:52:59.591724  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45153.2us
I20260709 14:52:59.595624  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:59.597162  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 679) took 1390 ms (client timeout 1589 ms). Trace:
I20260709 14:52:59.596292  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.600592  6598 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:52:59.614109  2681 rpcz_store.cc:269] 0709 14:52:58.201942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.609549 (+407607us) service_pool.cc:224] Handling call
0709 14:52:59.591326 (+981777us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540012c854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.592611 (+  1285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400319360}
I20260709 14:52:59.600780  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:52:59.598977  2755 raft_consensus.cc:2376] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11. Re-sending same reply.
I20260709 14:52:59.599004  6512 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:59.601377  2752 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:59.601415  6551 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.593369  6394 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:52:59.604298  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1104) took 2671 ms (client timeout 2552 ms). Trace:
W20260709 14:52:59.604691  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1108) took 2568 ms (client timeout 1955 ms). Trace:
I20260709 14:52:59.605460  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.573743  6542 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:52:59.607111  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 124081us
I20260709 14:52:59.609119  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 669) took 1429 ms. Trace:
I20260709 14:52:59.610586  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1152) took 3476 ms. Trace:
I20260709 14:52:59.599169  2678 rpcz_store.cc:276] 0709 14:52:56.650410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.806617 (+156207us) service_pool.cc:224] Handling call
0709 14:52:56.807046 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:56.899083 (+ 92037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.536323 (+2637240us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b54003435d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:59.588851 (+ 52528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120052608}
I20260709 14:52:59.617003  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.614022  2679 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
W20260709 14:52:59.667573  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.619719  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.617283  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.617003  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:52:59.620422  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81102.2us
I20260709 14:52:59.684908  2754 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:59.595469  6480 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:59.721009  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.724958  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.735110  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 171389us
I20260709 14:52:59.721597  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.695763  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.723315  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:59.657788  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1014) took 3275 ms. Trace:
I20260709 14:52:59.737417  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.737596  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.738230  2685 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:52:59.723905  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.736637  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.731833  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.684976  3545 rpcz_store.cc:276] 0709 14:52:56.134261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.134439 (+   178us) service_pool.cc:224] Handling call
0709 14:52:59.054321 (+2919882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.058162 (+  3841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.610310 (+552148us) spinlock_profiling.cc:243] Waited 2.88 s on lock 0x7b54004b41d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.610502 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6620894592}
I20260709 14:52:59.685879  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.721081  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.696661  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79496us
I20260709 14:52:59.734129  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.682793  2680 rpcz_store.cc:276] 0709 14:52:58.180372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.782454 (+602082us) service_pool.cc:224] Handling call
0709 14:52:59.608807 (+826353us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.609024 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662052096}
I20260709 14:52:59.741232  3542 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.739133  3540 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:59.722873  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:52:59.681975  2686 rpcz_store.cc:269] 0709 14:52:57.036407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.437984 (+401577us) service_pool.cc:224] Handling call
0709 14:52:59.604447 (+2166463us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.604599 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4427621120}
W20260709 14:52:59.681699  2683 rpcz_store.cc:269] 0709 14:52:56.932455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.933429 (+   974us) service_pool.cc:224] Handling call
0709 14:52:59.603823 (+2670394us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:52:59.604081 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33559936}
I20260709 14:52:59.742343  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:59.637416  6599 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 14:52:59.736519  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.615353  6399 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:59.752219  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.629302  6602 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:59.684010  6551 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:52:59.736785  3538 rpcz_store.cc:276] 0709 14:52:56.382019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:56.835892 (+453873us) service_pool.cc:224] Handling call
0709 14:52:56.874362 (+ 38470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.460310 (+1585948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.657255 (+1196945us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54004b64d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:52:59.657669 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3153990400}
I20260709 14:52:59.723538  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.752581  3537 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:52:59.692848  6385 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:52:59.698028  6593 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.386s	user 0.002s	sys 0.000s
I20260709 14:52:59.614773  6598 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:52:59.766762  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.752306  3546 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:52:59.683817  6354 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:52:59.782688  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.784150  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98367us
W20260709 14:52:59.786092  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:52:59.786686  2577 connection.cc:256] RPC call timeout handler was delayed by -1.6763867029999346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 14:52:59.772694  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:59.787096  2577 connection.cc:256] RPC call timeout handler was delayed by -1.2398867030000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2698s for a response
I20260709 14:52:59.753126  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:52:59.769353  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:52:59.789476  2577 connection.cc:256] RPC call timeout handler was delayed by -1.2011932420000448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2715s for a response
I20260709 14:52:59.752766  6574 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:52:59.769088  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:59.694024  6428 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 14:52:59.662699  6414 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:59.716061  2431 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 5 to 7, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:52:59.788267  6598 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:52:59.788404  6354 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.795341  6354 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:52:59.744057  6542 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:59.623448  6477 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:52:59.769186  6392 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:52:59.801666  2685 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:52:59.801642  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64191.3us
I20260709 14:52:59.705492  6512 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:59.772060  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.767136  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.789983  2679 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:59.790795  6574 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.697624  6440 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 14:52:59.769811  6602 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.752321  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:52:59.769886  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:52:59.820271  2680 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:52:59.799134  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.786197  6606 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:52:59.808072  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 671) took 1626 ms. Trace:
I20260709 14:52:59.823604  6606 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.823650  2683 rpcz_store.cc:276] 0709 14:52:58.181783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.750376 (+1568593us) service_pool.cc:224] Handling call
0709 14:52:59.807970 (+ 57594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.824424  6606 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:52:59.785166  6385 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:59.825707  6385 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:59.772548  2678 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:52:59.828079  6610 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:52:59.817782  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.822561  2680 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 14:52:59.830358  2680 pending_rounds.cc:85] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 4
I20260709 14:52:59.830684  2680 pending_rounds.cc:107] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 14:52:59.830998  2680 raft_consensus.cc:2889] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:52:59.832126  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1111) took 2398 ms. Trace:
I20260709 14:52:59.832502  2680 rpcz_store.cc:276] 0709 14:52:57.433971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.821364 (+2387393us) service_pool.cc:224] Handling call
0709 14:52:59.821991 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.831744 (+  9753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.832021 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.828097  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1185) took 1654 ms. Trace:
I20260709 14:52:59.834235  2678 rpcz_store.cc:276] 0709 14:52:58.173901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.734809 (+1560908us) service_pool.cc:224] Handling call
0709 14:52:59.828017 (+ 93208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.835608  2678 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:52:59.836309  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 675) took 1636 ms. Trace:
I20260709 14:52:59.836640  2678 rpcz_store.cc:276] 0709 14:52:58.200186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.835020 (+1634834us) service_pool.cc:224] Handling call
0709 14:52:59.835402 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.836014 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.836217 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:52:59.833868  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.839618  2678 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:52:59.839774  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.840621  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 678) took 1639 ms. Trace:
I20260709 14:52:59.840799  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:52:59.841233  2678 rpcz_store.cc:276] 0709 14:52:58.201448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.837441 (+1635993us) service_pool.cc:224] Handling call
0709 14:52:59.839218 (+  1777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.840232 (+  1014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.840534 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.818094  2754 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:52:59.843194  2678 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 14:52:59.844996  2677 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:52:59.846351  2677 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:52:59.847154  6495 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:59.847623  6495 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.847973  6495 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:59.848776  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.849785  3536 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:52:59.818913  6574 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:52:59.853435  2755 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:52:59.854318  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55333.4us
I20260709 14:52:59.851437  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1189) took 1613 ms. Trace:
I20260709 14:52:59.857030  2686 rpcz_store.cc:276] 0709 14:52:58.238218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.745062 (+1506844us) service_pool.cc:224] Handling call
0709 14:52:59.851360 (+106298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.842525  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.851143  6602 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:59.853804  2685 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:52:59.858342  6074 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 14:52:59.863549  6394 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:52:59.863842  6471 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:52:59.864039  6394 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:59.864385  6471 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:52:59.864879  6471 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:52:59.861639  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.867777  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.870353  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1110) took 2541 ms. Trace:
I20260709 14:52:59.871409  2685 rpcz_store.cc:276] 0709 14:52:57.320272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.829578 (+2509306us) service_pool.cc:224] Handling call
0709 14:52:59.853590 (+ 24012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:59.861131 (+  7541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.861345 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:52:59.871590  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1318) took 1309 ms. Trace:
I20260709 14:52:59.872865  2604 rpcz_store.cc:276] 0709 14:52:58.562552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.994034 (+431482us) service_pool.cc:224] Handling call
0709 14:52:58.994371 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:52:58.996920 (+  2549us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:52:59.079834 (+ 82914us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.281206 (+201372us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.281720 (+   514us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.446563 (+164843us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b180039d720. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:52:59.870727 (+424164us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b180039d720. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:52:59.871017 (+   290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:52:59.871097 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:59.871178 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.871298 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.871500 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7722,"prepare.queue_time_us":79665,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":161,"raft.queue_time_us":1955,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":13562,"spinlock_wait_cycles":51766912,"thread_start_us":146603,"threads_started":2,"wal-append.queue_time_us":245923}
W20260709 14:52:59.875530  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:52:59.878016  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.854724  6392 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:52:59.877563  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:52:59.847060  6613 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:52:59.881639  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:52:59.879079  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.887161  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 679) took 2891 ms. Trace:
I20260709 14:52:59.870280  6428 raft_consensus.cc:2764] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 14:52:59.889271  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:52:59.891541  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94303us
W20260709 14:52:59.897926  2575 connection.cc:256] RPC call timeout handler was delayed by -1.6870140210000955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25520000000000004s for a response
W20260709 14:52:59.899358  2575 connection.cc:256] RPC call timeout handler was delayed by -1.6203666059999478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31370000000000003s for a response
I20260709 14:52:59.882817  6074 raft_consensus.cc:2764] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 14:52:59.899784  2575 connection.cc:256] RPC call timeout handler was delayed by -1.5265653019999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2898s for a response
W20260709 14:52:59.906725  2575 connection.cc:256] RPC call timeout handler was delayed by -1.1560498759999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 14:52:59.907359  2575 connection.cc:256] RPC call timeout handler was delayed by -1.0021343370000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 14:52:59.898413  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:52:59.887589  2756 rpcz_store.cc:276] 0709 14:52:56.995602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:58.998277 (+2002675us) service_pool.cc:224] Handling call
0709 14:52:59.008377 (+ 10100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:52:59.013064 (+  4687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:52:59.013089 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:52:59.071280 (+ 58191us) log.cc:844] Serialized 26 byte log entry
0709 14:52:59.227823 (+156543us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:52:59.400792 (+172969us) log.cc:844] Serialized 14 byte log entry
0709 14:52:59.515072 (+114280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:52:59.515161 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:52:59.697237 (+182076us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:52:59.880272 (+183035us) raft_consensus.cc:1685] finished
0709 14:52:59.886964 (+  6692us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:52:59.887106 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113376,"prepare.queue_time_us":204810,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2377,"raft.queue_time_us":132981,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":8865,"spinlock_wait_cycles":39001216,"thread_start_us":248094,"threads_started":2,"wal-append.queue_time_us":240209}
I20260709 14:52:59.882150  6610 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:52:59.888622  6369 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:59.878118  6354 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:59.912937  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:52:59.907469  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.890081  6605 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:59.897893  6392 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:59.855177  6542 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:52:59.924194  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 155264us
I20260709 14:52:59.920624  6541 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:52:59.930452  6374 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:52:59.919659  6369 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:52:59.931183  6374 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:52:59.932723  6541 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:59.933471  6512 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:52:59.933684  6202 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:52:59.939803  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.939883  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.942963  3538 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.937563  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:52:59.963053  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81501.2us
I20260709 14:52:59.922012  6605 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:52:59.962850  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:52:59.985069  3543 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:52:59.983851  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:59.977778  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:52:59.965610  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.933460  6461 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:52:59.932736  6369 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:52:59.983455  2431 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
W20260709 14:52:59.967070  6607 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 14:52:59.933461  6374 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:52:59.983709  2423 thread.cc:641] dir /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.008s	sys 0.002s
I20260709 14:52:59.937947  6612 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:52:59.999394  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:00.001681  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 278315us
I20260709 14:53:00.012985  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:00.014498  6615 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:52:59.994918  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:00.015081  6615 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.015563  6615 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.016744  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.018230  6244 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:53:00.018870  6244 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.019073  6428 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:53:00.014635  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:52:59.968559  6440 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:52:59.980616  6354 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.024887  6440 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.025259  6354 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:00.025905  6354 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.026319  6354 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:53:00.027536  6609 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 14:52:59.971486  6541 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:00.021473  6428 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:00.019994  6244 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.017716  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:52:59.967636  6605 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:52:59.977712  6414 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.055403  6414 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:00.055115  6610 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.001434  6411 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:00.056910  6414 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:52:59.963702  6512 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:00.057474  6411 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:00.057902  6512 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:52:59.994263  6617 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:00.058640  6411 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.059023  6512 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.059520  6617 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:53:00.026358  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:00.035389  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 99354.1us
W20260709 14:53:00.036036  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 123254us
W20260709 14:53:00.004954  2423 thread.cc:608] dir /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.009s	sys 0.002s
I20260709 14:53:00.029484  6074 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:00.043126  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.044701  6374 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:00.004144  6461 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:00.006618  6369 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.046119  6428 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:00.061436  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.061527  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:00.063215  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 179526us
W20260709 14:53:00.088245  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45247.8us
I20260709 14:52:59.990876  6415 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 14:53:00.084955  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.026198  6440 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.057482  6414 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:00.064832  6612 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.086216  6374 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.081489  6541 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:00.086510  6477 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.101078  6392 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:00.089118  6428 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:00.117393  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 210048us
I20260709 14:53:00.089376  6202 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:00.113335  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.043220  6385 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.102478  6613 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:00.136570  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.139528  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:00.119359  6428 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.117365  6374 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:00.116219  6617 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:00.092263  6369 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.094663  6354 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.149341  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.150532  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.132758  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:00.130573  6616 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:00.139097  6613 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.134321  6622 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:00.145823  6477 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 14:53:00.199415  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44753us
I20260709 14:53:00.174023  6415 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:00.172127  6616 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:00.231896  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.225457  2682 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:00.154883  6615 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.248983  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:00.265645  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:00.252321  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134020us
I20260709 14:53:00.266767  2607 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:00.271700  2576 connection.cc:256] RPC call timeout handler was delayed by -2.0761917219999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2998s for a response
W20260709 14:53:00.277279  2576 connection.cc:256] RPC call timeout handler was delayed by -1.9346463970000514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2698s for a response
W20260709 14:53:00.276605  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98708.5us
W20260709 14:53:00.277921  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1653751700000612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2898s for a response
I20260709 14:53:00.280172  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:00.230291  6616 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.283854  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.175853  6461 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.181023  6414 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.285945  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:00.265556  6540 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.263863  6622 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:00.298508  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:00.298686  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.286312  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.299773  3540 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:00.247943  6074 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 14:53:00.304494  2607 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:00.222504  6392 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.309682  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:00.314329  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:00.314980  6626 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.315922  6626 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.314682  6369 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.244665  6541 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:00.325865  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 301757us
I20260709 14:53:00.297840  6622 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.299355  6617 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:00.302335  6635 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:00.305881  6620 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:00.347147  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54170.7us
W20260709 14:53:00.359525  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.301470  6540 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.330230  6369 raft_consensus.cc:2764] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:53:00.247432  6612 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.319244  6623 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.334903  6622 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:00.383597  6574 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.003s
I20260709 14:53:00.298154  6613 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.416440  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:00.410749  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 384442us
I20260709 14:53:00.419437  3542 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:00.398526  6574 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.389909  6626 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.344842  6610 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 14:53:00.419950  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:00.427100  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:00.416246  6495 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.009s
I20260709 14:53:00.405684  6354 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.436518  6495 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.399667  6624 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 14:53:00.423264  6480 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.006s
I20260709 14:53:00.465979  2760 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:00.354475  6620 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.428073  6623 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:00.443151  6628 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:00.481212  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 95536.5us
I20260709 14:53:00.442705  6495 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.484895  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.470810  6639 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:53:00.495913  6630 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 14:53:00.497884  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.497771  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:00.498479  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:00.500321  6551 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.717s	user 0.013s	sys 0.001s
I20260709 14:53:00.500850  6535 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.472319  6480 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.501001  6642 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:00.398579  6617 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.514564  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:00.505250  6598 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.710s	user 0.006s	sys 0.010s
I20260709 14:53:00.487376  6635 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:00.523453  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61214.5us
I20260709 14:53:00.525704  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.533895  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:00.520570  6602 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.003s
W20260709 14:53:00.522966  6606 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.007s
I20260709 14:53:00.516554  6615 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:00.546911  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.563400  6574 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:00.565217  6627 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 14:53:00.566947  6640 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:00.567795  6640 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:00.568678  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.568948  6640 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:00.570637  6471 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.705s	user 0.018s	sys 0.000s
W20260709 14:53:00.572045  6634 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 14:53:00.571470  6471 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:00.583262  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:00.586966  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 82000.4us
I20260709 14:53:00.500932  6551 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.532227  6480 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.590416  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.593470  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.543383  6535 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 14:53:00.589004  6399 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.806s	user 0.016s	sys 0.000s
I20260709 14:53:00.520216  6605 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.563513  6620 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.491521  6628 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.603174  2683 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:00.603314  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.589524  6610 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:00.606406  3540 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:00.533988  6598 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.550981  6602 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.616488  6551 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.596043  6535 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:00.604408  6399 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.602193  6615 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.610033  6440 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:00.559033  6635 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.560793  6606 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.618705  6602 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.642555  6605 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.619182  6535 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:53:00.645884  3541 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:00.621515  6399 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.621971  2680 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:00.623375  6621 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 14:53:00.628309  6414 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.648546  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:00.649021  6621 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:00.649240  3545 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.606630  6610 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:00.633778  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.650048  6610 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:00.633136  6628 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:00.654712  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61416.9us
I20260709 14:53:00.659238  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.664899  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.667687  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:00.671332  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 526540us
I20260709 14:53:00.649525  6621 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.640930  6606 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.637436  6635 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.701934  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.687533  6615 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:00.710283  6615 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:00.710927  6615 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:00.713910  2723 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.031s (SENT)
I20260709 14:53:00.685560  6399 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.720130  3545 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:00.722918  3540 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 14:53:00.724984  6602 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.724802  6399 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.725247  3547 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:00.725566  6602 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.726858  6628 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:00.727672  6628 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.728212  6660 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.713992  6623 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.731148  6621 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.732838  6621 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:00.735129  6598 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.698446  6616 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.715826  6661 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.724808  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.754869  6616 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:00.781373  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65155.8us
I20260709 14:53:00.783730  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.784152  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:00.785071  3536 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 14:53:00.787580  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.791275  6616 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.792126  6616 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.794971  6616 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:00.796792  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 72099.3us
I20260709 14:53:00.797814  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:00.798017  6602 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.798635  6628 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:00.786954  6661 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.799611  6467 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:00.799753  6661 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:00.800096  6467 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.800171  6661 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.800665  6661 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.800318  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:00.809386  6617 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:00.810117  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1182) took 2631 ms. Trace:
I20260709 14:53:00.781339  6598 raft_consensus.cc:2764] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 14:53:00.817834  6467 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.827191  6640 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:00.855130  6640 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.856614  6640 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:00.857398  3540 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:00.842869  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.844461  2683 rpcz_store.cc:276] 0709 14:52:58.171873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.824888 (+1653015us) service_pool.cc:224] Handling call
0709 14:52:59.825325 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:00.607999 (+782674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:00.790789 (+182790us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:00.802793 (+ 12004us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788375808}
W20260709 14:53:00.861167  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47090.9us
W20260709 14:53:00.863492  6428 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.722s	user 0.016s	sys 0.005s
I20260709 14:53:00.851735  6467 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.827852  6399 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.814544  6541 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 14:53:00.863201  6598 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:00.843945  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 458349us
W20260709 14:53:00.883546  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.891425  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:00.896417  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 677) took 2695 ms (client timeout 3573 ms). Trace:
W20260709 14:53:00.900556  2684 rpcz_store.cc:269] 0709 14:52:58.201016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.731383 (+1530367us) service_pool.cc:224] Handling call
0709 14:53:00.896085 (+1164702us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54002b2854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:00.896320 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254235520}
I20260709 14:53:00.797806  6615 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.871280  6428 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:00.906121  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 118742us
I20260709 14:53:00.893221  6541 consensus_queue.cc:1243] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:00.908066  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:00.924959  6369 raft_consensus.cc:2764] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:53:00.926347  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:00.932029  6369 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 14:53:00.932571  6369 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:00.932730  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:00.906265  6374 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.740s	user 0.018s	sys 0.002s
I20260709 14:53:00.933672  6374 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:00.935477  6648 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:00.936136  6374 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.914160  6615 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.915032  6612 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:00.937489  6612 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.921159  6428 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:00.920743  6610 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.941418  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:00.942152  6535 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.943053  6662 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.943377  6610 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.944144  3547 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:00.944474  6535 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:00.922394  6440 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:00.933013  6369 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:53:00.945844  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 670) took 2765 ms. Trace:
I20260709 14:53:00.946266  2680 rpcz_store.cc:276] 0709 14:52:58.180948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.833361 (+1652413us) service_pool.cc:224] Handling call
0709 14:52:59.833780 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:00.647677 (+813897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:00.945562 (+297885us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:00.945779 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811604352}
W20260709 14:53:00.908085  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 208743us
I20260709 14:53:00.899161  6642 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:00.938624  6612 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.905153  6616 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 14:53:00.951705  6394 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.952354  6616 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:53:00.952876  6616 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:00.953343  6394 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:00.944723  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.948414  6660 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.956017  6617 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.956398  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1308) took 1222 ms. Trace:
I20260709 14:53:00.953994  6616 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:00.956848  2760 rpcz_store.cc:276] 0709 14:52:59.733852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.733984 (+   132us) service_pool.cc:224] Handling call
0709 14:53:00.956105 (+1222121us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:00.956287 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508603776}
I20260709 14:53:00.957849  6616 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 14:53:00.961097  6638 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.545s	user 0.002s	sys 0.000s
I20260709 14:53:00.955821  6660 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:00.964947  6631 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:00.966720  6385 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.966508  6660 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.935672  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:00.968876  6645 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:00.969354  6645 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:00.969730  6645 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:00.970027  6622 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.010s
I20260709 14:53:00.970934  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.974761  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:00.976121  2760 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:00.976189  6660 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:00.977344  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:00.978034  2757 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:00.978479  6637 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:00.978911  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:00.979432  6385 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:00.974416  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:00.977134  6660 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:00.980525  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:00.973204  6471 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:00.973879  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:00.979691  2753 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:00.984691  2759 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:00.988999  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:00.991320  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:00.980588  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:00.991750  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:00.983816  2751 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:00.996407  2683 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:00.984125  2754 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:00.983238  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:01.009819  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59541.4us
I20260709 14:53:01.028141  2755 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:01.035688  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.035975  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.036688  2681 raft_consensus.cc:2376] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:53:01.036912  2677 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:01.035672  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.045610  2686 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:01.047400  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.048493  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.049510  2683 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:01.050408  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:00.983160  6385 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:00.970587  6622 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.050863  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.047061  6637 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:01.058650  2677 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 14:53:00.992041  6660 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:01.065105  2760 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 14:53:01.057835  2678 raft_consensus.cc:2376] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10. Re-sending same reply.
I20260709 14:53:01.059170  2677 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 4
W20260709 14:53:01.061802  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81305.6us
I20260709 14:53:01.066445  2677 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:53:01.066764  2677 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:01.069587  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:01.061645  6645 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:01.063217  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:00.989526  6369 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.073225  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.069375  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.073148  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:01.055841  6385 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.076033  2686 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 9: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:01.076778  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.085821  3537 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 14:53:01.068338  6541 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.087661  2685 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:01.089555  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.090301  2678 raft_consensus.cc:2376] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8. Re-sending same reply.
I20260709 14:53:01.090503  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.091682  2685 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:01.092525  6645 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:01.101411  2755 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:01.104249  2759 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:01.105176  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.106256  3537 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:01.110939  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.113804  3536 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 7: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:01.079977  6374 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 14:53:01.122401  2685 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:01.131250  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.133359  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:01.101253  6541 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.104249  6645 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 14:53:01.142210  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.143486  3540 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:53:01.139370  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.145717  6495 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:53:01.149016  3542 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:01.148702  2610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:01.149638  2610 raft_consensus.cc:606] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.154282  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:01.155720  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.155295  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.156994  3536 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:01.159924  2678 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 14:53:01.162077  6202 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:01.162648  6369 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:01.163422  6631 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:01.163331  6495 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.164121  6631 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:01.164584  6631 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:01.165040  6495 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.165911  6541 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:01.128787  6385 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.168234  6385 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:01.172024  6622 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:01.178594  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.179994  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.182386  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:01.183198  6414 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:01.183856  6628 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:01.188629  6628 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.189787  6628 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.185449  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.186151  6602 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:53:01.187196  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.192520  3536 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:01.194430  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:01.197751  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.212255  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.213192  2607 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:01.214514  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.146348  6640 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.204756  6385 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:01.236004  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.227294  6414 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:53:01.255031  6414 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.256371  6414 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.260583  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.261930  2604 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:53:01.263193  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.263967  2609 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:01.269418  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:01.244391  6668 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.188s	user 0.000s	sys 0.002s
I20260709 14:53:01.274935  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.244935  6495 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 14:53:01.246598  6385 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.278784  6385 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.244820  6202 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.281549  6202 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.282351  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.283636  2679 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 14:53:01.286262  2604 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 5. Ops: []
W20260709 14:53:01.280880  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.289s	user 0.007s	sys 0.007s
W20260709 14:53:01.287125  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.008s	sys 0.007s
W20260709 14:53:01.288105  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.36044e+06us
I20260709 14:53:01.300637  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:01.316347  2604 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 14:53:01.243307  6602 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 14:53:01.284310  6495 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:01.331689  6495 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.342495  2604 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:01.324956  6661 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:01.345103  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84650.7us
I20260709 14:53:01.348488  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.352663  3536 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:01.357156  6661 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:01.345352  6676 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 14:53:01.363667  6640 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:01.364171  6676 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:01.364682  6676 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:01.365705  6676 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.371447  6062 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:01.372143  6062 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:01.372851  6675 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:01.373514  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.373265  6062 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.373983  6645 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:53:01.374271  6675 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:01.375869  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:01.376600  2753 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:01.377564  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.378271  2610 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:01.379443  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.379943  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.380304  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.380518  2607 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:01.382043  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.382681  2605 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:01.383665  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.384378  2604 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:01.374526  6062 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:01.375149  6675 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:01.389873  2753 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:01.392939  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.387897  6675 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.396806  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.399501  6675 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:01.400256  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.400529  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.401389  2604 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:01.406572  6369 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:01.409041  6369 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:01.410387  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.013s	sys 0.014s
W20260709 14:53:01.411015  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.015s	sys 0.014s
W20260709 14:53:01.411368  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.12144e+06us
I20260709 14:53:01.410108  6369 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.419878  6369 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:01.411895  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.017s	sys 0.015s
I20260709 14:53:01.403753  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.417271  6613 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:01.421389  2607 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:53:01.421093  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.017s	sys 0.015s
I20260709 14:53:01.421881  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:01.415848  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.423897  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 675) took 1291 ms. Trace:
I20260709 14:53:01.424829  2612 rpcz_store.cc:276] 0709 14:53:00.132506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:00.138166 (+  5660us) service_pool.cc:224] Handling call
0709 14:53:00.138584 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:00.145523 (+  6939us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:00.287825 (+142302us) log.cc:844] Serialized 14 byte log entry
0709 14:53:01.422182 (+1134357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:01.422246 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:01.422313 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:01.422411 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:01.423839 (+  1428us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":144924,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":102,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":179,"thread_start_us":1214077,"threads_started":2,"wal-append.queue_time_us":197483}
I20260709 14:53:01.427399  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.421032  2759 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:53:01.428900  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.431345  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:01.433190  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.434137  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.434688  2605 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:53:01.420684  6676 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:01.424423  6648 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:01.438218  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.431262  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.438787  2612 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:01.436633  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.350134  6495 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.443406  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.443832  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.446197  2604 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:01.446894  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.448063  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.450295  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.451534  2607 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 14:53:01.453815  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.445449  6645 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:01.457561  2605 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:53:01.460899  2604 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:53:01.462765  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.468050  6619 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:01.469867  6619 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 14:53:01.452346  6677 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:01.473481  2610 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
W20260709 14:53:01.476522  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:01.477766  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.460427  6495 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:01.481364  2605 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:01.480829  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.481001  2612 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:01.483191  2607 raft_consensus.cc:2376] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9. Re-sending same reply.
I20260709 14:53:01.480365  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.486362  2759 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:01.491182  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.493353  6374 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:01.493961  6374 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:01.494374  6374 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:01.500741  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:01.500978  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.479818  6414 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.465809  6631 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.525293  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.526079  2681 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:01.526459  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.527031  6631 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:01.527290  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.528189  3538 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:53:01.507005  6677 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:01.529458  3538 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:53:01.533870  2759 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:53:01.536067  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.301s	user 0.003s	sys 0.012s
W20260709 14:53:01.536585  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.004s	sys 0.013s
I20260709 14:53:01.511196  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.511474  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.511777  2606 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:01.544209  2610 raft_consensus.cc:2376] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11. Re-sending same reply.
I20260709 14:53:01.512205  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.512467  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.546070  2605 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:01.550222  3547 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 14:53:01.553005  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63387.9us
W20260709 14:53:01.554639  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60099.2us
I20260709 14:53:01.556367  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.513226  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.513612  2678 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:01.557634  2607 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:01.514633  2679 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:01.515349  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.559650  2685 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:01.560225  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.560585  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 674) took 1428 ms. Trace:
I20260709 14:53:01.560942  2681 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:01.517653  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.545609  2684 raft_consensus.cc:2376] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13. Re-sending same reply.
I20260709 14:53:01.562038  3545 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:01.518057  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.560968  2611 rpcz_store.cc:276] 0709 14:53:00.132057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:00.138166 (+  6109us) service_pool.cc:224] Handling call
0709 14:53:00.167844 (+ 29678us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:00.177385 (+  9541us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:00.221768 (+ 44383us) log.cc:844] Serialized 14 byte log entry
0709 14:53:00.936294 (+714526us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b18003620e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:01.541535 (+605241us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:01.549895 (+  8360us) log.cc:844] Serialized 52 byte log entry
0709 14:53:01.550107 (+   212us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:01.550150 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:01.550231 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:01.552714 (+  2483us) raft_consensus.cc:1685] finished
0709 14:53:01.560351 (+  7637us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:01.560520 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4417,"prepare.queue_time_us":113472,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":103724,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":19280,"spinlock_wait_cycles":4689664,"thread_start_us":1313889,"threads_started":2,"wal-append.queue_time_us":307374}
I20260709 14:53:01.524264  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.524474  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.564291  3537 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:01.564458  3542 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:53:01.558269  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.566486  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:01.570883  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:01.573140  2680 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 7: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:01.575495  2683 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:01.553005  6637 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:01.594816  2576 leader_election.cc:400] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:53:01.591531  3537 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:01.605149  2576 leader_election.cc:403] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:53:01.596766  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.599583  2609 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:01.590536  3536 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 14:53:01.606700  3536 pending_rounds.cc:85] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:53:01.607028  3536 pending_rounds.cc:107] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 14:53:01.607295  3536 raft_consensus.cc:2889] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:01.590471  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:01.582522  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:01.595829  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1175) took 5714 ms (client timeout 2163 ms). Trace:
I20260709 14:53:01.594452  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:01.611698  2682 rpcz_store.cc:269] 0709 14:52:55.881199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:55.881374 (+   175us) service_pool.cc:224] Handling call
0709 14:53:01.595369 (+5713995us) spinlock_profiling.cc:243] Waited 4.18 s on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:01.595721 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9608728832}
W20260709 14:53:01.619220  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:01.620162  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77219.7us
I20260709 14:53:01.542255  6414 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:01.620764  2726 connection.cc:256] RPC call timeout handler was delayed by -1.163124512999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 14:53:01.579955  6660 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:01.600915  6682 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
W20260709 14:53:01.623106  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1523245129999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 14:53:01.593370  6369 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:01.586263  6622 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:01.628084  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.628420  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:01.592324  6619 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:01.586912  6244 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.632508  6369 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:01.573807  6612 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 14:53:01.632516  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.616677  6675 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:01.650175  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.643646  2681 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:01.652858  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.654064  2612 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:01.634421  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:01.646584  2726 connection.cc:256] RPC call timeout handler was delayed by -1.0389574839999796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 14:53:01.656154  2726 connection.cc:256] RPC call timeout handler was delayed by -1.028731592999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 14:53:01.627055  6660 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:01.662649  6369 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:01.658819  6600 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:01.664008  6600 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:01.665247  6600 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.666409  3540 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:01.688149  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:01.689510  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68509.5us
I20260709 14:53:01.689908  2606 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:01.651657  6631 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.639871  6619 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:01.681066  6676 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:01.626504  6661 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 14:53:01.659143  6675 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.625674  6613 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 14:53:01.684561  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.648030  6244 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:01.688980  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.676477  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.669636  6374 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.645516  6688 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.712412  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:01.711069  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.687724  6660 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:01.645486  6622 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.666635  6600 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:01.724264  6661 consensus_queue.cc:1243] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:01.726534  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.705318  6619 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:01.699908  6369 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:53:01.712244  6640 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:01.730151  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 87415.1us
I20260709 14:53:01.740351  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.713755  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.741823  2608 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 14:53:01.732717  2606 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:01.735316  6244 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:01.736471  6622 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:01.698071  6612 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 14:53:01.704579  6631 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.744810  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.727952  6693 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:01.747311  3540 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:53:01.743754  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 153394us
I20260709 14:53:01.751991  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.739180  6661 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:01.753701  2608 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:01.760960  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.730091  6062 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 14:53:01.765249  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:01.768317  2756 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:01.761090  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:01.770457  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.771435  2606 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:01.772876  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.773113  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.757517  6369 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:01.774250  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:01.776677  2752 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:53:01.781637  2609 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:53:01.782951  6661 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:01.783491  2604 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:01.783812  6414 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:01.784176  6661 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.786230  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.754074  6244 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.747138  6640 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.787652  6244 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.789043  2607 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:01.789773  6640 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:01.789937  6244 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:01.791725  6698 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:53:01.792217  2612 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:01.792740  6698 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:53:01.793644  2612 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:01.752724  6693 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:01.798812  6693 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:01.803964  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:01.729946  6688 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:01.808851  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 213910us
I20260709 14:53:01.811187  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:01.787115  3540 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:01.813563  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:01.807950  6688 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:01.817373  6625 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:01.817653  6688 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:01.819610  6696 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:01.819993  6675 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:01.812369  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.825627  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:01.820255  6696 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:01.748206  6695 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:01.842351  2609 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:01.872452  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.867622  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.872867  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:01.866006  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54917.4us
W20260709 14:53:01.875119  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 100642us
I20260709 14:53:01.758924  6540 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:01.868135  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:01.839035  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.888072  2753 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:01.879669  2612 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:01.893272  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.896415  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:01.882190  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56575.4us
I20260709 14:53:01.818733  6688 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:01.884459  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.905380  3547 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:01.899662  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:01.901607  2757 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:01.908654  2757 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:01.834896  6612 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:01.911266  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:01.917387  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:01.919094  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.919157  2753 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:01.922051  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.870160  6675 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:01.870175  6495 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:01.930285  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:01.931895  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:01.923872  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:01.931967  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.933718  2679 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:01.888828  6244 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:01.935843  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:01.936655  2758 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:01.936528  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:01.835422  6640 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:01.895879  6622 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 14:53:02.128482  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1192) took 2406 ms (client timeout 1939 ms). Trace:
W20260709 14:53:02.129297  3541 rpcz_store.cc:269] 0709 14:52:59.721855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:59.724605 (+  2750us) service_pool.cc:224] Handling call
0709 14:53:02.128027 (+2403422us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54004c4ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:02.128365 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2091855232}
I20260709 14:53:02.131630  3543 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:02.132941  6640 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:02.133442  6640 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:02.135504  6202 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:02.136142  2607 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:02.136670  2607 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:02.137688  2607 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:02.139155  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:02.140246  2607 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:02.140384  2610 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:53:02.142211  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:02.143425  6623 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:02.143877  6623 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:02.144208  6623 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:02.145749  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:02.152951  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:02.154032  2610 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:02.154975  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:01.935200  6354 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 14:53:02.038558  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 326473us
I20260709 14:53:02.161051  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1054) took 1648 ms. Trace:
I20260709 14:53:02.162094  3543 rpcz_store.cc:276] 0709 14:53:00.512338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:00.518510 (+  6172us) service_pool.cc:224] Handling call
0709 14:53:02.131221 (+1612711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:02.132263 (+  1042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:02.160773 (+ 28510us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54004c4ad0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:02.160962 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3674514176}
I20260709 14:53:02.161828  6712 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 14:53:02.164093  6648 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:02.166363  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:02.173962  6712 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:02.174707  6712 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:02.175611  6698 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 14:53:02.176463  6467 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:02.177004  6698 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:02.177610  6698 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:02.178653  6698 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:02.180172  6698 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:02.005403  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:01.923694  6540 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 14:53:02.174036  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:02.183777  2604 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:02.184610  6540 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:53:01.867995  6614 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 14:53:02.185669  2604 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 14:53:02.186164  6614 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:02.186547  6614 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:02.187216  6700 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:02.187429  6614 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:02.188993  6614 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:01.935518  6394 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 14:53:02.189906  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:02.099691  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:02.190742  6648 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:02.191201  6648 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:02.191584  6648 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:02.192523  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:02.192797  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:02.193248  2679 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:02.101055  6369 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:02.194221  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:02.194769  2603 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:02.195248  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:02.195503  6600 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.939863  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:02.196475  2680 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:02.197817  2603 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:02.197989  2681 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:02.198441  6394 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:02.200654  3412 leader_election.cc:400] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:02.201282  3412 leader_election.cc:403] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 14:53:01.939069  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:02.103530  6701 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.904979  6688 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:01.955891  6696 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.492187  2682 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:01.932790  6675 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:03.492266  6701 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.493193  6675 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:03.495042  6701 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:02.226280  6700 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:03.492990  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1115) took 1564 ms. Trace:
W20260709 14:53:03.500089  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.013s	sys 0.038s
W20260709 14:53:03.501420  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.346s	user 0.013s	sys 0.038s
I20260709 14:53:02.243875  6693 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.500099  2682 rpcz_store.cc:276] 0709 14:53:01.928816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.928987 (+   171us) service_pool.cc:224] Handling call
0709 14:53:03.492911 (+1563924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:03.504827  6693 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:02.265975  6712 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:02.305562  6720 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:02.304404  6718 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 14:53:02.346279  2423 maintenance_manager.cc:643] P 3f6dfe8cfa964684b547b44a4b308ca2: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.477s	user 0.206s	sys 0.060s Metrics: {"bytes_written":29396,"cfile_init":1,"dirs.queue_time_us":709816,"dirs.run_cpu_time_us":903,"dirs.run_wall_time_us":71897,"drs_written":1,"lbm_read_time_us":503,"lbm_reads_lt_1ms":4,"lbm_write_time_us":171441,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"rows_written":63,"spinlock_wait_cycles":4978560,"thread_start_us":839929,"threads_started":2,"wal-append.queue_time_us":28653}
I20260709 14:53:02.248553  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:02.265311  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:03.652974  6359 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:53:03.653442  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.38814e+06us
W20260709 14:53:03.653038  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.40452e+06us
I20260709 14:53:03.654723  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:03.656792  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:02.296530  2607 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:02.118595  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:02.184947  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:01.995096  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:03.767426  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
W20260709 14:53:03.768203  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1321) took 1891 ms (client timeout 1984 ms). Trace:
W20260709 14:53:03.768684  2609 rpcz_store.cc:269] 0709 14:53:01.876864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.994565 (+117701us) service_pool.cc:224] Handling call
0709 14:53:03.768109 (+1773544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:03.769908  2609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:02.215605  2605 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:02.204922  2603 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:03.776178  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1322) took 1899 ms. Trace:
I20260709 14:53:03.776639  2603 rpcz_store.cc:276] 0709 14:53:01.877255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.198813 (+321558us) service_pool.cc:224] Handling call
0709 14:53:02.204370 (+  5557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.775760 (+1571390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.776096 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:02.189046  2604 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:01.941677  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:03.779655  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 685) took 1979 ms. Trace:
I20260709 14:53:03.780401  2683 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:03.781113  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1117) took 1849 ms. Trace:
W20260709 14:53:03.613222  2650 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.486s	user 0.012s	sys 0.032s
W20260709 14:53:03.232659  6683 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.743s	user 0.006s	sys 0.000s
I20260709 14:53:03.266624  6726 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.267156  6727 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.292695  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:03.332523  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:03.788964  6726 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.789113  6727 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.366691  6687 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:02.121358  2755 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:03.804950  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1328) took 1684 ms. Trace:
W20260709 14:53:02.108546  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:02.112323  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:01.939420  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 178429us
I20260709 14:53:02.115981  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:03.814563  2756 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 14:53:03.815104  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1325) took 1924 ms (client timeout 1759 ms). Trace:
W20260709 14:53:03.815508  2756 rpcz_store.cc:269] 0709 14:53:01.890512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.916827 (+ 26315us) service_pool.cc:224] Handling call
0709 14:53:03.815024 (+1898197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:02.222982  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:03.817755  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1225) took 1644 ms (client timeout 1681 ms). Trace:
W20260709 14:53:03.818135  2757 rpcz_store.cc:269] 0709 14:53:02.173374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.222503 (+ 49129us) service_pool.cc:224] Handling call
0709 14:53:03.817689 (+1595186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:02.115790  2758 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:03.820102  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1327) took 1705 ms. Trace:
I20260709 14:53:03.820592  2758 rpcz_store.cc:276] 0709 14:53:02.114940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.115131 (+   191us) service_pool.cc:224] Handling call
0709 14:53:02.115551 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.819819 (+1704268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.820040 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:02.352120  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:03.212834  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.002s	sys 0.031s
W20260709 14:53:03.825344  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.609s	user 0.003s	sys 0.032s
W20260709 14:53:03.390900  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.188s	user 0.006s	sys 0.015s
W20260709 14:53:03.407369  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08001f0fa0 after lost signal to thread 434
I20260709 14:53:03.828224  6741 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.829486  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf060 after lost signal to thread 2469
I20260709 14:53:03.828994  6741 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:02.113826  6374 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:02.120402  6414 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 14:53:03.836859  6414 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:02.176959  6467 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:03.840523  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.48838e+06us
I20260709 14:53:03.840884  6467 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.937033  6495 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:03.844512  6495 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:02.196010  6600 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:01.912799  6612 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:03.849433  6612 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:03.853076  6751 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:53:03.211336  6614 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.014s	user 0.013s	sys 0.014s
I20260709 14:53:03.867535  6614 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:03.868296  6614 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:02.349546  6616 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:03.872458  6616 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:03.332901  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.204s	user 0.023s	sys 0.026s
W20260709 14:53:03.876029  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.747s	user 0.025s	sys 0.026s
I20260709 14:53:03.876535  6622 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:03.383988  6623 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.239s	user 0.055s	sys 0.026s
I20260709 14:53:03.880641  6623 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.883123  6740 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.884332  6740 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:01.933226  6625 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 14:53:03.889670  6625 consensus_queue.cc:1243] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:03.893138  6743 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:03.893961  6743 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:01.925133  6631 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.365437  6637 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:03.897116  6637 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.898205  6616 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.898504  6625 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:03.899000  6616 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:03.899206  6625 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.899771  6637 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:03.899621  6616 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.900454  6637 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.900988  6637 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.901849  6625 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.902256  6616 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:02.222679  6642 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:03.903815  6637 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:02.125828  6619 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.905508  6623 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:02.332773  6648 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.899721  6756 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.909242  6648 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:02.121230  6660 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.910184  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:03.911836  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:03.912210  6660 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.912653  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:03.912863  6648 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:03.913053  2608 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:03.913411  2756 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:03.877965  6622 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.915927  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 689) took 1785 ms. Trace:
I20260709 14:53:03.916124  6748 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:03.916394  2608 rpcz_store.cc:276] 0709 14:53:02.130540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.201068 (+ 70528us) service_pool.cc:224] Handling call
0709 14:53:02.201405 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.913568 (+1712163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.915659 (+  2091us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540042e0d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:03.915860 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3935884160}
W20260709 14:53:03.486706  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.372s	user 0.000s	sys 0.020s
I20260709 14:53:03.917701  6648 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:03.918195  6648 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:03.918296  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.804s	user 0.000s	sys 0.022s
I20260709 14:53:03.918438  6614 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.919441  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1319) took 2034 ms. Trace:
W20260709 14:53:03.488544  2680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.012s	sys 0.014s
I20260709 14:53:03.919253  6648 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.918349  6740 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:03.525064  6715 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:03.527156  6675 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.911875  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:03.921147  2751 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:03.919859  6743 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.525888  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1119) took 1593 ms. Trace:
I20260709 14:53:03.921988  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1354) took 1761 ms. Trace:
I20260709 14:53:03.922451  6675 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.525799  6700 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
W20260709 14:53:03.575489  6640 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.442s	user 0.028s	sys 0.000s
W20260709 14:53:03.592897  2753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.489s	user 0.012s	sys 0.004s
I20260709 14:53:03.593135  6733 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:03.580480  2431 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 5 to 8, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:03.599625  6732 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.923754  6733 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.924387  6732 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.924696  2431 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 5 to 6, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:03.597783  6701 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.606280  6734 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.624504  6735 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.611804  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.395s	user 0.006s	sys 0.008s
I20260709 14:53:03.627667  6736 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.927703  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.711s	user 0.006s	sys 0.009s
I20260709 14:53:03.927361  6734 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.928028  6736 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.928752  6732 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:03.928875  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1314) took 2108 ms. Trace:
I20260709 14:53:03.912249  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:03.922281  2677 rpcz_store.cc:276] 0709 14:53:01.932316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.933150 (+   834us) service_pool.cc:224] Handling call
0709 14:53:02.136237 (+203087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:02.154783 (+ 18546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:02.154816 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:02.155443 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 14:53:03.501880 (+1346437us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:03.515778 (+ 13898us) log.cc:844] Serialized 14 byte log entry
0709 14:53:03.524858 (+  9080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.525571 (+   713us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:03.525630 (+    59us) raft_consensus.cc:1685] finished
0709 14:53:03.525643 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:03.525818 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13032,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":16811,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":179,"thread_start_us":1385754,"threads_started":4,"wal-append.queue_time_us":167820,"wal-append.run_cpu_time_us":2289,"wal-append.run_wall_time_us":1002323}
W20260709 14:53:03.641185  2652 connection.cc:256] RPC call timeout handler was delayed by -1.1834094249999225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 14:53:03.653540  6702 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:03.661233  6359 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 14:53:03.661053  6645 consensus_queue.cc:1175] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 12 status { last_received { term: 10 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 14:53:03.669523  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.503s	user 0.013s	sys 0.015s
I20260709 14:53:03.669441  6737 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.694761  6739 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:03.734155  6626 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 14:53:03.749717  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1349) took 2045 ms. Trace:
I20260709 14:53:02.103830  2685 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:53:03.780210  2604 rpcz_store.cc:276] 0709 14:53:01.800634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.188364 (+387730us) service_pool.cc:224] Handling call
0709 14:53:02.188707 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.779249 (+1590542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.779579 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:03.781544  2683 rpcz_store.cc:276] 0709 14:53:01.931674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.940352 (+  8678us) service_pool.cc:224] Handling call
0709 14:53:03.781036 (+1840684us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:03.795424  2650 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.668s	user 0.016s	sys 0.032s
I20260709 14:53:03.794449  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:03.800877  6687 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.794872  6726 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:03.286756  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.488s	user 0.044s	sys 0.028s
I20260709 14:53:03.922844  2606 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:03.898135  6495 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.927477  6735 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.945319  6739 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:03.810328  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:03.971346  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:03.973832  6758 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:03.974217  2724 connection.cc:256] RPC call timeout handler was delayed by -1.8206848009999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 14:53:03.812301  6202 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.813019  2755 rpcz_store.cc:276] 0709 14:53:02.120555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.120760 (+   205us) service_pool.cc:224] Handling call
0709 14:53:02.121160 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.804587 (+1683427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.804874 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:03.823824  2433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 160) took 50188 ms (client timeout 15000 ms). Trace:
I20260709 14:53:03.974686  6758 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.975172  6202 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.827042  6738 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.828560  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.626s	user 0.007s	sys 0.015s
W20260709 14:53:03.976779  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.77431e+06us
I20260709 14:53:03.929430  2752 rpcz_store.cc:276] 0709 14:53:01.820895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.821050 (+   155us) service_pool.cc:224] Handling call
0709 14:53:01.870255 (+ 49205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:01.870517 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:01.902461 (+ 31944us) log.cc:844] Serialized 14 byte log entry
0709 14:53:03.371860 (+1469399us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b1800445240. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:53:03.928435 (+556575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:03.928507 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:03.928559 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.928634 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:03.928781 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61335,"prepare.queue_time_us":40108,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":217967488,"thread_start_us":1711230,"threads_started":1,"wal-append.queue_time_us":423343,"wal-append.run_cpu_time_us":1486,"wal-append.run_wall_time_us":1001793}
I20260709 14:53:03.903973  2605 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 14:53:03.980201  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:01.947132  6244 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:03.992609  6244 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:03.831193  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800383c40 after lost signal to thread 2636
I20260709 14:53:03.957229  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:03.996192  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08002834c0 after lost signal to thread 2710
W20260709 14:53:03.996543  2633 debug-util.cc:398] Leaking SignalData structure 0x7b080035e6a0 after lost signal to thread 2711
I20260709 14:53:03.997208  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:03.998147  3536 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:03.998179  2633 debug-util.cc:398] Leaking SignalData structure 0x7b080030e060 after lost signal to thread 2784
W20260709 14:53:03.975244  2433 rpcz_store.cc:269] 0709 14:52:13.635925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:13.636327 (+   402us) service_pool.cc:224] Handling call
0709 14:53:03.823540 (+50187213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:52:49.791819 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:52:49.792627 (+   808us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:52:49.792684 (+    57us) write_op.cc:435] Acquired schema lock
0709 14:52:49.792703 (+    19us) tablet.cc:662] Decoding operations
0709 14:52:49.793938 (+  1235us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:52:49.794038 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 14:52:49.794070 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 14:52:49.864711 (+ 70641us) tablet.cc:701] Row locks acquired
0709 14:52:49.864762 (+    51us) write_op.cc:260] PREPARE: finished
0709 14:52:49.864999 (+   237us) write_op.cc:270] Start()
0709 14:52:49.865191 (+   192us) write_op.cc:276] Timestamp: P: 1783608769864921 usec, L: 0
0709 14:52:49.865209 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 14:52:49.866123 (+   914us) log.cc:844] Serialized 5197 byte log entry
0709 14:52:50.524759 (+658636us) op_driver.cc:464] REPLICATION: finished
0709 14:52:51.069538 (+544779us) write_op.cc:301] APPLY: starting
0709 14:52:51.069660 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 14:52:51.069986 (+   326us) tablet.cc:1370] finished BulkCheckPresence
0709 14:52:51.070145 (+   159us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:52:51.150049 (+ 79904us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:52:51.150472 (+   423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 14:52:51.150524 (+    52us) write_op.cc:312] APPLY: finished
0709 14:52:51.151073 (+   549us) log.cc:844] Serialized 94 byte log entry
0709 14:52:51.168400 (+ 17327us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:51.188116 (+ 19716us) write_op.cc:454] Released schema lock
0709 14:52:51.194937 (+  6821us) write_op.cc:341] FINISH: Updating metrics
0709 14:52:51.195518 (+   581us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:52:51.195554 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":472531,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":451927,"apply.run_cpu_time_us":6227,"apply.run_wall_time_us":125977,"num_ops":13,"prepare.queue_time_us":292159,"prepare.run_cpu_time_us":42718,"prepare.run_wall_time_us":370859,"raft.queue_time_us":235907,"raft.run_cpu_time_us":7313,"raft.run_wall_time_us":488819,"replication_time_us":629622,"thread_start_us":757171,"threads_started":4,"wal-append.queue_time_us":273089,"wal-append.run_cpu_time_us":24495,"wal-append.run_wall_time_us":2016975}]]}
I20260709 14:53:04.000299  6702 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:03.994877  6739 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:03.835163  6745 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.847515  6747 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:03.848482  6749 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:01.925437  6613 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:53:03.856980  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:03.842569  6467 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:03.859319  2678 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:04.070986  6702 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.071936  6702 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.859437  6741 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:03.859577  6612 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.074442  3540 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:04.075604  3540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1066) took 2180 ms (client timeout 2008 ms). Trace:
I20260709 14:53:04.075747  6612 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.838157  6414 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:04.076050  3540 rpcz_store.cc:269] 0709 14:53:01.895537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.910786 (+ 15249us) service_pool.cc:224] Handling call
0709 14:53:04.075362 (+2164576us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:04.075514 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4971633536}
I20260709 14:53:04.077080  6414 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:53:04.078051  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:03.850517  6600 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:03.861294  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:03.904027  6642 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:04.079720  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.080001  6642 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.081125  6642 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:04.081363  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:02.217152  6394 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.893757  6688 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:03.906738  2611 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:53:04.084677  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1353) took 2269 ms (client timeout 2077 ms). Trace:
W20260709 14:53:04.085130  2611 rpcz_store.cc:269] 0709 14:53:01.815353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.118252 (+302899us) service_pool.cc:224] Handling call
0709 14:53:04.084465 (+1966213us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540016e4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:04.084595 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393937280}
I20260709 14:53:04.085613  3541 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:53:03.909674  6756 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:03.920078  2680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.722s	user 0.013s	sys 0.014s
I20260709 14:53:04.088104  6702 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:03.506196  6720 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:03.920499  6648 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:03.920679  6715 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:03.584141  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1120) took 1651 ms. Trace:
I20260709 14:53:04.085131  6394 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.095353  2684 rpcz_store.cc:276] 0709 14:53:01.932624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.932747 (+   123us) service_pool.cc:224] Handling call
0709 14:53:01.933100 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:01.933350 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:01.938615 (+  5265us) log.cc:844] Serialized 14 byte log entry
0709 14:53:03.574891 (+1636276us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b18000a34e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:03.583632 (+  8741us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:03.583705 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:03.583777 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.583882 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:03.584061 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7671,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1383,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":100,"spinlock_wait_cycles":36215552,"thread_start_us":8418,"threads_started":1,"wal-append.queue_time_us":1485360}
I20260709 14:53:04.096776  6720 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.097527  2681 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:53:04.097729  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.098138  6715 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:03.923556  6640 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.099118  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.099666  6756 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.100405  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.100698  6640 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.658035  6677 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 14:53:03.638953  2424 maintenance_manager.cc:419] P 3f6dfe8cfa964684b547b44a4b308ca2: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6219 bytes on disk
I20260709 14:53:03.659428  6693 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:04.102106  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:04.102440  6693 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:03.946646  6735 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.930603  6736 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:04.103478  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:03.915251  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:04.107681  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 192483us
I20260709 14:53:04.093516  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 691) took 1893 ms. Trace:
I20260709 14:53:03.947611  6712 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.948302  6687 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:03.951439  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.785s	user 0.016s	sys 0.017s
I20260709 14:53:04.110044  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:04.113996  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.94802e+06us
I20260709 14:53:04.113080  3541 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:03.952054  2431 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 5 to 6, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:53:04.115339  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87143.4us
I20260709 14:53:03.930686  6619 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.117496  6677 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 14:53:03.919858  2754 rpcz_store.cc:276] 0709 14:53:01.885025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.887176 (+  2151us) service_pool.cc:224] Handling call
0709 14:53:01.887640 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:01.905580 (+ 17940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:01.905631 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:01.930709 (+ 25078us) log.cc:844] Serialized 26 byte log entry
0709 14:53:03.919049 (+1988340us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:03.919101 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.919178 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:03.919222 (+    44us) raft_consensus.cc:1685] finished
0709 14:53:03.919234 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:03.919368 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24196,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":17128,"thread_start_us":1804498,"threads_started":1,"wal-append.queue_time_us":387693,"wal-append.run_cpu_time_us":2080,"wal-append.run_wall_time_us":1002092}
I20260709 14:53:04.121431  6677 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:53:04.121908  6677 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:03.950769  6495 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:03.947623  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1352) took 2132 ms (client timeout 1514 ms). Trace:
I20260709 14:53:03.955050  6737 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:04.123487  2606 rpcz_store.cc:269] 0709 14:53:01.815064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.838649 (+ 23585us) service_pool.cc:224] Handling call
0709 14:53:03.947339 (+2108690us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:03.947503 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4679714688}
I20260709 14:53:04.125591  6737 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.922493  2603 rpcz_store.cc:276] 0709 14:53:02.161172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:03.777766 (+1616594us) service_pool.cc:224] Handling call
0709 14:53:03.781052 (+  3286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:03.855033 (+ 73981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:03.855075 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:03.855851 (+   776us) log.cc:844] Serialized 26 byte log entry
0709 14:53:03.892876 (+ 37025us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:03.892933 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.896458 (+  3525us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:03.921760 (+ 25302us) raft_consensus.cc:1685] finished
0709 14:53:03.921783 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:03.921921 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44890,"raft.queue_time_us":46174,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":36536,"threads_started":1,"wal-append.queue_time_us":47201}
I20260709 14:53:04.126945  6756 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:04.127396  6756 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.127780  6756 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:03.956205  6696 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:03.927147  6701 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:03.951474  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 2.01763e+06us
W20260709 14:53:04.135711  2650 connection.cc:256] RPC call timeout handler was delayed by -2.0599544010000272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 14:53:03.957437  6359 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.960680  2723 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.518s (SENT) [suppressed 41 similar messages]
W20260709 14:53:04.138973  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 178327us
I20260709 14:53:04.139468  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:03.962224  2677 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:53:03.957516  6637 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:03.965776  2753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.862s	user 0.013s	sys 0.005s
W20260709 14:53:03.966327  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.168s	user 0.046s	sys 0.028s
I20260709 14:53:03.806929  6727 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.946472  2607 rpcz_store.cc:276] 0709 14:53:01.704751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.790075 (+ 85324us) service_pool.cc:224] Handling call
0709 14:53:01.790436 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.733902 (+1943466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.749216 (+ 15314us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54002cdb54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:03.749597 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794614656}
I20260709 14:53:03.968103  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1122) took 2033 ms. Trace:
I20260709 14:53:04.157181  2685 rpcz_store.cc:276] 0709 14:53:01.934368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.940797 (+  6429us) service_pool.cc:224] Handling call
0709 14:53:02.103276 (+162479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:03.967610 (+1864334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:03.968013 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:03.984145  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:03.984925  6758 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:03.985146  6202 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:03.985438  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:03.986804  3542 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:03.987185  6700 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:03.985178  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:03.926574  6733 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:03.989845  6732 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.990603  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:04.168145  3546 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:03.990986  6742 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:03.991233  6738 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:03.999413  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:03.830744  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:03.998682  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800376b20 after lost signal to thread 2785
W20260709 14:53:04.172432  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08003b39a0 after lost signal to thread 3580
W20260709 14:53:04.172762  2633 debug-util.cc:398] Leaking SignalData structure 0x7b080031cce0 after lost signal to thread 3581
W20260709 14:53:04.173588  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08000fde20 after lost signal to thread 6636
I20260709 14:53:04.001768  6745 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.008066  2682 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:04.011552  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1355) took 1845 ms. Trace:
W20260709 14:53:04.013897  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1342) took 2381 ms (client timeout 1647 ms). Trace:
W20260709 14:53:04.177177  2605 rpcz_store.cc:269] 0709 14:53:01.636239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.684092 (+ 47853us) service_pool.cc:224] Handling call
0709 14:53:04.013584 (+2329492us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54002feed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:04.013789 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166730496}
I20260709 14:53:04.017135  6762 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:04.020344  2724 connection.cc:256] RPC call timeout handler was delayed by -1.6880514829999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 14:53:04.007504  6747 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.010946  6739 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 14:53:04.047487  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:04.048415  3411 connection.cc:256] RPC call timeout handler was delayed by -1.3621365899999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260709 14:53:04.138634  6741 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 14:53:04.182761  3411 connection.cc:256] RPC call timeout handler was delayed by -1.2667365900000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 14:53:04.183271  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1704365900000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 14:53:04.183641  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1146365900000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 14:53:04.184033  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0894365899999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 14:53:04.183284  2685 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:04.011013  6749 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.154860  6359 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.174750  6732 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:04.188513  6762 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.189100  6749 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.189435  6359 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.191154  6739 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.191869  6739 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:04.194276  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.052845  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:04.049289  6751 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:04.061112  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 690) took 1870 ms. Trace:
I20260709 14:53:04.063328  6616 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 14:53:04.064728  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1116) took 2133 ms (client timeout 2106 ms). Trace:
I20260709 14:53:04.197129  2683 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:04.086577  2609 raft_consensus.cc:606] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.089344  2679 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:04.072732  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:04.197453  2686 rpcz_store.cc:269] 0709 14:53:01.931282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.931745 (+   463us) service_pool.cc:224] Handling call
0709 14:53:04.064396 (+2132651us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400269154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:04.064640 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4847102848}
I20260709 14:53:04.198200  2609 raft_consensus.cc:614] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:04.102898  6693 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:04.105465  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:04.106060  2423 maintenance_manager.cc:643] P 3f6dfe8cfa964684b547b44a4b308ca2: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":351,"lbm_reads_lt_1ms":4}
I20260709 14:53:03.947104  6734 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.203042  3540 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:04.197147  2604 rpcz_store.cc:276] 0709 14:53:02.190333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:03.948999 (+1758666us) service_pool.cc:224] Handling call
0709 14:53:03.949479 (+   480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:03.949945 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:03.949985 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:03.950668 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 14:53:04.018438 (+ 67770us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:04.018887 (+   449us) log.cc:844] Serialized 14 byte log entry
0709 14:53:04.019151 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.019259 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:04.060819 (+ 41560us) raft_consensus.cc:1685] finished
0709 14:53:04.060840 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.061035 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3027,"prepare.queue_time_us":242,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":4970,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":9114,"thread_start_us":67282,"threads_started":1,"wal-append.queue_time_us":67758}
I20260709 14:53:04.109980  2611 rpcz_store.cc:276] 0709 14:53:02.199796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.086173 (+1886377us) service_pool.cc:224] Handling call
0709 14:53:04.092964 (+  6791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:04.093167 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:04.093194 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:04.093250 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:04.093281 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.093335 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.093461 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:04.114760  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:03.936553  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.180222  6727 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:04.182566  6747 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:04.119086  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:04.209321  2752 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:04.119812  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:04.210157  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:04.210922  2758 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 14:53:04.119381  2431 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:04.116308  6702 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:04.211222  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.147439  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.140427  3541 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
W20260709 14:53:04.149073  2650 connection.cc:256] RPC call timeout handler was delayed by -1.9996544010000434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 14:53:04.212038  2650 connection.cc:256] RPC call timeout handler was delayed by -1.999234781000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 14:53:04.212330  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 97638.8us
W20260709 14:53:04.212705  2650 connection.cc:256] RPC call timeout handler was delayed by -1.5875503290000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 14:53:04.213205  2650 connection.cc:256] RPC call timeout handler was delayed by -1.4876503290000755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 14:53:04.213214  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:04.213589  2650 connection.cc:256] RPC call timeout handler was delayed by -1.4759503290000566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
W20260709 14:53:04.213937  2650 connection.cc:256] RPC call timeout handler was delayed by -1.3649240500000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 14:53:04.214184  2650 connection.cc:256] RPC call timeout handler was delayed by -1.202350329000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 14:53:04.214421  2650 connection.cc:256] RPC call timeout handler was delayed by -1.1978503290000617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 14:53:04.214730  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0334038269999155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 14:53:04.215190  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0038197950000267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260709 14:53:04.160662  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1312) took 2406 ms. Trace:
I20260709 14:53:04.162719  3547 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.158133  6640 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:04.128549  6712 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:04.179768  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 349083us
I20260709 14:53:04.219990  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.220388  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:04.180362  2724 connection.cc:256] RPC call timeout handler was delayed by -1.6532216599999856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 14:53:04.153939  6244 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.177048  2608 rpcz_store.cc:276] 0709 14:53:02.165697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:03.935617 (+1769920us) service_pool.cc:224] Handling call
0709 14:53:03.936110 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:03.958742 (+ 22632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:03.958790 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:03.959423 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 14:53:03.976409 (+ 16986us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:03.976457 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.007664 (+ 31207us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54002ff8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:04.007722 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:04.007773 (+    51us) raft_consensus.cc:1685] finished
0709 14:53:04.007787 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.011444 (+  3657us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2413,"raft.queue_time_us":2730,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":43355008,"thread_start_us":16550,"threads_started":1,"wal-append.queue_time_us":43654}
I20260709 14:53:04.199645  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:04.225318  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.225543  3542 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:04.225432  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:04.159669  6637 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:04.228696  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.229401  2607 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:04.226195  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:04.194698  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 704) took 2001 ms. Trace:
I20260709 14:53:04.116787  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:04.117861  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:04.127526  6677 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.252811  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.216318  2759 rpcz_store.cc:276] 0709 14:53:01.754153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.754329 (+   176us) service_pool.cc:224] Handling call
0709 14:53:01.769539 (+ 15210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:01.797584 (+ 28045us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:01.797955 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 14:53:04.160078 (+2362123us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:04.160179 (+   101us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:04.160263 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.160374 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.160573 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":972,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":70,"raft.queue_time_us":8054,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2329,"thread_start_us":2354813,"threads_started":1,"wal-append.queue_time_us":312763,"wal-append.run_cpu_time_us":1244,"wal-append.run_wall_time_us":1001275}
I20260709 14:53:04.206271  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:04.221498  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.207289  6749 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:04.227738  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.194366  6739 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:04.199378  6616 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:04.255201  2724 connection.cc:256] RPC call timeout handler was delayed by -1.4037421919999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
I20260709 14:53:04.189211  6732 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:04.255893  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 138172us
W20260709 14:53:04.272197  2724 connection.cc:256] RPC call timeout handler was delayed by -1.3887770640000099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 14:53:04.273188  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.254657  2605 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:04.259385  2755 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:04.260707  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:04.262488  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.264230  3540 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.275763  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:04.265398  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:04.265237  2680 rpcz_store.cc:276] 0709 14:53:02.193599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.197140 (+  3541us) service_pool.cc:224] Handling call
0709 14:53:02.197384 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:04.087857 (+1890473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:04.087898 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:04.088500 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 14:53:04.161961 (+ 73461us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:04.162040 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.162138 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:04.194400 (+ 32262us) raft_consensus.cc:1685] finished
0709 14:53:04.194428 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.194616 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115488,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"thread_start_us":1962896,"threads_started":2,"wal-append.queue_time_us":102123}
I20260709 14:53:04.267747  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:04.270356  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.271237  2681 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:04.279582  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.281096  6637 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:04.281262  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.281965  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:04.282217  2685 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:04.228405  6734 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 14:53:04.217983  6394 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.175947  6742 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.281689  2431 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 3 to 7, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:04.263212  6202 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.297484  6202 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.299691  6701 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.300450  6202 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.301900  6394 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:04.303194  2677 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:04.283469  6677 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:53:04.305379  6394 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.170174  6700 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.262143  6622 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 14:53:04.289196  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.309044  3543 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
W20260709 14:53:04.287308  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67023.9us
I20260709 14:53:04.294703  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.276631  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:04.282914  3546 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:04.290483  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.311784  3545 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:04.312335  2752 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.292140  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.313239  3547 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:04.315980  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.222115  6715 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:04.223402  6540 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.232430  6720 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:04.321254  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48165.6us
I20260709 14:53:04.340885  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:04.340994  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.341620  2611 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:04.322532  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.307734  6700 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.360255  2686 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:04.360543  2608 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:04.327180  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.333174  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:04.217943  6702 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.341958  2685 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:04.361580  2755 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:04.360599  2683 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.355618  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:04.362500  2752 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.338568  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:04.325412  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.272167  6749 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.271967  6741 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:04.354321  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.375306  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.285291  6733 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:04.293011  6244 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.380548  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.356992  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.284632  6359 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:04.288153  6688 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:04.346096  6495 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.369817  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:04.374272  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 148260us
I20260709 14:53:04.262053  6762 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.377280  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.377461  2605 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:04.228788  6640 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:04.397086  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.358450  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.377640  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.396067  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.398418  3540 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:04.379578  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.335479  6698 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:04.382088  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.405746  6698 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:04.406141  2608 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:04.406241  6766 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 14:53:04.406296  6698 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:04.384610  6359 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:04.408864  6726 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:04.409503  6726 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:04.359784  6540 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:04.380165  6733 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:04.382537  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 215773us
I20260709 14:53:04.285181  6745 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.331813  6202 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:04.340000  6758 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:04.292546  6738 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.331815  6676 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:04.363229  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:04.484267  2607 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:53:04.484586  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 121468us
W20260709 14:53:04.386495  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76131.1us
I20260709 14:53:04.486671  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:04.386759  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.488236  2682 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:04.387326  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.387840  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.489928  2680 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:04.490031  2681 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:53:04.293565  6619 consensus_queue.cc:1175] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 7 status { last_received { term: 6 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:53:04.347538  6637 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:04.203730  6751 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.350358  6631 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.345763  6720 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:04.495092  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.376109  6749 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:04.395283  2609 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 5: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:04.508397  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.395593  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:04.366969  6702 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:04.377092  2677 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 14:53:04.378661  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.399485  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.382748  6394 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:04.381059  6741 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:04.380213  6675 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:04.402937  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 137633us
I20260709 14:53:04.485550  6676 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:04.405020  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:04.431304  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:04.526062  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 121156us
I20260709 14:53:04.437372  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.494712  6626 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:04.446571  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:04.447149  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90309.3us
W20260709 14:53:04.447633  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 253708us
I20260709 14:53:04.375214  6715 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 14:53:04.456099  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.291035  6693 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.403558  6743 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:04.480952  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.416039  6742 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:04.457720  2678 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:04.399256  6495 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:04.456269  6540 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:04.410499  6726 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:04.501116  2604 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:04.511152  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 111762us
W20260709 14:53:04.553198  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.546595  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.500527  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:04.520816  3538 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:04.403785  6740 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:04.522715  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:04.557806  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:04.528256  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91022.3us
I20260709 14:53:04.559576  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.560400  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.444356  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:04.519555  6648 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:04.529915  3541 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:04.527525  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:04.503993  6720 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:04.534039  2686 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:04.539067  2685 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:04.540277  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.513125  6394 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.522231  6741 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:04.540962  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.556283  2680 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:04.526281  6637 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:04.529182  6676 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:04.562297  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.538976  6737 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:04.566776  3537 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:04.568858  2608 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:04.569880  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080024a620 after lost signal to thread 431
W20260709 14:53:04.665822  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080024a900 after lost signal to thread 434
I20260709 14:53:04.558275  6712 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:04.571542  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.574442  6753 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:04.580704  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.586440  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.588888  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.533602  6626 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.608139  2684 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:04.513459  6631 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:04.591655  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 147414us
I20260709 14:53:04.642812  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:04.563010  6740 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.544514  6715 consensus_queue.cc:1243] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:04.543864  6743 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:04.667440  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0060 after lost signal to thread 2469
W20260709 14:53:04.571295  6774 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 14:53:04.528712  6675 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.736230  6613 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:04.751036  6781 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:04.751674  6613 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.752346  6781 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:04.753026  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:04.755132  6773 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 14:53:04.757275  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1357) took 1114 ms. Trace:
I20260709 14:53:04.547886  6374 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:04.573573  6756 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.698076  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:04.568416  6540 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:04.590533  6394 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:04.705655  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 119331us
I20260709 14:53:04.715885  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:04.720443  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162727us
W20260709 14:53:04.721750  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 150273us
W20260709 14:53:04.722633  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:04.522588  6733 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:04.641672  6202 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:04.640623  6648 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:04.647626  6642 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:04.733263  6715 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:04.733796  6791 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:04.664077  6737 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:04.738538  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080024a0e0 after lost signal to thread 2636
I20260709 14:53:04.590793  6758 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 14:53:04.729182  6740 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:04.764554  2605 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:04.764467  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 237008us
I20260709 14:53:04.714175  6626 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:04.707222  6753 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:04.544013  6783 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.185s
I20260709 14:53:04.776682  2604 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:04.717506  6712 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:04.705308  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.720377  6696 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 14:53:04.758034  2606 rpcz_store.cc:276] 0709 14:53:03.643062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.124736 (+481674us) service_pool.cc:224] Handling call
0709 14:53:04.125320 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:04.125836 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:04.125875 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:04.126589 (+   714us) log.cc:844] Serialized 26 byte log entry
0709 14:53:04.219428 (+ 92839us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:04.219473 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:04.219549 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:04.756904 (+537355us) raft_consensus.cc:1685] finished
0709 14:53:04.756942 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:04.757178 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1240,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":89753,"threads_started":1,"wal-append.queue_time_us":219500}
I20260709 14:53:04.699450  6777 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:04.766186  6640 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.967609  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.743714  6743 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:05.012117  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 306952us
I20260709 14:53:04.700663  6616 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:04.974593  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 394213us
I20260709 14:53:04.977494  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:04.983834  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:04.986912  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:04.858229  6784 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 14:53:04.867413  6675 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:04.856805  6788 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 14:53:04.967865  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000815a0 after lost signal to thread 2683
W20260709 14:53:05.023801  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000cbca0 after lost signal to thread 2685
W20260709 14:53:05.024510  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080037d1c0 after lost signal to thread 2710
I20260709 14:53:04.752413  6613 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:04.780885  6374 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.007877  2759 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:05.015030  2755 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:53:04.862221  6631 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.865147  6751 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:04.976130  6737 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:05.013422  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:04.849578  6540 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:05.084126  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70866.8us
I20260709 14:53:05.016552  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:04.969100  6715 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:05.027887  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:05.030390  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080036bbe0 after lost signal to thread 2711
I20260709 14:53:04.859280  6778 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.533s
I20260709 14:53:05.125813  2610 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 14:53:05.006081  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 290283us
I20260709 14:53:04.996915  6753 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:05.074867  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.002s	sys 0.023s
W20260709 14:53:05.134868  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.003s	sys 0.023s
W20260709 14:53:05.136547  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 649988us
I20260709 14:53:05.139312  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:05.113371  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96933.6us
W20260709 14:53:05.117640  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.024921  6726 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:05.031539  6677 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:53:05.118922  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5560 after lost signal to thread 2723
W20260709 14:53:05.148079  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800329a00 after lost signal to thread 2724
W20260709 14:53:05.148518  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5900 after lost signal to thread 2726
I20260709 14:53:04.842653  6637 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:05.120357  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:05.023209  6701 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:05.027734  6738 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:05.030627  6733 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:05.076972  2753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.003s	sys 0.023s
I20260709 14:53:05.028553  6777 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:05.144915  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:05.149434  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7b80 after lost signal to thread 2755
W20260709 14:53:05.164458  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080026aaa0 after lost signal to thread 2756
W20260709 14:53:05.164821  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000dbf00 after lost signal to thread 2757
W20260709 14:53:05.165113  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080026ab20 after lost signal to thread 2758
W20260709 14:53:05.028069  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 495292us
I20260709 14:53:05.014374  6791 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 14:53:05.028020  6642 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 14:53:05.122390  6640 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:53:05.040686  6687 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260709 14:53:05.158264  2753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.003s	sys 0.023s
I20260709 14:53:05.188593  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1324) took 3301 ms. Trace:
I20260709 14:53:05.124678  6715 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:05.160168  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.125535  6751 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:05.128809  6764 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:05.165908  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800169f60 after lost signal to thread 2784
I20260709 14:53:05.166787  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:05.165782  6645 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.020s	sys 0.015s
W20260709 14:53:05.153996  6467 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.029s	sys 0.082s
I20260709 14:53:05.157850  6701 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:05.157855  6762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.016s	sys 0.019s
I20260709 14:53:05.132210  6540 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.176543  6495 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:05.189179  2753 rpcz_store.cc:276] 0709 14:53:01.889992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.930386 (+ 40394us) service_pool.cc:224] Handling call
0709 14:53:01.934099 (+  3713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:01.934373 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:04.152023 (+2217650us) log.cc:844] Serialized 14 byte log entry
0709 14:53:04.491876 (+339853us) log.cc:844] Serialized 14 byte log entry
0709 14:53:05.187921 (+696045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:05.188039 (+   118us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:05.188126 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:05.188244 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:05.188491 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1576647,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":96,"raft.queue_time_us":119060,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":24769,"spinlock_wait_cycles":8084864,"thread_start_us":3081129,"threads_started":5,"wal-append.queue_time_us":420023}
I20260709 14:53:05.162721  6631 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:05.195191  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4a20 after lost signal to thread 2785
I20260709 14:53:05.154533  6796 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:05.142977  6726 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:05.196805  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 229300us
I20260709 14:53:05.161329  6677 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:05.212177  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45455.9us
I20260709 14:53:05.028286  6648 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:05.239182  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:05.214522  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75298.9us
I20260709 14:53:05.213922  6640 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:05.217996  6756 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:04.754045  6781 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:05.197683  6645 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.021s	sys 0.016s
W20260709 14:53:05.200482  6467 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.029s	sys 0.083s
W20260709 14:53:05.264853  6720 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 14:53:05.193356  6727 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:05.211088  6762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.016s	sys 0.019s
W20260709 14:53:05.230620  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1780 after lost signal to thread 3580
W20260709 14:53:05.254792  6792 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 14:53:05.238358  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.239456  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.240442  6726 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:05.245325  6798 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 14:53:05.244977  6778 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 14:53:05.260599  6797 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:05.239462  6783 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 14:53:05.217161  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 204093us
I20260709 14:53:05.238701  6631 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:05.242697  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:05.181524  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 259466us
W20260709 14:53:05.286525  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:05.290817  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 313447us
I20260709 14:53:05.227456  6786 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.270s
I20260709 14:53:05.237193  6801 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:53:05.318728  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003c38c0 after lost signal to thread 3581
I20260709 14:53:05.202008  6701 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:05.247392  6640 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:05.229956  6799 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 14:53:05.340086  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101849us
W20260709 14:53:05.356349  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 236091us
W20260709 14:53:05.360231  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117643us
W20260709 14:53:05.357368  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117984us
W20260709 14:53:05.369191  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003c40c0 after lost signal to thread 6244
W20260709 14:53:05.362344  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.377501  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:05.379197  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.384992  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:05.352656  6796 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:05.380100  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.380537  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.371872  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:05.318214  6720 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:05.378963  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800214700 after lost signal to thread 6374
I20260709 14:53:05.396404  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:05.396520  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080019bfc0 after lost signal to thread 6394
I20260709 14:53:05.391263  3540 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:05.351853  6797 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:05.347373  6806 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:05.399497  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:05.400312  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000eaa60 after lost signal to thread 6414
W20260709 14:53:05.415979  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab140 after lost signal to thread 6467
W20260709 14:53:05.400282  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 234896us
W20260709 14:53:05.416451  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800350b40 after lost signal to thread 6495
W20260709 14:53:05.416846  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080014f260 after lost signal to thread 6600
W20260709 14:53:05.417191  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080036cfa0 after lost signal to thread 6612
W20260709 14:53:05.417436  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080035bb80 after lost signal to thread 6613
W20260709 14:53:05.417788  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800376f40 after lost signal to thread 6625
W20260709 14:53:05.418057  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800289f00 after lost signal to thread 6631
I20260709 14:53:05.374393  6640 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:05.418401  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003020a0 after lost signal to thread 6642
I20260709 14:53:05.420672  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.400698  3542 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 14:53:05.399081  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:05.422430  3541 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:05.408427  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.423836  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.369741  6631 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:05.282106  6777 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:05.376034  6798 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:05.375325  6783 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:05.357642  6790 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:05.436512  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51661.5us
I20260709 14:53:05.400645  6791 consensus_queue.cc:1243] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:05.443814  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.445842  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:05.424847  6756 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:05.446991  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:05.447290  6756 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:05.450980  6756 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:05.451771  6698 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.045s	user 0.014s	sys 0.000s
I20260709 14:53:05.452344  6698 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:05.426702  6808 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 14:53:05.432380  6687 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.432679  6738 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:05.435848  6645 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:05.406752  6807 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:05.373906  6677 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:05.469179  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92740.4us
I20260709 14:53:05.441483  6762 raft_consensus.cc:2764] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:53:05.430473  6244 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.449975  6791 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:53:05.478793  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101430us
I20260709 14:53:05.468767  6698 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.492774  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:05.495187  2684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:05.496517  2684 raft_consensus.cc:606] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Received request to transfer leadership to TS 56620e612f0549e6abf320aa80398e74
I20260709 14:53:05.499106  6786 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:53:05.499336  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52570.1us
I20260709 14:53:05.501843  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:05.503527  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:05.503223  6786 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:05.504683  6801 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:05.505564  6767 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:53:05.506343  6767 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:05.507381  6812 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.508255  6812 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:05.502710  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 686) took 3701 ms (client timeout 3071 ms). Trace:
I20260709 14:53:05.503894  2607 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 14:53:05.527073  2607 pending_rounds.cc:85] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:53:05.527419  2607 pending_rounds.cc:107] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 14:53:05.527683  2607 raft_consensus.cc:2889] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:05.515048  2610 rpcz_store.cc:269] 0709 14:53:01.800968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.165980 (+365012us) service_pool.cc:224] Handling call
0709 14:53:05.502475 (+3336495us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:05.502636 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6800212096}
I20260709 14:53:05.472208  6797 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.528131  6817 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 14:53:05.553323  6702 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.028s	user 0.013s	sys 0.000s
I20260709 14:53:05.553860  6702 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:05.487396  6810 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.555176  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:05.555888  6743 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.007s
I20260709 14:53:05.556326  6743 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.483719  6807 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:05.479457  6738 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:05.495729  6688 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.102s	user 0.014s	sys 0.001s
I20260709 14:53:05.559818  2603 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:05.567173  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.568084  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.490976  6645 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:05.506784  6767 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:05.546880  6805 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
W20260709 14:53:05.549105  6813 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 14:53:05.546634  6803 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 14:53:05.578205  2612 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:05.483017  6687 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:05.547545  6809 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 14:53:05.590497  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1366) took 1193 ms. Trace:
I20260709 14:53:05.591141  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.569104  6738 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:05.603561  6645 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.547444  6820 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:05.609751  6820 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:05.610431  6820 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:05.613257  6749 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.086s	user 0.013s	sys 0.000s
I20260709 14:53:05.613878  6749 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.596982  2604 rpcz_store.cc:276] 0709 14:53:04.397114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.002316 (+605202us) service_pool.cc:224] Handling call
0709 14:53:05.024256 (+ 21940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:05.043619 (+ 19363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:05.043672 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:05.044417 (+   745us) log.cc:844] Serialized 26 byte log entry
0709 14:53:05.414242 (+369825us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:05.414308 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:05.414403 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:05.590208 (+175805us) raft_consensus.cc:1685] finished
0709 14:53:05.590234 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:05.590423 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18733,"raft.queue_time_us":18988,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9716,"thread_start_us":275775,"threads_started":1,"wal-append.queue_time_us":408373}
I20260709 14:53:05.555258  6810 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.619691  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 700) took 1289 ms. Trace:
I20260709 14:53:05.620978  6810 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:05.621423  6720 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.563426  6807 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:05.598701  6675 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.528748  6783 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.620234  2608 rpcz_store.cc:276] 0709 14:53:04.330481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.689388 (+358907us) service_pool.cc:224] Handling call
0709 14:53:04.695095 (+  5707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:04.695818 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:04.742723 (+ 46905us) log.cc:844] Serialized 14 byte log entry
0709 14:53:05.339368 (+596645us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:05.351654 (+ 12286us) log.cc:844] Serialized 26 byte log entry
0709 14:53:05.351883 (+   229us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:05.351938 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:05.352047 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:05.619404 (+267357us) raft_consensus.cc:1685] finished
0709 14:53:05.619435 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:05.619612 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51161,"prepare.queue_time_us":47555,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":70,"raft.queue_time_us":33082,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":10006,"thread_start_us":369636,"threads_started":2,"wal-append.queue_time_us":398617}
I20260709 14:53:05.496032  6244 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.581560  6767 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:05.636814  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:05.641710  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 147933us
I20260709 14:53:05.643239  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:05.648746  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.650043  3543 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:05.577212  6762 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 14:53:05.568830  6817 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 14:53:05.655293  6613 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.003s
W20260709 14:53:05.656677  2711 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:39575 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:39575 timed out after 15.000s (SENT)
W20260709 14:53:05.660077  6676 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.960s	user 0.008s	sys 0.006s
I20260709 14:53:05.631737  6675 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.646270  6825 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:53:05.669906  6747 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.437s	user 0.012s	sys 0.005s
I20260709 14:53:05.630959  6806 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:05.675349  2684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:05.679280  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.695753  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.680533  6640 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.681874  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:05.707633  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.633231  6743 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:05.684343  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:05.666356  6626 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.001s
I20260709 14:53:05.687192  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.639505  6791 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 14:53:05.730499  2686 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:05.732646  2682 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:05.675257  6810 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.736085  2679 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:05.736591  2681 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:53:05.741465  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:05.741536  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.656755  6824 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:05.743011  2686 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:05.691032  6745 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.745955  6745 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.748517  6745 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:05.750171  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120627us
I20260709 14:53:05.687984  6698 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.753064  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.691814  6825 consensus_queue.cc:1243] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 14:53:05.755088  6394 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.770s	user 0.014s	sys 0.000s
I20260709 14:53:05.684384  6613 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.740214  6626 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:05.759316  6823 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:05.684731  6676 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:05.764575  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.782451  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.772949  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:05.776968  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.760710  6812 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.780692  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.733700  6747 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:05.781454  6737 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.000s
I20260709 14:53:05.843178  6737 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.759375  6702 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:05.786353  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:05.809262  6749 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.777376  6394 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.750578  6810 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.825714  6701 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.806973  3541 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:05.833081  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1218) took 1677 ms. Trace:
I20260709 14:53:05.831964  6726 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:05.835412  6613 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:05.836483  6414 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:53:05.789999  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.838033  6626 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.839176  6687 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:05.896548  6741 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.185s	user 0.009s	sys 0.009s
I20260709 14:53:05.841205  6616 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:05.842265  6676 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:05.845955  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69078.4us
W20260709 14:53:05.845881  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65248.5us
W20260709 14:53:05.846957  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:05.931740  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:05.915838  6753 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.005s
I20260709 14:53:05.838045  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 706) took 1468 ms. Trace:
I20260709 14:53:05.825743  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:05.871918  3543 rpcz_store.cc:276] 0709 14:53:04.155879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.346037 (+190158us) service_pool.cc:224] Handling call
0709 14:53:05.832711 (+1486674us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400224cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:05.832933 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876351232}
I20260709 14:53:05.934625  2609 rpcz_store.cc:276] 0709 14:53:04.370002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.642401 (+272399us) service_pool.cc:224] Handling call
0709 14:53:05.837762 (+1195361us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:05.837959 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739773696}
I20260709 14:53:05.935365  3546 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9. Re-sending same reply.
I20260709 14:53:05.767004  6698 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:05.883996  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:05.862465  6467 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.867005  6815 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:05.865738  6786 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:05.871812  6740 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.863s	user 0.013s	sys 0.002s
I20260709 14:53:05.949133  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:05.914978  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:05.840145  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:05.910548  6414 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:05.781994  6824 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
W20260709 14:53:05.926741  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 250410us
W20260709 14:53:05.941787  6712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.015s	sys 0.043s
I20260709 14:53:05.936823  3541 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:05.936925  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.959107  3546 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:05.960145  2608 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:05.943747  2684 raft_consensus.cc:606] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:05.962709  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:05.963486  3546 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:05.950731  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:05.923377  6756 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.955251  6613 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:05.999002  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:05.972218  6715 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.005s
I20260709 14:53:05.972049  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:05.976630  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.007952  2681 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:05.977474  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:05.979460  6640 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:05.949136  6814 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.973165  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:05.973615  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:05.985150  6394 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:05.986238  6747 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:05.961536  6676 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:06.039170  6244 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.039513  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.040388  3547 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.044322  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 708) took 1673 ms. Trace:
I20260709 14:53:06.044526  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.045331  2681 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:06.046419  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:05.987639  6726 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:05.987972  6810 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:05.981925  6645 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:05.972293  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.068037  2604 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 14:53:05.992295  6733 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.826s	user 0.015s	sys 0.000s
I20260709 14:53:05.992599  6820 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.970774  6700 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:05.946748  6467 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.993777  6738 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:05.954892  6783 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:05.984234  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1356) took 3806 ms (client timeout 2870 ms). Trace:
W20260709 14:53:05.995437  6830 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 14:53:05.995039  6414 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:05.995924  6701 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:05.997390  6822 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.398s	user 0.000s	sys 0.003s
I20260709 14:53:05.942127  6815 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:06.002301  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:05.952220  6742 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:05.964990  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:05.927657  6741 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.003958  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:06.016007  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44438us
I20260709 14:53:06.115479  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:06.117915  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114005us
I20260709 14:53:06.118155  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.013957  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.035454  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.119688  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.037042  6756 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.126768  2606 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:05.973249  6828 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 14:53:05.987082  6764 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:06.044952  2603 rpcz_store.cc:276] 0709 14:53:04.370623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.546187 (+175564us) service_pool.cc:224] Handling call
0709 14:53:06.044064 (+1497877us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:06.044245 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308234112}
W20260709 14:53:05.964627  6712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.015s	sys 0.044s
I20260709 14:53:06.051190  2681 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.133595  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:05.943401  6753 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.057219  2683 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:06.063400  3541 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:06.136893  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.137893  2609 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:05.983963  6737 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.008795  6613 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:06.079677  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.080909  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.084843  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.086141  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.087451  2757 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:53:06.088833  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.035494  6749 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:06.089018  2612 rpcz_store.cc:269] 0709 14:53:02.178302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:02.184611 (+  6309us) service_pool.cc:224] Handling call
0709 14:53:05.984016 (+3799405us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54003c8554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:05.984160 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4206978944}
W20260709 14:53:06.088003  6831 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 14:53:06.015331  6715 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:06.008116  6767 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.097131  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.011646  6824 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 14:53:06.097961  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.023605  6762 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 14:53:06.103982  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:06.104648  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102419us
I20260709 14:53:06.112888  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.115479  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:06.096650  6414 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:06.121088  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:06.096669  6631 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.006s
I20260709 14:53:06.039209  6676 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:06.092000  6783 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:06.051983  6814 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:06.135622  3546 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:06.134461  3543 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:05.956954  6740 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.188956  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:06.205904  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84910.5us
I20260709 14:53:06.171301  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.206642  2755 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:06.207024  2682 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:06.206091  2758 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:06.207880  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:06.138579  3547 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:06.208611  3538 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:06.193459  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.174208  3536 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:06.112213  6815 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.073139  6645 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:06.116802  6726 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:06.218529  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:06.223461  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.129197  6786 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:06.232610  2679 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:53:06.128130  6701 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:53:06.235252  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.128358  6700 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:06.133409  6764 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:06.248276  2685 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:06.249713  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:06.092658  6637 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 14:53:06.214241  6613 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:06.242203  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.237407  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.214056  6783 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:06.163247  6840 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 14:53:06.244179  2677 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:06.246623  3538 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:06.232370  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.261241  2612 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:06.261788  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.262519  2684 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:06.215595  6749 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:06.166548  6756 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.253230  3545 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:53:06.133536  6712 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:06.178121  6741 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.254818  2608 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:53:06.306260  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.125990  6742 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.255796  2611 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:06.239794  3543 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:53:06.218394  6733 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:06.175647  6467 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.166213  6626 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:06.304654  6700 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.312372  6626 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.312942  6700 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:06.258626  2682 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:06.316766  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.318162  6797 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.259536  2683 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:06.320340  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.321575  2683 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:06.322477  6751 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.266709  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.271977  6820 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:06.273447  6826 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 14:53:06.274623  6789 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.748s	user 0.006s	sys 0.000s
I20260709 14:53:06.247777  6798 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:06.275059  6738 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.265108  6701 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:06.275617  6749 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.275964  6202 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:06.255542  6786 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:06.276494  6799 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 14:53:06.232538  6244 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 14:53:06.279224  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 348845us
I20260709 14:53:06.218448  6631 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.280498  6359 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:06.281575  6807 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.000s
I20260709 14:53:06.232136  6747 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:06.275084  6793 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:06.284426  6693 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.288851  6645 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.291204  6675 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.279146  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.293836  6753 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.294920  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.309514  6756 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.296123  3538 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [8.6-11.7]
I20260709 14:53:06.297887  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.299861  2681 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:06.300755  2684 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:06.303203  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 701) took 1972 ms. Trace:
W20260709 14:53:06.304239  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54842.7us
I20260709 14:53:06.322785  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.368711  2683 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:53:06.323391  6751 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:06.372925  6751 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.324048  2612 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:06.373698  6815 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:06.375475  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.377475  2612 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:06.325987  6742 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.327114  6712 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.379170  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.431079  2612 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:53:06.433020  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.349601  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.355859  3537 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:53:06.356658  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.438306  2756 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:06.358450  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.440593  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.441442  2757 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.361620  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:06.443307  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.318169  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.363201  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.446514  2753 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:06.328713  6740 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.331177  6202 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:06.331898  6676 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.332249  6786 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:06.332516  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.335242  3543 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:06.335412  6244 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:06.336304  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.340970  6359 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.341368  6807 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:06.334282  6767 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:06.390079  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:06.495820  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 105833us
I20260709 14:53:06.496847  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:06.341856  6747 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:06.498072  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:06.391480  6742 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.343978  3540 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 14:53:06.344870  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50093.5us
I20260709 14:53:06.345201  6675 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:53:06.346006  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48228us
I20260709 14:53:06.343778  2679 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:06.396276  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.397614  6790 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:06.400818  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.557543  2685 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:06.405390  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:06.410530  6795 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.299s	user 0.000s	sys 0.014s
I20260709 14:53:06.412467  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.450428  6616 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.347222  6645 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:53:06.347271  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.417465  6715 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.347592  2607 rpcz_store.cc:276] 0709 14:53:04.330889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.713663 (+382774us) service_pool.cc:224] Handling call
0709 14:53:05.503488 (+789825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:05.528283 (+ 24795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:06.302632 (+774349us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400135950. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:06.303097 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815416576}
I20260709 14:53:06.599121  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.600622  2607 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:06.600912  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.601410  2607 raft_consensus.cc:606] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.602108  2682 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:53:06.349362  3411 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.506s (SENT)
I20260709 14:53:06.603374  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.420974  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.423923  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.425771  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1075) took 1046 ms. Trace:
I20260709 14:53:06.604987  3536 rpcz_store.cc:276] 0709 14:53:05.378902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.379397 (+   495us) service_pool.cc:224] Handling call
0709 14:53:06.425331 (+1045934us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b5400200154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:06.425665 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743698176}
W20260709 14:53:06.427281  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 703) took 2067 ms (client timeout 1606 ms). Trace:
W20260709 14:53:06.606444  2611 rpcz_store.cc:269] 0709 14:53:04.360254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.378374 (+ 18120us) service_pool.cc:224] Handling call
0709 14:53:06.427002 (+2048628us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54000ac154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:06.427181 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3997196672}
I20260709 14:53:06.608548  2611 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:06.349692  3538 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:06.455233  2752 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:06.456876  2612 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:06.615126  6807 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:06.459460  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.469108  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:06.470110  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120626us
W20260709 14:53:06.470374  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 108884us
W20260709 14:53:06.474591  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 138435us
I20260709 14:53:06.475271  2608 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.480085  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.483471  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.354990  6414 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 14:53:06.511793  6753 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:06.489343  6824 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:06.494498  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:06.645920  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.499387  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.646799  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.646974  3543 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.346949  6756 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.595783  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:06.513140  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.657636  2686 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 14:53:06.555361  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57388.4us
I20260709 14:53:06.383036  6626 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.349462  6762 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:06.665019  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.565675  6790 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:06.662814  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 67152.2us
I20260709 14:53:06.415424  6374 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:06.365713  6814 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:06.363305  6631 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:06.324615  6826 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:06.568109  2610 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:06.564745  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.586014  2757 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:06.490850  6793 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:06.596843  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:06.348078  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:06.504163  6820 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:06.616349  2612 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 14:53:06.618394  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 269049us
I20260709 14:53:06.695101  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.629592  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:06.916149  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 286678us
I20260709 14:53:06.630438  3541 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.917523  2755 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:53:06.702517  3545 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:53:06.631383  2753 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:06.688717  3538 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:06.631562  2752 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:06.706161  2607 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:06.512477  6701 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:06.690021  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:06.644241  6715 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:06.647784  2754 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:06.588383  6751 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 14:53:06.355180  6798 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.202s
I20260709 14:53:06.645124  6776 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:06.504062  6817 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 14:53:06.671314  6807 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:06.697008  6766 raft_consensus.cc:993] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:53:06.645484  6467 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:06.857545  6700 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:06.567229  6742 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:06.646209  6676 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:06.599175  6616 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.644418  6613 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:06.618095  6740 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 14:53:06.617486  6202 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:06.854786  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.020s	sys 0.087s
W20260709 14:53:06.995460  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.028s	sys 0.094s
I20260709 14:53:06.841084  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.996501  3540 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:06.806290  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.034s	sys 0.059s
W20260709 14:53:06.997552  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.042s	sys 0.078s
I20260709 14:53:06.913290  2683 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:06.998162  3541 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:06.759392  2608 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:06.629348  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:06.755589  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:07.000445  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 371200us
W20260709 14:53:07.000769  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 245360us
I20260709 14:53:06.692548  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:06.858325  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.819290  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.691203  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:07.005939  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 346452us
W20260709 14:53:06.636819  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153459us
W20260709 14:53:06.686661  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1072) took 1308 ms (client timeout 1501 ms). Trace:
W20260709 14:53:07.007520  3537 rpcz_store.cc:269] 0709 14:53:05.377796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.379647 (+  1851us) service_pool.cc:224] Handling call
0709 14:53:06.686297 (+1306650us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002012d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:06.686556 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218548992}
I20260709 14:53:07.009593  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:06.681186  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:07.011669  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 330528us
I20260709 14:53:07.012743  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:06.930141  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:06.936298  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 241310us
I20260709 14:53:07.014964  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.941035  6751 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:06.941166  6776 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:06.942783  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:06.944063  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.943876  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:07.022742  3547 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:07.024456  2755 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
W20260709 14:53:06.843277  6764 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.572s	user 0.001s	sys 0.011s
I20260709 14:53:06.670034  6767 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.560322  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:06.942251  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:06.914489  6712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.054s	sys 0.136s
W20260709 14:53:06.935917  6797 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.002s
I20260709 14:53:06.833467  6748 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.941686  6693 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:06.681890  6495 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:06.617690  6645 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:06.955746  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:06.955379  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:06.677641  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.022567  6751 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:06.880188  6815 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:06.703467  6698 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:06.965171  2685 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:06.933971  6758 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:06.937534  6631 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:06.967927  2679 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:06.806407  6810 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:06.983529  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:06.983462  2753 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:06.951401  6807 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:06.950402  6753 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:06.706988  6244 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:06.992061  6742 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:07.003049  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:06.927130  2603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.010s	sys 0.029s
I20260709 14:53:07.146498  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:07.147111  2603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.010s	sys 0.030s
I20260709 14:53:07.147351  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:06.972242  6798 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:07.002578  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:06.838116  6747 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.019548  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:07.020515  3542 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:07.020702  2751 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:07.025854  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:07.022266  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:07.145625  6807 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:07.154796  6613 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:07.155668  6613 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:07.156045  6613 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:07.156411  6613 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:07.052752  6748 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:07.160822  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141385us
I20260709 14:53:07.161108  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.161691  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:07.029947  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:06.972795  6622 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 14:53:07.041015  2754 raft_consensus.cc:2368] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11.
I20260709 14:53:07.033473  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:07.142635  6854 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 14:53:07.022890  3537 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:07.036924  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.167145  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.501s (SENT)
I20260709 14:53:07.167634  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:06.999795  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:07.170051  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:07.171154  6824 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 14:53:07.171881  6824 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:07.049654  2683 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:07.172698  6824 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:07.172937  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:07.017710  6414 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:07.173403  6824 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:07.173672  6414 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:07.174082  6824 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.174315  6414 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:07.174566  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.175329  2754 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:07.176211  6715 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:07.176362  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.177090  2682 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:07.053681  2678 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:07.178786  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 728) took 1062 ms. Trace:
I20260709 14:53:06.928208  6701 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:07.102530  6847 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 14:53:07.028894  6374 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:07.050714  6817 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 14:53:07.189334  6701 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:07.150889  6798 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:07.117718  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:07.209746  2612 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:07.054340  2758 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:07.052829  3412 leader_election.cc:400] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:07.043383  6740 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:07.120523  6359 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.001s
I20260709 14:53:06.988278  6826 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:07.123811  6783 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.000s
I20260709 14:53:07.026394  6777 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:53:07.124984  6749 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.798s	user 0.004s	sys 0.010s
I20260709 14:53:07.130141  2606 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:07.130889  3543 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:07.022178  6756 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:06.989257  6676 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.243252  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:07.145777  6810 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.033732  6764 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.043486  6495 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:07.149308  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146357us
W20260709 14:53:07.152609  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 130414us
I20260709 14:53:07.027515  2757 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
I20260709 14:53:07.163426  3536 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 4.
I20260709 14:53:07.046795  6767 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:07.255224  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 339942us
I20260709 14:53:07.251037  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:07.257660  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:07.165236  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.171757  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:07.179234  2682 rpcz_store.cc:276] 0709 14:53:06.116773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.842983 (+726210us) service_pool.cc:224] Handling call
0709 14:53:06.859924 (+ 16941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:07.177543 (+317619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:07.178485 (+   942us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:07.178714 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728805760}
I20260709 14:53:07.177070  2754 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:07.261341  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:07.042778  2604 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:07.173709  3537 raft_consensus.cc:2368] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:07.262076  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.263327  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:07.264223  3537 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:07.265790  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.162102  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:07.185003  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.146595  6758 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:07.017987  6762 raft_consensus.cc:1081] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Signalling peer 56620e612f0549e6abf320aa80398e74 to start an election
I20260709 14:53:07.269284  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.180532  6793 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:07.181442  6742 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:07.239830  2609 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 14:53:07.144353  6631 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.152256  6797 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 14:53:07.025612  6675 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:07.133701  3540 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:07.288183  3540 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:07.134102  6776 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.266273  3537 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:53:07.212314  3412 leader_election.cc:403] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:07.213241  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.293856  2758 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:07.295334  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.272265  2753 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:07.240628  6715 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:53:07.274346  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 246743us
I20260709 14:53:07.299479  3537 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:07.300283  2677 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:07.300624  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:07.276978  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 114991us
I20260709 14:53:07.189906  6824 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.305250  3537 raft_consensus.cc:1217] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Deduplicated request from leader. Original: 7.4->[10.5-10.5]   Dedup: 10.5->[]
I20260709 14:53:07.305830  3537 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 14:53:07.312443  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1080) took 1527 ms. Trace:
I20260709 14:53:07.313110  3547 rpcz_store.cc:276] 0709 14:53:05.784997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.208967 (+423970us) service_pool.cc:224] Handling call
0709 14:53:06.209308 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:07.024633 (+815325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:07.311969 (+287336us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b54004b23d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:07.312317 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870096256}
I20260709 14:53:07.281562  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 725) took 1213 ms. Trace:
I20260709 14:53:07.315974  2680 rpcz_store.cc:276] 0709 14:53:06.067984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.261972 (+193988us) service_pool.cc:224] Handling call
0709 14:53:06.262350 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:06.310128 (+ 47778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:06.310169 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:06.310815 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 14:53:06.708472 (+397657us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:06.708515 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:06.708599 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:07.281292 (+572693us) raft_consensus.cc:1685] finished
0709 14:53:07.281315 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:07.281484 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47087,"raft.queue_time_us":48555,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":397071,"threads_started":1,"wal-append.queue_time_us":687674}
I20260709 14:53:07.281467  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:07.284515  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.320289  2757 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:07.321754  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:07.301625  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 248991us
I20260709 14:53:07.301927  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:07.256975  6777 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:07.327414  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:07.327627  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:07.322631  2757 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:07.328755  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.328879  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:07.329675  2758 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:07.326292  3537 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:53:07.330781  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.334421  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:07.337002  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54887us
I20260709 14:53:07.249241  6740 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:07.271212  6687 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.346728  3540 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:07.352859  2757 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:07.275887  6793 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:07.273263  6820 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:07.354759  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:07.301890  6359 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 14:53:07.368032  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 711) took 2640 ms. Trace:
I20260709 14:53:07.290715  6733 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:07.408319  3411 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.515s (SENT)
I20260709 14:53:07.421083  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:07.276949  6860 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:53:07.242842  6783 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.247340  6749 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.305606  6467 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:07.391993   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:53:07.326325  6374 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:07.261969  6790 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.368625  2605 rpcz_store.cc:276] 0709 14:53:04.716351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.132870 (+416519us) service_pool.cc:224] Handling call
0709 14:53:05.133483 (+   613us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:05.348011 (+214528us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 14:53:05.374126 (+ 26115us) log.cc:844] Serialized 14 byte log entry
0709 14:53:06.048339 (+674213us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b180039d4e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:06.996044 (+947705us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:06.996967 (+   923us) log.cc:844] Serialized 52 byte log entry
0709 14:53:06.997139 (+   172us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:53:06.997172 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:07.354743 (+357571us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b54002ff3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:07.354814 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:07.356883 (+  2069us) raft_consensus.cc:1685] finished
0709 14:53:07.356911 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:07.357121 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3823,"prepare.queue_time_us":31812,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":48602,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":22956,"spinlock_wait_cycles":81880832,"thread_start_us":1127242,"threads_started":2,"wal-append.queue_time_us":1663272}
I20260709 14:53:07.344990  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:07.415378  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.435680  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 505646us
I20260709 14:53:07.242928  6645 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:07.471184  6853 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 14:53:07.472772  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1226) took 1908 ms. Trace:
I20260709 14:53:07.327500  6777 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:07.473229  3546 rpcz_store.cc:276] 0709 14:53:05.563916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.206962 (+643046us) service_pool.cc:224] Handling call
0709 14:53:06.207413 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:06.229348 (+ 21935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:06.229377 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:06.236322 (+  6945us) log.cc:844] Serialized 26 byte log entry
0709 14:53:06.998155 (+761833us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:06.998199 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:06.998295 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:07.472511 (+474216us) raft_consensus.cc:1685] finished
0709 14:53:07.472537 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:07.472698 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13767,"raft.queue_time_us":10366,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6397,"thread_start_us":717968,"threads_started":1,"wal-append.queue_time_us":763821}
I20260709 14:53:07.354236  6817 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.355587  6814 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:07.366667  6359 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:07.383630  6776 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:07.439049  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.024s	sys 0.033s
I20260709 14:53:07.340291  6742 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 14:53:07.346185  6861 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 14:53:07.497192  6751 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.498250  6861 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 14:53:07.364609  6798 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.499347  3542 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:07.499750  6798 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:07.435880  6758 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.416256  2754 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:07.364945  6700 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 14:53:07.504508  6700 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:07.417384  6734 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:07.450526  2751 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:07.450649  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 420088us
I20260709 14:53:07.428982  6762 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.462513  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1370) took 1505 ms. Trace:
I20260709 14:53:07.539121  2610 rpcz_store.cc:276] 0709 14:53:05.957294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.957566 (+   272us) service_pool.cc:224] Handling call
0709 14:53:07.462153 (+1504587us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:07.462408 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287458688}
I20260709 14:53:07.305025  6753 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.447134  6374 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:07.362321  6687 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.468816   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:53:07.360415  6495 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.469584  6868 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:07.540319  2612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
 from {username='slave'} at 127.0.0.1:55870
I20260709 14:53:07.564085  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:07.478018  2608 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:07.466895  6698 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:07.486887  6715 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:07.488469  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.024s	sys 0.033s
I20260709 14:53:07.532977  2756 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:07.541761  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 212975us
I20260709 14:53:07.507761  6359 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 5 FOLLOWER]: Advancing to term 6
W20260709 14:53:07.566262  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 638723us
W20260709 14:53:07.594561  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 592027us
I20260709 14:53:07.523821  6700 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:07.597235  3537 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:07.605860  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:07.524304  6467 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 14:53:07.559091  6631 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:07.615329  6613 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.624943  6764 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:07.615073  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1385) took 1195 ms. Trace:
I20260709 14:53:07.703593  2604 rpcz_store.cc:276] 0709 14:53:06.419615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.420549 (+   934us) service_pool.cc:224] Handling call
0709 14:53:07.614845 (+1194296us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:07.614995 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887971200}
W20260709 14:53:07.669716  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:07.474848  6777 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.605427  6815 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.475585  6740 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.678634  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:07.679247  6726 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:07.680691  6801 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:07.683172  6749 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:07.692867  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 703) took 1392 ms (client timeout 1612 ms). Trace:
W20260709 14:53:07.746487  2756 rpcz_store.cc:269] 0709 14:53:06.299941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.456327 (+156386us) service_pool.cc:224] Handling call
0709 14:53:07.692621 (+1236294us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:07.692771 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267290496}
I20260709 14:53:07.605146  6797 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.127s
W20260709 14:53:07.748618  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70134.7us
I20260709 14:53:07.740950  6801 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:07.755607  6733 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:07.580134  6868 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:07.749147  6810 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:07.756214  6868 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:07.539218  6734 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.706991  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:07.711701  6767 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.717412  6751 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:07.595846  6715 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.719926  6613 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:07.720431  6701 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 14:53:07.486497  6820 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.611889  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:07.785130  3540 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:07.611433  6495 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:07.547158  6762 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:07.725023  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:07.606786  6676 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.566977  6807 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.596135  6414 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:07.699054  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:07.712378  6867 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.720803  6631 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:07.578998  6698 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:07.819270  6698 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:07.821300  6878 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:07.507287  6817 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.782868  6790 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.823710  6734 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:07.824656  6817 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.782929  6748 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.759873  2604 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:07.826418  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:07.819327  6631 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:07.710340  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.740476  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 332230us
W20260709 14:53:07.828346  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.828498  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.828882  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:07.829566  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:07.830340  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:07.725448  6866 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:07.671350  6862 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 14:53:07.772552  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:07.779882  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:07.834105  3411 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.612s (SENT)
I20260709 14:53:07.834373  2756 raft_consensus.cc:2376] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:07.740600  6777 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:07.703950  6864 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 14:53:07.748028  6764 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:07.733008  6766 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 14:53:07.810576  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 467914us
I20260709 14:53:07.737898  6815 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.745198  6675 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:07.800900  6701 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:07.830399  2726 leader_election.cc:400] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:07.888692  2726 leader_election.cc:403] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:07.769938  6733 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.740446  6799 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:07.780819  6767 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:53:07.834699  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62257.4us
I20260709 14:53:07.834698  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:07.747205  6726 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.844112  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:07.840261  6783 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.848929  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:07.852754  2755 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:07.863693  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 381304us
W20260709 14:53:07.927086  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78266.4us
I20260709 14:53:07.929126  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:07.879024  3540 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:07.908643  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:07.847813  6762 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:07.915627  2682 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 14:53:07.917071  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:07.946686  2679 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
W20260709 14:53:07.913903  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 79843us
I20260709 14:53:07.930449  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:07.931939  2756 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 5.
W20260709 14:53:07.948333  3411 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.719s (SENT)
W20260709 14:53:07.915153  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84901.6us
I20260709 14:53:07.948805  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:07.933759  3541 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:07.878667  6374 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:07.944871  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:07.952793  2681 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:07.984902  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:07.989965  3411 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.781s (SENT)
I20260709 14:53:07.853166  6790 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
W20260709 14:53:07.992496  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 62154.2us
I20260709 14:53:08.012899  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:07.944108  6786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.242s	user 0.019s	sys 0.011s
I20260709 14:53:07.923269  6874 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:07.853036  6764 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.926560  6613 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:07.836141  6867 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:07.999653  2726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1356) took 1834 ms. Trace:
I20260709 14:53:07.957715  6783 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:07.999734  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1348) took 1837 ms (client timeout 1504 ms). Trace:
I20260709 14:53:08.009243  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.008961  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:08.044669  2752 rpcz_store.cc:269] 0709 14:53:06.162396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.256252 (+ 93856us) service_pool.cc:224] Handling call
0709 14:53:07.999532 (+1743280us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400570154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:07.999657 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340076800}
W20260709 14:53:08.045547  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:07.935415  6883 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:08.047286  6883 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:08.047736  6883 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:08.049409  6801 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:07.957397  6876 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:08.022614  6747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.023s	sys 0.033s
W20260709 14:53:08.022972  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 268774us
I20260709 14:53:08.026919  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:07.927980  6869 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
W20260709 14:53:08.027921  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 307139us
I20260709 14:53:08.017379  6645 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:07.996469  6797 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:08.090267  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:07.994387  6749 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:08.102327  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:07.925542  6799 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.051307  6876 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:07.923461  6676 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 14:53:08.055472  6817 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 14:53:08.021937  6786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.020s	sys 0.012s
I20260709 14:53:08.014075  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:08.113493  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 123593us
W20260709 14:53:08.114233  3411 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.809s (SENT)
I20260709 14:53:07.998310  6879 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:08.037983  2726 rpcz_store.cc:276] 0709 14:53:06.165230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.999570 (+1834340us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 14:53:08.010576  6860 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.016s
I20260709 14:53:08.086472  2681 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 14:53:08.033169  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.091590  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:08.123950  2725 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items
W20260709 14:53:08.091538  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78740.2us
I20260709 14:53:08.132292  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:08.073114  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:08.095650  2685 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:08.025393  6766 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:08.046499  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.104960  2679 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:08.097294  6875 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:08.111050  3537 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:08.117236  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:08.116017  3411 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.033s (SENT)
I20260709 14:53:08.247869  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:08.248507  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 175553us
I20260709 14:53:08.087801  6801 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:08.249732  6881 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:53:08.143606  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.147324  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 303269us
I20260709 14:53:08.098940  6877 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:08.094769  6797 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:08.161108  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:08.088614  6747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.027s	sys 0.033s
I20260709 14:53:08.089641  6890 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:08.209118  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.200309  2685 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:53:08.200819  6766 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.122881  6613 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:08.116194  6749 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:08.119467  6817 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:08.132367  6860 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.006s	sys 0.016s
I20260709 14:53:08.125608  6645 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:08.278393  6860 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:08.279528  6645 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.280498  6815 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.128065  6767 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:08.283288  6749 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.150508  6879 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.284750  6767 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:08.178844  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.287765  6879 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.248689  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.287576  6817 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.288700  2684 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 14:53:08.289120  6817 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:08.249002  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 133006us
I20260709 14:53:08.254196  6875 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:08.291343  3411 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.085s (SENT)
I20260709 14:53:08.293517  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:08.293802  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.251444  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.294539  2605 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:08.294598  2679 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:08.295759  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.295900  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.296641  2605 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:08.296892  6799 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.068832  6872 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:08.296690  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:08.297698  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.297850  6872 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:08.298559  2605 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 14:53:08.299019  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.159861  6715 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.256785  2724 leader_election.cc:400] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:08.305361  2724 leader_election.cc:403] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 14:53:08.306116  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54109.6us
W20260709 14:53:08.307545  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.261211  2681 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:08.207495  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:08.261272  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:08.266193  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105160us
I20260709 14:53:08.106521  6374 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.202382  6892 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 14:53:08.252133  6870 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
W20260709 14:53:08.277499  6871 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
W20260709 14:53:08.284595  6901 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 14:53:08.024437  6675 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.325098  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.326166  2679 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:08.327560  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.293789  6875 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.255234  6790 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:08.332616  2605 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:08.333760  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.299171  6867 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.201548  6786 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
W20260709 14:53:08.308625  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 101278us
I20260709 14:53:08.252561  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1236) took 2165 ms. Trace:
I20260709 14:53:08.307003  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:08.336746  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:08.316735  3411 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.773s (SENT)
W20260709 14:53:08.278568  6885 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 14:53:08.339280  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:08.321133  6749 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.314510  6631 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.317700  6751 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:08.327907  2684 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:08.313721  6877 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:08.274811  6807 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:08.355424  3411 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.845s (SENT)
W20260709 14:53:08.362376  3411 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.037s (SENT)
I20260709 14:53:08.314292  6897 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.371799  6897 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.275600  6766 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.322382  6715 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:08.379935  6883 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.330021  6797 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.360971  6877 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:08.382628  6877 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:08.359877  6756 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.113s	user 0.008s	sys 0.008s
I20260709 14:53:08.384392  6756 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:08.316967  6892 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:08.364112  3411 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.496s (SENT)
I20260709 14:53:08.318478  6890 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.346108  2758 rpcz_store.cc:276] 0709 14:53:06.087433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.363020 (+1275587us) service_pool.cc:224] Handling call
0709 14:53:07.363501 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:07.364143 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:07.795951 (+431808us) log.cc:844] Serialized 14 byte log entry
0709 14:53:08.250535 (+454584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:08.250696 (+   161us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:08.250779 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.250951 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:08.252398 (+  1447us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":715640,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":29509,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":150,"thread_start_us":858502,"threads_started":2}
I20260709 14:53:08.365027  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.131796  2726 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 746)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 742)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1239)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1240)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 696)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 697)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 712)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1247)
I20260709 14:53:08.357249  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:08.359347  6751 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:08.365833  6613 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:53:08.367938  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.310250  6887 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:08.388168  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1237) took 2300 ms. Trace:
I20260709 14:53:08.319706  6815 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:08.391966  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.400666  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:08.400743  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.412798  2684 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 8: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:08.404592  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78403.4us
I20260709 14:53:08.405777  2679 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:08.406234  2611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.011s	sys 0.020s
I20260709 14:53:08.387068  6799 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:08.408852  2757 rpcz_store.cc:276] 0709 14:53:06.087702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.379519 (+1291817us) service_pool.cc:224] Handling call
0709 14:53:07.417389 (+ 37870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:07.417988 (+   599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:07.418035 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:07.452356 (+ 34321us) log.cc:844] Serialized 26 byte log entry
0709 14:53:07.834615 (+382259us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:07.834695 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:07.834854 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:08.387830 (+552976us) raft_consensus.cc:1685] finished
0709 14:53:08.387864 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:08.388088 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4702,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":7570,"thread_start_us":360029,"threads_started":1,"wal-append.queue_time_us":496580}
I20260709 14:53:08.413065  2758 raft_consensus.cc:2376] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
W20260709 14:53:08.414003  2611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.012s	sys 0.020s
I20260709 14:53:08.416468  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.419178  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:08.413293  6887 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:08.426242  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.419437  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:08.394111  6715 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:08.450510  6887 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:08.416128  6613 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:08.428821  6742 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:08.396564  6892 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:08.430337  2757 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:08.279517  6860 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.416492  6797 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:08.429692  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1381) took 2293 ms. Trace:
I20260709 14:53:08.460337  2611 rpcz_store.cc:276] 0709 14:53:06.134801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.610540 (+475739us) service_pool.cc:224] Handling call
0709 14:53:06.617842 (+  7302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:06.618089 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:07.135174 (+517085us) log.cc:844] Serialized 14 byte log entry
0709 14:53:08.053383 (+918209us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b1800397ea0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559557 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:08.428028 (+374645us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:08.428103 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:08.428169 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.428256 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:08.428450 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1031,"prepare.queue_time_us":407033,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":342719,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":29671,"spinlock_wait_cycles":3265152,"thread_start_us":1782947,"threads_started":3,"wal-append.queue_time_us":707876}
I20260709 14:53:08.463006  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:08.457930  6616 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.465857  6883 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:08.435137  2604 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:08.428833  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.403394  6751 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:08.365518  6867 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 14:53:08.417949  6753 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:53:08.481530  6873 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.361s	user 0.000s	sys 0.002s
I20260709 14:53:08.481763  6896 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:08.428090  6726 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.508296  6766 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:08.508630  6495 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.005s
I20260709 14:53:08.481554  6903 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:08.419265  6764 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.517596  6903 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:08.517275  6715 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:08.519204  6698 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.004s
I20260709 14:53:08.519621  6698 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.518754  6903 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.522073  2612 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:53:08.522296  6698 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.522593  2612 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:08.345199  6872 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:08.525606  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 951) took 1646 ms (client timeout 1653 ms). Trace:
I20260709 14:53:08.407306  6890 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.131637  2725 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 746)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 742)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1239)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1240)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 696)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 697)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 712)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1247)
W20260709 14:53:08.527697  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1349) took 2364 ms (client timeout 2265 ms). Trace:
W20260709 14:53:08.528198  2755 rpcz_store.cc:269] 0709 14:53:06.162875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.033190 (+870315us) service_pool.cc:224] Handling call
0709 14:53:08.527437 (+1494247us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.527609 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563514752}
W20260709 14:53:08.526214  3545 rpcz_store.cc:269] 0709 14:53:06.879984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.957570 (+ 77586us) service_pool.cc:224] Handling call
0709 14:53:08.525332 (+1567762us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.525527 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907117952}
I20260709 14:53:08.486075  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:08.483558  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.491255  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.495569  6807 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.496409  6700 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:08.497994  6359 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.896s	user 0.014s	sys 0.000s
I20260709 14:53:08.504833  6897 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:08.505193  3411 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.883s (SENT)
I20260709 14:53:08.548210  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:08.481652  6891 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.150s
I20260709 14:53:08.511324  2752 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:08.514676  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:08.459079  6815 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.559330  6815 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.481690  6888 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.189s
W20260709 14:53:08.509558  6798 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.009s	user 0.013s	sys 0.001s
I20260709 14:53:08.391012  6879 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 14:53:08.502234  6877 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.535256  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.575357  2757 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:08.527938  6753 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:08.545769  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.546314  3547 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
W20260709 14:53:08.549296  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44167.7us
I20260709 14:53:08.544556  2725 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 746)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1239)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1240)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 696)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 697)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 712)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1247)
I20260709 14:53:08.491187  6799 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 14:53:08.569521  6861 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.071s	user 0.009s	sys 0.005s
I20260709 14:53:08.562958  6467 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:08.469136  6742 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:08.538268  6793 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.164s	user 0.014s	sys 0.006s
I20260709 14:53:08.573892  2753 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:53:08.580071  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.562105  6815 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:08.583351  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 732) took 1093 ms. Trace:
I20260709 14:53:08.598943  2755 rpcz_store.cc:276] 0709 14:53:07.489782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.545481 (+1055699us) service_pool.cc:224] Handling call
0709 14:53:08.583267 (+ 37786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.557380  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:08.600481  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:08.572289  6868 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.001s
I20260709 14:53:08.556025  6912 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.573670  6786 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:08.576970  6866 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.724s	user 0.015s	sys 0.001s
I20260709 14:53:08.604878  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.550900  6807 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.582929  6700 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:08.606328  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 249186us
W20260709 14:53:08.603235  6914 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 14:53:08.557197  6359 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.527490  6613 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.585075  6799 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:08.587795  6753 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.600975  6861 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 14:53:08.602938  6879 consensus_queue.cc:1243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:08.651952  6807 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.662216  6877 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.664036  6740 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:08.664570  6740 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:08.665340  6631 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:08.665761  6631 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:08.619854  6815 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:08.621372  6866 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:08.602474  6793 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.618343  6868 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.675997  6799 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.683758  6799 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:08.620882  6912 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:08.685494  6874 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.003s
W20260709 14:53:08.690584  6899 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 14:53:08.552788  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:08.544436  2605 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:08.534329  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.616566  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59262us
I20260709 14:53:08.539680  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.567047  2612 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.618328  2755 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:08.769594  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:08.758755  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.746964  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.784646  6631 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.804483  6687 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.694578  6874 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:08.738363  2606 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:08.750735  2610 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 11. Ops: []
I20260709 14:53:08.745407  2680 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:08.749243  2756 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 14:53:08.742730  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.804996  6903 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.745208  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.811008  6766 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:08.744119  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.813333  6766 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.818765  3540 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:08.782651  6359 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.819559  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1383) took 2643 ms. Trace:
I20260709 14:53:08.820307  6877 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.820168  2609 rpcz_store.cc:276] 0709 14:53:06.176038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.712485 (+536447us) service_pool.cc:224] Handling call
0709 14:53:06.713022 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:07.270911 (+557889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.818794 (+1547883us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:08.819270 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210315520}
I20260709 14:53:08.821501  6877 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:08.824400  6930 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 14:53:08.824759  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 737) took 2535 ms. Trace:
I20260709 14:53:08.825299  2677 rpcz_store.cc:276] 0709 14:53:06.288755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.647751 (+358996us) service_pool.cc:224] Handling call
0709 14:53:08.824223 (+2176472us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.824527 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432834304}
W20260709 14:53:08.799199  6810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.007s	sys 0.007s
I20260709 14:53:08.827772  2679 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:08.829118  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1337) took 1068 ms. Trace:
I20260709 14:53:08.830338  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.829793  2604 rpcz_store.cc:276] 0709 14:53:07.760844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.826115 (+ 65271us) service_pool.cc:224] Handling call
0709 14:53:08.828480 (+1002365us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54003c8054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.828811 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305686272}
I20260709 14:53:08.835552  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.835963  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:08.790733  6817 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.501s	user 0.014s	sys 0.001s
W20260709 14:53:08.839509  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.840380  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.840131  2684 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:08.840966  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.841528  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:08.844246  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 944) took 2035 ms (client timeout 1963 ms). Trace:
W20260709 14:53:08.844997  3541 rpcz_store.cc:269] 0709 14:53:06.808978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.171441 (+362463us) service_pool.cc:224] Handling call
0709 14:53:08.843585 (+1672144us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54002baf54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.843939 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549320960}
W20260709 14:53:08.845652  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1345) took 2751 ms (client timeout 1819 ms). Trace:
I20260709 14:53:08.847234  2758 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:08.848229  6815 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.848973  6817 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:08.757995  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 746) took 1263 ms. Trace:
I20260709 14:53:08.850229  2755 rpcz_store.cc:276] 0709 14:53:07.493967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.604487 (+1110520us) service_pool.cc:224] Handling call
0709 14:53:08.757675 (+153188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.850159  6817 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.852388  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:08.854125  6202 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:08.760798  2577 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items
I20260709 14:53:08.849072  6815 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:08.832073  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:08.827934  6810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.008s	sys 0.008s
I20260709 14:53:08.832922  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.756695  3536 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 5.
I20260709 14:53:08.863189  6881 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.863557  6753 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:08.864022  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103553us
W20260709 14:53:08.864580  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.864459  6881 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.868103  2757 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:08.816171  6923 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.822041  6740 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.869760  6740 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:08.838374  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:08.846290  2760 rpcz_store.cc:269] 0709 14:53:06.094513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.118555 (+ 24042us) service_pool.cc:224] Handling call
0709 14:53:08.845067 (+2726512us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.845373 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4602141568}
I20260709 14:53:08.809664  6903 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.869585  6923 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.873885  6903 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.874105  6923 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.874359  6766 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.805202  6912 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.875280  6631 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:08.875778  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:08.876899  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 739) took 1385 ms. Trace:
I20260709 14:53:08.877310  2760 rpcz_store.cc:276] 0709 14:53:07.491826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.875231 (+1383405us) service_pool.cc:224] Handling call
0709 14:53:08.876815 (+  1584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.878798  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:08.883821  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 114704us
I20260709 14:53:08.848217  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 749) took 1353 ms. Trace:
I20260709 14:53:08.853485  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 733) took 1363 ms. Trace:
W20260709 14:53:08.884439  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.744499  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:08.822705  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:08.891448  6807 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:08.893128  6875 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.894342  6875 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:08.895383  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1341) took 1097 ms. Trace:
I20260709 14:53:08.895898  2609 rpcz_store.cc:276] 0709 14:53:07.798149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.822147 (+1023998us) service_pool.cc:224] Handling call
0709 14:53:08.895284 (+ 73137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.897727  6875 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.898420  2609 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 14:53:08.899574  6753 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.899092  2677 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:08.903880  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:08.905282  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.884847  2758 rpcz_store.cc:276] 0709 14:53:07.505407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.835112 (+1329705us) service_pool.cc:224] Handling call
0709 14:53:08.848101 (+ 12989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.907622  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:08.909391  3537 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:08.885051  2755 rpcz_store.cc:276] 0709 14:53:07.490140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.851950 (+1361810us) service_pool.cc:224] Handling call
0709 14:53:08.853362 (+  1412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.912220  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:08.906514  3541 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:08.912957  6701 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.922303  3538 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:08.914579  6815 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:08.919718  6753 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.926005  2683 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:08.929097  6753 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:08.929961  6824 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:08.931645  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1327) took 1214 ms. Trace:
I20260709 14:53:08.933439  2609 rpcz_store.cc:276] 0709 14:53:07.708346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.897079 (+1188733us) service_pool.cc:224] Handling call
0709 14:53:08.897747 (+   668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:08.920349 (+ 22602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.922390 (+  2041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:08.937536  3542 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:08.938248  6865 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:08.939045  6865 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:08.939344  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.936544  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:08.949250  6892 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:08.956002  6824 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:08.957147  2683 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:08.958814  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1374) took 2999 ms. Trace:
I20260709 14:53:08.959877  6877 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:08.960775  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57085.1us
I20260709 14:53:08.974068  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:08.961321  6867 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:08.975831  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:08.976603  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.977932  3541 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:08.944536  6865 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.980767  6865 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:08.980923  6817 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:08.981370  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:08.982251  3540 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:08.983793  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1233) took 1073 ms. Trace:
W20260709 14:53:08.984227  6777 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.141s	user 0.015s	sys 0.011s
W20260709 14:53:08.985430  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1219) took 3308 ms (client timeout 2963 ms). Trace:
I20260709 14:53:08.986086  2685 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
W20260709 14:53:08.988742  6935 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:53:08.992169  6767 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.993348  2608 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:08.993026  6767 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:08.994186  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1330) took 1269 ms. Trace:
I20260709 14:53:08.997189  6767 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:08.996795  6875 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:08.999819  6676 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:08.969051  2603 rpcz_store.cc:276] 0709 14:53:05.959019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.236774 (+277755us) service_pool.cc:224] Handling call
0709 14:53:06.266906 (+ 30132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:06.318515 (+ 51609us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:07.147784 (+829269us) log.cc:844] Serialized 14 byte log entry
0709 14:53:08.497582 (+1349798us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b180036dc60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287
0709 14:53:08.497714 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:08.497781 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:08.497840 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.958114 (+460274us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:08.958195 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:08.958547 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28627,"prepare.queue_time_us":665043,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":2322,"raft.queue_time_us":359128,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":22104,"spinlock_wait_cycles":126033024,"thread_start_us":1535130,"threads_started":3,"wal-append.queue_time_us":732107}
I20260709 14:53:09.002996  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:09.000447  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.004743  3536 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:08.964214  6898 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.703s	user 0.001s	sys 0.003s
I20260709 14:53:09.006585  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:08.966277  6925 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.378s	user 0.002s	sys 0.000s
I20260709 14:53:09.008769  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.010241  3547 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 14:53:08.985961  2678 rpcz_store.cc:269] 0709 14:53:05.676767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:05.676937 (+   170us) service_pool.cc:224] Handling call
0709 14:53:08.984881 (+3307944us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:08.985178 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039496576}
I20260709 14:53:08.962378  2604 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:08.962013  6903 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:08.984313  2680 rpcz_store.cc:276] 0709 14:53:07.910478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.914203 (+  3725us) service_pool.cc:224] Handling call
0709 14:53:07.914654 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:08.820097 (+905443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.983427 (+163330us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:08.983690 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908808576}
I20260709 14:53:08.984659  6753 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 14:53:09.014391  6824 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:53:09.015326  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 271406us
I20260709 14:53:09.015497  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:08.994575  2608 rpcz_store.cc:276] 0709 14:53:07.724569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.992491 (+1267922us) service_pool.cc:224] Handling call
0709 14:53:08.992969 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:08.993850 (+   881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:08.994096 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.016340  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.017149  2609 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 4. Current term is 6. Ops: []
W20260709 14:53:08.962494  6893 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.752s	user 0.004s	sys 0.000s
I20260709 14:53:09.001520  6676 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:09.017912  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
W20260709 14:53:08.967612  6913 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.399s	user 0.002s	sys 0.000s
W20260709 14:53:09.044620  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.020921  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.042632  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:09.043228  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69335.2us
I20260709 14:53:09.049358  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1328) took 1320 ms. Trace:
I20260709 14:53:09.050148  2609 rpcz_store.cc:276] 0709 14:53:07.708599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.946374 (+1237775us) service_pool.cc:224] Handling call
0709 14:53:09.016724 (+ 70350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:09.028263 (+ 11539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.028529 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.052965  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:09.055326  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:09.057104  6887 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.574s	user 0.014s	sys 0.009s
I20260709 14:53:09.057459  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.057816  6887 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:09.058964  6892 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.060156  6777 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:09.060709  6777 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:09.061082  6777 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.062107  6676 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.067075  6877 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:09.067296  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.068020  6877 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:09.069305  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.069437  3545 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:09.069574  6881 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.070163  6867 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:09.057154  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.071267  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.053339  6747 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:09.053071  2603 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
I20260709 14:53:09.083351  3540 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:09.071321  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.074529  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:09.071234  2682 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:09.086637  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.020188  6903 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:09.092199  2758 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:09.092705  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.096823  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.053013  6613 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:09.098392  2677 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:09.080672  6747 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:09.096441  3545 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:09.102051  3541 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:09.112790  6876 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.116034  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 710) took 2562 ms. Trace:
I20260709 14:53:09.116936  2753 rpcz_store.cc:276] 0709 14:53:06.553706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.344590 (+790884us) service_pool.cc:224] Handling call
0709 14:53:09.115303 (+1770713us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.115710 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598177536}
W20260709 14:53:09.119391  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.120620  6887 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.122697  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:09.123467  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49117.6us
I20260709 14:53:09.127842  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:09.129078  2603 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.129844  2759 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:09.130121  6824 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:09.130808  6824 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:09.131341  6824 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.137140  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.133984  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.106398  6747 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:09.100215  6613 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:09.139536  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:09.133360  6700 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.015s
I20260709 14:53:09.069573  6877 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.150115  2755 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:09.154372  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.157279  2677 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:09.159381  2682 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:09.157653  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 734) took 1667 ms (client timeout 2085 ms). Trace:
W20260709 14:53:09.160516  2755 rpcz_store.cc:269] 0709 14:53:07.490469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.947632 (+1457163us) service_pool.cc:224] Handling call
0709 14:53:09.157575 (+209943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.162209  6467 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.163160  6467 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:09.145188  6202 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.165508  3547 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:09.167380  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.172904  3545 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:53:09.177399  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:09.179100  6748 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:09.182991  6748 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:09.181771  6876 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:09.183830  6876 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:09.184605  6876 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.184813  6748 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:09.186033  6748 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:09.192241  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.194674  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:09.183851  6786 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.007s
I20260709 14:53:09.190755  6202 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:09.196810  6783 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.108s	user 0.010s	sys 0.015s
I20260709 14:53:08.738389   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:09.202651  6903 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:09.203701  2683 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:09.203941  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1326) took 1495 ms. Trace:
I20260709 14:53:09.206166  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:09.171253  6924 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 14:53:09.179102  6890 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:09.206307  2609 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:09.212152  2605 rpcz_store.cc:276] 0709 14:53:07.708066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.333471 (+625405us) service_pool.cc:224] Handling call
0709 14:53:09.203535 (+870064us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540042dbd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.203730 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482493696}
I20260709 14:53:09.207427  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.211678  2604 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:09.215394  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106984us
I20260709 14:53:09.220180  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1331) took 1485 ms. Trace:
I20260709 14:53:09.220683  2604 rpcz_store.cc:276] 0709 14:53:07.734397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.146019 (+1411622us) service_pool.cc:224] Handling call
0709 14:53:09.188061 (+ 42042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:09.219724 (+ 31663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.220098 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.223742  2753 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:09.225353  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 748) took 1727 ms (client timeout 2209 ms). Trace:
W20260709 14:53:09.225932  2755 rpcz_store.cc:269] 0709 14:53:07.494611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.173630 (+1679019us) service_pool.cc:224] Handling call
0709 14:53:09.221948 (+ 48318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.224042  2677 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:09.227381  3538 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 14:53:09.228193  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 720) took 2244 ms (client timeout 1773 ms). Trace:
I20260709 14:53:09.228535  6766 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.228997  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:09.229349  3537 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:09.229187  6766 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:09.230921  6815 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:09.231415  6815 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:09.232015  6815 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:09.235111  6751 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.735s	user 0.014s	sys 0.010s
I20260709 14:53:09.235769  6751 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:09.231314  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:09.238978  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:09.239487  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 737) took 1748 ms (client timeout 2213 ms). Trace:
W20260709 14:53:09.240005  2755 rpcz_store.cc:269] 0709 14:53:07.491306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.230939 (+1739633us) service_pool.cc:224] Handling call
0709 14:53:09.239285 (+  8346us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:09.242803  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.242926  3542 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:09.215440  6797 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.013s
W20260709 14:53:09.228693  2751 rpcz_store.cc:269] 0709 14:53:06.966599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.164859 (+198260us) service_pool.cc:224] Handling call
0709 14:53:09.211041 (+2046182us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400236ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.211343 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437777664}
I20260709 14:53:09.248144  3538 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:09.250782  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.253579  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.227353  6786 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:09.227766  6783 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:09.259959  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:09.263046  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.268363  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:09.269130  3538 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:53:09.268031  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:09.272456  6626 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:09.273018  6626 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:09.273811  2760 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 14:53:09.282164  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.078s (SENT) [suppressed 44 similar messages]
I20260709 14:53:09.213757  6613 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:09.287739  2758 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:09.289525  2682 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:53:09.288180  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.291863  2605 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:09.291764  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.292773  2652 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
I20260709 14:53:09.295545  2652 leader_election.cc:403] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 14:53:09.297472  2680 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 14:53:09.299975  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:09.295261  2755 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:53:09.298503  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 743) took 1797 ms (client timeout 2078 ms). Trace:
I20260709 14:53:09.301589  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1249) took 1380 ms. Trace:
I20260709 14:53:09.302171  2755 rpcz_store.cc:276] 0709 14:53:07.921380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.244124 (+1322744us) service_pool.cc:224] Handling call
0709 14:53:09.301517 (+ 57393us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:09.301888  2758 rpcz_store.cc:269] 0709 14:53:07.493015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.922164 (+1429149us) service_pool.cc:224] Handling call
0709 14:53:09.290362 (+368198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.287918  6903 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:09.309306  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 720) took 2350 ms. Trace:
I20260709 14:53:09.309926  2607 rpcz_store.cc:276] 0709 14:53:06.953537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.963652 (+ 10115us) service_pool.cc:224] Handling call
0709 14:53:06.964388 (+   736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:07.273915 (+309527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:07.273966 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:07.274016 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:07.274059 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.291445 (+2017386us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:09.303667 (+ 12222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:09.303971 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711042304}
I20260709 14:53:09.311857  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:09.313757  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.318347  2607 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:09.323689  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:09.325109  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.322780  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:09.282155  6626 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.334043  6786 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:09.334625  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.335871  6613 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.336711  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.337491  2677 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:09.339395  6202 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:09.315778  6793 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:09.334988  6626 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:09.343046  2680 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:09.304700  6797 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 14:53:09.347177  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 119140us
I20260709 14:53:09.349086  2682 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:09.350643  6751 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.350919  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:09.351933  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45432.7us
I20260709 14:53:09.291428  6700 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 14:53:09.358516  6766 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:09.361156  6799 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.006s
W20260709 14:53:09.363960  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 727) took 2108 ms (client timeout 1506 ms). Trace:
W20260709 14:53:09.364530  2754 rpcz_store.cc:269] 0709 14:53:07.255127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.260980 (+  5853us) service_pool.cc:224] Handling call
0709 14:53:09.363456 (+2102476us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.363699 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239400192}
I20260709 14:53:09.366281  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.360288  6675 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:09.367074  6675 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:09.367702  6675 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:09.368466  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.368592  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:09.371126  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:09.371954  2682 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 14:53:09.372885  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.373200  2607 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:09.362353  6742 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.374377  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.373979  6866 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
I20260709 14:53:09.383249  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.376402  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:09.384179  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90419.7us
W20260709 14:53:09.385859  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.376806  3538 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:53:09.385910  2603 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:09.369511  2608 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:09.375283  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.375384  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:09.378676  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:09.386615  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.389254  2755 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 14:53:09.391722  2604 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:09.394136  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:09.394965  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 755) took 1444 ms. Trace:
I20260709 14:53:09.398509  2755 rpcz_store.cc:276] 0709 14:53:07.950352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.303276 (+1352924us) service_pool.cc:224] Handling call
0709 14:53:09.394867 (+ 91591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.394068  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 756) took 1439 ms. Trace:
I20260709 14:53:09.401782  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:09.402402  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.403038  2608 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:09.403257  3543 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:09.403995  6867 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 14:53:09.404417  2608 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:09.405143  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1362) took 1775 ms (client timeout 2054 ms). Trace:
W20260709 14:53:09.405828  2753 rpcz_store.cc:269] 0709 14:53:07.630186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.118571 (+1488385us) service_pool.cc:224] Handling call
0709 14:53:09.404902 (+286331us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.405058 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172875520}
I20260709 14:53:09.407642  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.408391  3541 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:09.408807  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 753) took 1459 ms. Trace:
I20260709 14:53:09.409399  2753 rpcz_store.cc:276] 0709 14:53:07.949743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.407144 (+1457401us) service_pool.cc:224] Handling call
0709 14:53:09.408715 (+  1571us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.405274  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1335) took 1655 ms. Trace:
I20260709 14:53:09.411532  2608 rpcz_store.cc:276] 0709 14:53:07.749823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.403793 (+1653970us) service_pool.cc:224] Handling call
0709 14:53:09.404208 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:09.404966 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.405204 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.400576  2758 rpcz_store.cc:276] 0709 14:53:07.950705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.307406 (+1356701us) service_pool.cc:224] Handling call
0709 14:53:09.389856 (+ 82450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.404449  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:09.417374  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 135412us
I20260709 14:53:09.416582  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:09.419850  3543 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:09.413506  2754 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11. Re-sending same reply.
I20260709 14:53:09.421568  3538 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:53:09.422052  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 735) took 1931 ms (client timeout 2496 ms). Trace:
W20260709 14:53:09.422531  2754 rpcz_store.cc:269] 0709 14:53:07.490739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.365697 (+1874958us) service_pool.cc:224] Handling call
0709 14:53:09.421976 (+ 56279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.396749  6799 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.396052  6202 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:09.429639  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 70215us
I20260709 14:53:09.433538  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:09.440917  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:09.443233  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.446105  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:09.447098  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.446280  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.432426  6786 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:09.439014  6748 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:09.450289  6786 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.451486  6748 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:09.432520  6202 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:09.461066  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.448119  6631 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.463687  2753 raft_consensus.cc:2376] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13. Re-sending same reply.
I20260709 14:53:09.464056  6631 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:09.464990  6202 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:09.465219  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.466331  6631 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:09.444205  6799 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:09.467057  3543 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:09.468204  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:09.469003  6799 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:09.469336  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.459972  6748 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.474007  2755 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:09.463691  2608 raft_consensus.cc:1217] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[7.4-10.5]   Dedup: 10.5->[]
I20260709 14:53:09.475558  2608 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [7.4-10.5]
I20260709 14:53:09.476492  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1336) took 1724 ms. Trace:
I20260709 14:53:09.476961  2608 rpcz_store.cc:276] 0709 14:53:07.752154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.415820 (+1663666us) service_pool.cc:224] Handling call
0709 14:53:09.444837 (+ 29017us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:09.476014 (+ 31177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.476368 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.481190  2608 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 14:53:09.442523  6783 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:09.482800  6783 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.448163  6924 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:09.484812  6924 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:09.484951  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.485349  6924 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:09.485796  6783 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.487560  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1338) took 1725 ms. Trace:
I20260709 14:53:09.488873  2608 rpcz_store.cc:276] 0709 14:53:07.762545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.478363 (+1715818us) service_pool.cc:224] Handling call
0709 14:53:09.480408 (+  2045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:09.486776 (+  6368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.487411 (+   635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.100907  2608 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:10.104315  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:10.105214  2677 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:53:09.499523  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 238109us
I20260709 14:53:09.466689  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:09.491308  2758 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:09.487807  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:09.491008  3543 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 14:53:10.255234  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 967) took 1002 ms. Trace:
I20260709 14:53:10.255466  6958 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:10.255838  3543 rpcz_store.cc:276] 0709 14:53:09.253013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.469389 (+216376us) service_pool.cc:224] Handling call
0709 14:53:09.469858 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:10.254856 (+784998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.255152 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.688944  2609 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
I20260709 14:53:10.258328  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 728) took 1086 ms. Trace:
I20260709 14:53:10.258723  2609 rpcz_store.cc:276] 0709 14:53:09.171958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.247606 (+ 75648us) service_pool.cc:224] Handling call
0709 14:53:10.258103 (+1010497us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.258256 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322170752}
W20260709 14:53:09.698932  2725 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items [suppressed 2 similar messages]
I20260709 14:53:09.727630  2607 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:10.260485  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 736) took 1059 ms. Trace:
I20260709 14:53:10.260912  2607 rpcz_store.cc:276] 0709 14:53:09.201616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.359340 (+157724us) service_pool.cc:224] Handling call
0709 14:53:10.260408 (+901068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:09.739697  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.262508  2754 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:10.260040  2725 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42488 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1238)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1239)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1240)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 696)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 697)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 712)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1245)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 757)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 758)
W20260709 14:53:10.263149  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1248) took 2343 ms (client timeout 2191 ms). Trace:
W20260709 14:53:10.263597  2754 rpcz_store.cc:269] 0709 14:53:07.920176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.739180 (+1819004us) service_pool.cc:224] Handling call
0709 14:53:10.263059 (+523879us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:09.744364  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 750) took 2145 ms (client timeout 1968 ms). Trace:
I20260709 14:53:09.783967  2603 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:10.264815  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1250) took 2171 ms (client timeout 1952 ms). Trace:
W20260709 14:53:09.790504  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1361) took 2169 ms (client timeout 1542 ms). Trace:
W20260709 14:53:10.265465  2758 rpcz_store.cc:269] 0709 14:53:08.093557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.416497 (+1322940us) service_pool.cc:224] Handling call
0709 14:53:10.264588 (+848091us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.264738 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99931904}
W20260709 14:53:10.264941  2760 rpcz_store.cc:269] 0709 14:53:07.598980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.878197 (+1279217us) service_pool.cc:224] Handling call
0709 14:53:09.743958 (+865761us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.744216 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905623808}
I20260709 14:53:09.792023  3538 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:09.799902  2611 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:10.267411  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.267628  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.268131  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1390) took 1276 ms. Trace:
I20260709 14:53:09.801218  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1167) took 1687 ms. Trace:
I20260709 14:53:10.268316  2754 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.268458  2758 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:09.814357  3537 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: [9.5-10.6]
W20260709 14:53:10.269186  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 747) took 2774 ms (client timeout 2965 ms). Trace:
W20260709 14:53:09.833822  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:09.844782  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:10.269639  2758 rpcz_store.cc:269] 0709 14:53:07.494304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.267276 (+2772972us) service_pool.cc:224] Handling call
0709 14:53:10.269088 (+  1812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.270735  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1340) took 2502 ms. Trace:
I20260709 14:53:09.850607  2605 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:09.857867  3547 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 14:53:09.885115  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 705) took 3451 ms (client timeout 1809 ms). Trace:
I20260709 14:53:09.571072  6414 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.582115  6786 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.587033  6675 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:09.706609  6467 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:09.508832  6747 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.305313  3536 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:10.306387  6747 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.503321  6777 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:09.578075  6815 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:10.314289  6777 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:09.552474  6824 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.316571  6815 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:10.317948  2431 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 159) took 51189 ms (client timeout 15000 ms). Trace:
I20260709 14:53:10.318325  6824 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:09.789258  6826 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:09.487648  6862 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 14:53:09.560987  6876 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:09.772470  6868 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:09.573082  6924 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:09.860785  6936 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.656s	user 0.005s	sys 0.000s
W20260709 14:53:09.904510  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 741) took 2412 ms (client timeout 1662 ms). Trace:
W20260709 14:53:10.336655  2756 rpcz_store.cc:269] 0709 14:53:07.492404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.948749 (+456345us) service_pool.cc:224] Handling call
0709 14:53:09.904096 (+1955347us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b54002765d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.904390 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1747309312}
I20260709 14:53:09.915122  2612 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:10.337527  6924 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.338006  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55870 (request call id 1387) took 2807 ms. Trace:
I20260709 14:53:09.919514  2606 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:10.338116  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:09.926607  2686 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:10.339007  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1392) took 1346 ms. Trace:
I20260709 14:53:10.339080  2756 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:10.339444  2606 rpcz_store.cc:276] 0709 14:53:08.992785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.918752 (+925967us) service_pool.cc:224] Handling call
0709 14:53:09.919273 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.338615 (+419342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.338934 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:09.937652  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:09.928556  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.340991  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1156) took 4048 ms. Trace:
W20260709 14:53:10.341331  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.341424  2686 rpcz_store.cc:276] 0709 14:53:06.292355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.833558 (+541203us) service_pool.cc:224] Handling call
0709 14:53:06.834086 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.339114 (+3505028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.340655 (+  1541us) spinlock_profiling.cc:243] Waited 3.09 s on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:10.340905 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7110903552}
I20260709 14:53:10.342242  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:09.947621  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1388) took 2416 ms. Trace:
I20260709 14:53:10.343083  2682 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:53:09.958748  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.012s
I20260709 14:53:10.343413  6747 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.343644  2682 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:10.345063  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.009s	sys 0.012s
I20260709 14:53:10.344918  6747 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:10.345598  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 898599us
I20260709 14:53:10.345927  2682 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:10.010476  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1165) took 1896 ms (client timeout 1715 ms). Trace:
W20260709 14:53:10.347025  2685 rpcz_store.cc:269] 0709 14:53:08.113541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.260919 (+147378us) service_pool.cc:224] Handling call
0709 14:53:10.010162 (+1749243us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.010363 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557066496}
W20260709 14:53:10.028661  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 754) took 2078 ms (client timeout 2142 ms). Trace:
W20260709 14:53:10.035022  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 745) took 1692 ms (client timeout 2234 ms). Trace:
W20260709 14:53:10.348012  2755 rpcz_store.cc:269] 0709 14:53:07.950058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.438379 (+1488321us) service_pool.cc:224] Handling call
0709 14:53:10.028374 (+589995us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.028565 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15533824}
I20260709 14:53:10.349002  2685 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:10.349521  2682 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 11
W20260709 14:53:10.039561  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.041388  2680 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:10.059242  6892 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:10.351138  6892 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:10.351377  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.351264  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:10.064136  6937 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.807s	user 0.005s	sys 0.000s
I20260709 14:53:10.352244  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:10.073539  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:10.352952  6878 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.080947  2604 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:10.352228  6892 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:10.353574  6878 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.354435  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 735) took 1153 ms. Trace:
I20260709 14:53:10.354941  6892 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.354848  2604 rpcz_store.cc:276] 0709 14:53:09.201157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.079700 (+878543us) service_pool.cc:224] Handling call
0709 14:53:10.080391 (+   691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.354111 (+273720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.354369 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.089953  6905 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.356345  6905 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:10.343032  2610 rpcz_store.cc:276] 0709 14:53:07.531503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.704178 (+172675us) service_pool.cc:224] Handling call
0709 14:53:07.717119 (+ 12941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:08.854079 (+1136960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.947165 (+1093086us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002ce054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:09.947528 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375636352}
I20260709 14:53:10.111828  2753 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:10.358584  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:10.358875  6905 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:10.114616  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.090813  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.359452  3543 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
W20260709 14:53:10.359835  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 269102us
I20260709 14:53:10.360821  2681 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:10.361147  6878 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.361465  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.361881  6878 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:10.362310  6878 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.090680  6820 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 14:53:10.165946  2577 connection.cc:256] RPC call timeout handler was delayed by -1.2089218830000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 14:53:10.363487  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.365058  6740 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:10.365517  2577 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.504s (SENT)
W20260709 14:53:10.193305  6938 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.936s	user 0.000s	sys 0.007s
W20260709 14:53:10.199211  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.365850  6740 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.371596  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.371034  6924 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.373256  6924 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.269899  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:10.252291  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 764514us
I20260709 14:53:10.372278  6740 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:10.374269  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:10.374749  2760 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:10.375281  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1364) took 1845 ms (client timeout 2120 ms). Trace:
W20260709 14:53:10.265996  2752 rpcz_store.cc:269] 0709 14:53:07.621277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.046014 (+424737us) service_pool.cc:224] Handling call
0709 14:53:09.790178 (+1744164us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.790324 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808453504}
W20260709 14:53:10.375774  2760 rpcz_store.cc:269] 0709 14:53:08.529432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.269516 (+1740084us) service_pool.cc:224] Handling call
0709 14:53:10.375208 (+105692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.376581  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.376768  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.271171  2603 rpcz_store.cc:276] 0709 14:53:07.763308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.782912 (+2019604us) service_pool.cc:224] Handling call
0709 14:53:09.783731 (+   819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.265115 (+481384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.265436 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:10.377424  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 744) took 2884 ms (client timeout 3263 ms). Trace:
W20260709 14:53:10.377491  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 253573us
W20260709 14:53:10.377830  2752 rpcz_store.cc:269] 0709 14:53:07.493313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.376222 (+2882909us) service_pool.cc:224] Handling call
0709 14:53:10.377346 (+  1124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.272325  2611 rpcz_store.cc:276] 0709 14:53:08.992115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.799273 (+807158us) service_pool.cc:224] Handling call
0709 14:53:09.799669 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.267780 (+468111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.268039 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.372303  6862 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:10.318517  2431 rpcz_store.cc:269] 0709 14:52:19.128426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:19.141009 (+ 12583us) service_pool.cc:224] Handling call
0709 14:53:10.317636 (+51176627us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:53:04.556781 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:53:05.027235 (+470454us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:53:05.071377 (+ 44142us) write_op.cc:435] Acquired schema lock
0709 14:53:05.078441 (+  7064us) tablet.cc:662] Decoding operations
0709 14:53:05.181258 (+102817us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:53:05.181383 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 14:53:05.181412 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 14:53:05.182488 (+  1076us) tablet.cc:701] Row locks acquired
0709 14:53:05.182560 (+    72us) write_op.cc:260] PREPARE: finished
0709 14:53:05.182890 (+   330us) write_op.cc:270] Start()
0709 14:53:05.183098 (+   208us) write_op.cc:276] Timestamp: P: 1783608785182742 usec, L: 0
0709 14:53:05.183131 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 14:53:05.184337 (+  1206us) log.cc:844] Serialized 3257 byte log entry
0709 14:53:06.041584 (+857247us) op_driver.cc:464] REPLICATION: finished
0709 14:53:06.263946 (+222362us) write_op.cc:301] APPLY: starting
0709 14:53:06.264074 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 14:53:06.370244 (+106170us) tablet.cc:1370] finished BulkCheckPresence
0709 14:53:06.370484 (+   240us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:53:07.114707 (+744223us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:53:07.115202 (+   495us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 14:53:07.115280 (+    78us) write_op.cc:312] APPLY: finished
0709 14:53:07.115815 (+   535us) log.cc:844] Serialized 80 byte log entry
0709 14:53:07.601644 (+485829us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:07.634763 (+ 33119us) write_op.cc:454] Released schema lock
0709 14:53:07.722865 (+ 88102us) write_op.cc:341] FINISH: Updating metrics
0709 14:53:07.772909 (+ 50044us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:07.778567 (+  5658us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7612196,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":213246,"apply.run_cpu_time_us":61084,"apply.run_wall_time_us":1486621,"cfile_cache_hit":30,"cfile_cache_hit_bytes":31335,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6216,"cfile_init":1,"lbm_read_time_us":4912,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":7,"num_ops":8,"prepare.queue_time_us":159752,"prepare.run_cpu_time_us":30129,"prepare.run_wall_time_us":810701,"raft.queue_time_us":301092,"raft.run_cpu_time_us":29647,"raft.run_wall_time_us":120153,"replication_time_us":858124,"thread_start_us":960642,"threads_started":5,"wal-append.queue_time_us":926734,"wal-append.run_cpu_time_us":45822,"wal-append.run_wall_time_us":3039499}]]}
I20260709 14:53:10.275381  6957 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.268702  2679 rpcz_store.cc:276] 0709 14:53:08.114051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.743846 (+629795us) service_pool.cc:224] Handling call
0709 14:53:08.827127 (+ 83281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:08.828612 (+  1485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:09.800860 (+972248us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b5400445550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.801126 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176941824}
I20260709 14:53:10.277031  6958 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:10.389206  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.390255  2752 raft_consensus.cc:2368] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:10.390053  6958 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:10.390756  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 736) took 2899 ms (client timeout 3648 ms). Trace:
W20260709 14:53:10.391175  2752 rpcz_store.cc:269] 0709 14:53:07.491035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.378680 (+2887645us) service_pool.cc:224] Handling call
0709 14:53:10.390694 (+ 12014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.392216  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.392457  6958 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:10.284233  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:10.392550  2577 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.819s (SENT)
I20260709 14:53:10.286506  6414 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:10.393595  6414 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:09.888474  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 745) took 2394 ms (client timeout 1845 ms). Trace:
I20260709 14:53:10.394788  6693 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:53:10.395027  2757 rpcz_store.cc:269] 0709 14:53:07.493622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.579715 (+1086093us) service_pool.cc:224] Handling call
0709 14:53:09.888042 (+1308327us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.888320 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623715072}
I20260709 14:53:10.284972  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:10.396559  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1246) took 2623 ms (client timeout 2932 ms). Trace:
W20260709 14:53:10.397082  2758 rpcz_store.cc:269] 0709 14:53:07.773059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.277093 (+2504034us) service_pool.cc:224] Handling call
0709 14:53:10.396483 (+119390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.398214  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.398458  6820 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.317595  2435 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:10.399081  6820 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:10.309600  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1239) took 1361 ms. Trace:
I20260709 14:53:10.400599  3536 rpcz_store.cc:276] 0709 14:53:08.948399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.010131 (+ 61732us) service_pool.cc:224] Handling call
0709 14:53:10.309251 (+1299120us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002012d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.309503 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902284672}
I20260709 14:53:10.400162  2435 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.402073  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:10.402405  6414 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:10.307752  6675 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:10.326622  6876 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.403203  2758 raft_consensus.cc:2368] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:10.402719  2435 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.403776  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 742) took 2911 ms. Trace:
W20260709 14:53:10.339840  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1365) took 1810 ms (client timeout 2188 ms). Trace:
W20260709 14:53:09.959605  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 503715us
I20260709 14:53:09.968845  2751 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:10.404325  2756 rpcz_store.cc:269] 0709 14:53:08.529739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.337688 (+1807949us) service_pool.cc:224] Handling call
0709 14:53:10.339677 (+  1989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.389309  6957 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.405639  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:10.405982  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 752) took 2456 ms (client timeout 1866 ms). Trace:
I20260709 14:53:10.406243  2756 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11. Re-sending same reply.
I20260709 14:53:10.340619  2612 rpcz_store.cc:276] 0709 14:53:07.531074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:07.531994 (+   920us) service_pool.cc:224] Handling call
0709 14:53:09.913385 (+2381391us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54002ce054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e63da3d 00007fa17360dcd0 00007fa16cb92254 00007fa16cb921e2 00007fa16cb92111 00007fa16cb9208c 00007fa16cb91342 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.337836 (+424451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201963264}
W20260709 14:53:10.348088  2684 rpcz_store.cc:269] 0709 14:53:08.342451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.416148 (+ 73697us) service_pool.cc:224] Handling call
0709 14:53:10.034620 (+1618472us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54002b2854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.034931 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947016448}
I20260709 14:53:10.407438  6877 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:10.407922  6877 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:10.349460  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.039494  6797 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.347601  6747 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:10.343151  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.382419  6924 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.352334  2680 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:53:10.356897  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:10.139020  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1246) took 1187 ms. Trace:
I20260709 14:53:10.140216  3546 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:10.130493  2677 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
W20260709 14:53:10.352231  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.350613  2685 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:10.371521  6495 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:10.431579  6495 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:10.430934  2435 catalog_manager.cc:5697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.200202  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:10.373719  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:10.271936  2759 rpcz_store.cc:269] 0709 14:53:06.434158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:06.690804 (+256646us) service_pool.cc:224] Handling call
0709 14:53:09.878745 (+3187941us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:09.884992 (+  6247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4749809024}
I20260709 14:53:10.371380  6864 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.372073  6764 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 14:53:10.273584  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.272189  6613 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.274191  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1391) took 1281 ms. Trace:
W20260709 14:53:10.370553  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.381348  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:10.482942  2605 rpcz_store.cc:276] 0709 14:53:08.992475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.849648 (+857173us) service_pool.cc:224] Handling call
0709 14:53:09.850155 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.271428 (+421273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.274115 (+  2687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.375166  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:10.375651  6824 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:10.375211  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:10.379549  6862 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.484550  3540 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.374114  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:10.396495  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:10.485131  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61219.7us
I20260709 14:53:10.312565  6959 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:10.036630  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.408022  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 763) took 1157 ms. Trace:
I20260709 14:53:10.491376  2756 rpcz_store.cc:276] 0709 14:53:09.249863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.405267 (+1155404us) service_pool.cc:224] Handling call
0709 14:53:10.406764 (+  1497us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:10.406626  2751 rpcz_store.cc:269] 0709 14:53:07.949389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.262603 (+1313214us) service_pool.cc:224] Handling call
0709 14:53:10.405747 (+1143144us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.405885 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519741696}
I20260709 14:53:10.408241  6868 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 14:53:10.325392  2754 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:10.496767  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1373) took 1735 ms (client timeout 1607 ms). Trace:
I20260709 14:53:10.326210  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:10.413901  2753 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:53:10.498420  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1370) took 1914 ms (client timeout 1784 ms). Trace:
W20260709 14:53:10.499034  2753 rpcz_store.cc:269] 0709 14:53:08.583801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.466316 (+882515us) service_pool.cc:224] Handling call
0709 14:53:10.498306 (+1031990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.404191  2758 rpcz_store.cc:276] 0709 14:53:07.492674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.397825 (+2905151us) service_pool.cc:224] Handling call
0709 14:53:10.403670 (+  5845us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:10.025000  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.010s	sys 0.014s
I20260709 14:53:10.362673  6878 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:10.425236  6820 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:10.502820  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 128868us
I20260709 14:53:10.423992  6957 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:10.427215  3537 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:10.408075  6626 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 14:53:10.432106  6495 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:10.513065  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.478376  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:10.479756  2435 catalog_manager.cc:5697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.488806  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:10.511437  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113865us
I20260709 14:53:10.429456  3542 rpcz_store.cc:276] 0709 14:53:08.951368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.057063 (+105695us) service_pool.cc:224] Handling call
0709 14:53:10.138640 (+1081577us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400318154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.138923 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420671232}
I20260709 14:53:10.482726  3543 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:10.487533  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.514940  2685 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:10.493193  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.402776  3536 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:10.493762  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 118666us
I20260709 14:53:10.515938  2751 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:10.494088  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:10.516692  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.494678  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:10.517570  2677 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:10.485526  2757 raft_consensus.cc:2376] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
W20260709 14:53:10.500537  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.011s	sys 0.014s
I20260709 14:53:10.423259  2755 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 14:53:10.497355  2754 rpcz_store.cc:269] 0709 14:53:08.761827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.264621 (+1502794us) service_pool.cc:224] Handling call
0709 14:53:10.496688 (+232067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.421872  2686 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:10.518707  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 760) took 1344 ms. Trace:
I20260709 14:53:10.503983  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:10.506292  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.506362  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:10.519302  2757 rpcz_store.cc:276] 0709 14:53:09.174772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.396142 (+1221370us) service_pool.cc:224] Handling call
0709 14:53:10.518639 (+122497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.519726  3545 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.520237  3538 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:10.509609  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1092) took 1596 ms. Trace:
I20260709 14:53:10.529305  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.530797  2756 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:10.431499  6877 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.513438  2752 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:10.482577  6764 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:10.520720  3546 rpcz_store.cc:276] 0709 14:53:08.901529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:08.901860 (+   331us) service_pool.cc:224] Handling call
0709 14:53:09.006716 (+104856us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:53:10.139839 (+1133123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:10.430264 (+290425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.492210 (+ 61946us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400318150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.498309 (+  6099us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460253440}
W20260709 14:53:10.531948  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133708us
W20260709 14:53:10.532322  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1371) took 1775 ms (client timeout 1981 ms). Trace:
W20260709 14:53:10.539033  2755 rpcz_store.cc:269] 0709 14:53:08.756899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.349115 (+1592216us) service_pool.cc:224] Handling call
0709 14:53:10.532249 (+183134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.538151  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:10.514154  2680 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:10.493321  6864 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.535727  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.502064  6777 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.547472  6924 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:10.548036  6924 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:10.547783  6777 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.548511  6924 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:10.550374  6777 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.538026  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.491194  6905 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:10.541218  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63074.4us
I20260709 14:53:10.493427  6693 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 14:53:10.575649  6693 consensus_queue.cc:1243] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:10.575830  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.496894  6959 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.577623  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.578480  6959 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:10.575870  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:10.579221  6868 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:10.579739  2609 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:10.580009  2610 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:10.580153  6693 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:10.544571  6495 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:10.580621  6693 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:10.546324  6963 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:53:10.546964  6740 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.547048  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:10.502856  6786 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.512152  6820 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.551040  6748 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.497680  6675 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 14:53:10.508606  6414 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.551234  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.595372  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.595770  2754 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:10.596019  2755 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:10.596268  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1366) took 2065 ms. Trace:
I20260709 14:53:10.596696  2754 rpcz_store.cc:276] 0709 14:53:08.531348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.535009 (+2003661us) service_pool.cc:224] Handling call
0709 14:53:10.596199 (+ 61190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.432367  6797 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.552722  6877 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:10.511547  6747 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:10.552845  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:10.553817  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 227773us
I20260709 14:53:10.554033  6764 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.508845  6613 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:10.535544  2435 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 9, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.537566  6815 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.554989  6961 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:10.611068  6797 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.612236  6961 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:10.612953  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.613790  2754 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.613391  6961 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:10.556329  6864 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.556948  2758 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:10.556982  6862 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.557739  6906 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.620904  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.621495  6862 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.546517  3542 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:10.621732  2758 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:10.621991  6862 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.622386  6862 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.593570  6786 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:10.623096  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:10.623788  3542 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:10.602415  6905 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.624778  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:10.626222  2683 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:10.568307  2677 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:10.584403  6964 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 14:53:10.580049  6868 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 14:53:10.628350  6868 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:10.570533  2751 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:10.629097  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 762) took 1409 ms. Trace:
I20260709 14:53:10.536999  2759 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:10.629421  6868 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:10.572721  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:10.576143  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:10.594063  6748 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.534157  6626 consensus_queue.cc:1243] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:10.644074  2680 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:10.581682  6693 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.582250  6740 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:10.582538  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:10.563850  6957 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:10.666949  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71381.2us
I20260709 14:53:10.581902  2611 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:10.670933  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.672885  2760 raft_consensus.cc:2368] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
W20260709 14:53:10.673940  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 115579us
I20260709 14:53:10.676702  2754 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:10.678419  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.679450  2754 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:10.585511  2682 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:10.583747  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:10.687839  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104172us
W20260709 14:53:10.588856  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.592586  6777 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:10.584913  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 966) took 1332 ms. Trace:
I20260709 14:53:10.594085  2607 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:53:10.607661  2757 raft_consensus.cc:2368] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
W20260709 14:53:10.617314  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1077931410000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 14:53:10.697211  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0483931410000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 14:53:10.697719  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0429931410000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 14:53:10.612326  6764 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:10.630239  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.571610  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.629439  2751 rpcz_store.cc:276] 0709 14:53:09.220197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.492876 (+1272679us) service_pool.cc:224] Handling call
0709 14:53:10.629028 (+136152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:10.631026  3542 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:10.632948  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.633562  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.704551  2758 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.639557  3545 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:10.704984  2753 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:10.638069  2435 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 12, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:53:10.650494  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116012us
I20260709 14:53:10.706607  2435 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.652881  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 761) took 1459 ms. Trace:
I20260709 14:53:10.708994  2752 rpcz_store.cc:276] 0709 14:53:09.193467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.391899 (+1198432us) service_pool.cc:224] Handling call
0709 14:53:10.652558 (+260659us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540040ecd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:10.652778 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277131520}
W20260709 14:53:10.649567  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72001.4us
I20260709 14:53:10.656682  3536 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:10.584156  6826 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 14:53:10.656119  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:10.693132  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.632948  6959 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:10.693197  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.632174  6748 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.617832  6675 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:10.696419  2607 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 14:53:10.701742  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:10.702361  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.608124  6467 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 14:53:10.714162  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.710912  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.714654  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:10.600605  6414 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:10.601495  6820 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.719594  2756 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:10.715431  2435 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 12, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:10.717149  2607 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:10.710738  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118950us
I20260709 14:53:10.717659  2685 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:10.667727  6957 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:10.719264  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:10.694897  3541 rpcz_store.cc:276] 0709 14:53:09.252563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:09.409497 (+156934us) service_pool.cc:224] Handling call
0709 14:53:09.410095 (+   598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:09.421064 (+ 10969us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:53:09.442371 (+ 21307us) log.cc:844] Serialized 14 byte log entry
0709 14:53:10.564230 (+1121859us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:10.576245 (+ 12015us) log.cc:844] Serialized 52 byte log entry
0709 14:53:10.576479 (+   234us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:10.576518 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:10.576590 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:10.584568 (+  7978us) raft_consensus.cc:1685] finished
0709 14:53:10.584602 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:10.584824 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21744,"prepare.queue_time_us":18180,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":80685,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":15605,"thread_start_us":1059627,"threads_started":1,"wal-append.queue_time_us":125352}
I20260709 14:53:10.632400  6815 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.728678  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:10.726459  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.731403  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.741417  2752 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:10.731731  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 219552us
I20260709 14:53:10.621543  6906 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:10.736570  3411 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.968s (SENT)
I20260709 14:53:10.739254  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:10.743875  2683 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:10.746934  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.747901  6820 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.748626  6820 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:53:10.749478  6414 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.750473  2686 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:10.751542  6626 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:10.714805  6693 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.716182  6675 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:10.713511  6797 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 14:53:10.776261  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.756121  6905 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.756263  2751 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:10.757835  6740 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:10.759406  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:10.759128  2435 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 10, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 10 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:10.762748  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:10.711094  6895 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:10.772156  2754 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:10.792685  6895 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.776664  2685 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:10.793359  6895 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:10.761689  6878 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.796169  6878 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:10.797088  2677 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:10.792690  2758 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:10.790544  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:10.740636  6957 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.741178  6912 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 14:53:10.818073  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 89554.8us
I20260709 14:53:10.817602  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:10.830502  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 159663us
I20260709 14:53:10.777297  6969 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 14:53:10.849411  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 147143us
I20260709 14:53:10.789620  6797 consensus_queue.cc:1243] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:10.789232  6892 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:10.826704  3540 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:10.813078  2435 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 12, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
W20260709 14:53:10.834468  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106164us
I20260709 14:53:10.723274  6777 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:10.852849  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.776878  6626 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:10.831893  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:10.855290  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 144527us
W20260709 14:53:10.857254  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 131527us
I20260709 14:53:10.860410  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.856042  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:10.864013  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120355us
W20260709 14:53:10.865576  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106323us
I20260709 14:53:10.857659  6878 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:10.864864  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:10.869047  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:10.803618  6972 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.822670  6976 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.830642  6414 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:10.854364  6826 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.880249  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:10.875476  2606 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:53:10.818989  6958 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 14:53:10.857339  6626 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:10.862171  6740 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:10.854912  6892 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.885011  2752 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 14:53:10.875624  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 182579us
I20260709 14:53:10.848318  6969 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:10.919173  6969 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:10.923262  6971 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:10.927543  2435 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:10.929617  2435 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 9, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.930918  6740 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.931574  6740 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:10.882001  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.893631  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:10.898619  6467 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:10.899236  6978 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:10.866690  6878 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:10.978515  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 85008.3us
W20260709 14:53:10.896411  3572 debug-util.cc:398] Leaking SignalData structure 0x7b080028ede0 after lost signal to thread 434
I20260709 14:53:10.880122  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:10.980628  2608 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:10.880280  6675 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:10.863397  6964 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:10.903952  6815 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.875550  6972 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:10.824057  6905 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 14:53:10.876559  6976 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.908737  6777 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:10.888038  6826 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:10.912855  2606 pending_rounds.cc:85] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:53:10.917469  6895 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:10.953361  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:10.954979  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:10.935333  2435 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 10, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:10.978437  2755 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
W20260709 14:53:10.999347  3572 debug-util.cc:398] Leaking SignalData structure 0x7b0800157220 after lost signal to thread 2469
W20260709 14:53:11.000296  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50362.8us
W20260709 14:53:11.034765  3572 debug-util.cc:398] Leaking SignalData structure 0x7b080026a8e0 after lost signal to thread 2636
W20260709 14:53:11.004756  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 466793us
I20260709 14:53:11.008711  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:11.037303  2752 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 14:53:11.016888  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:11.018328  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 104472us
I20260709 14:53:11.035678  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.038157  2751 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 5 FOLLOWER]: Advancing to term 6
I20260709 14:53:11.009763  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:11.018428  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 759) took 1866 ms (client timeout 1904 ms). Trace:
I20260709 14:53:11.021513  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:10.978000  6978 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:11.040524  2754 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 14:53:11.023324  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141441us
I20260709 14:53:10.935292  6892 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:10.868168  6912 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 14:53:11.026679  2606 pending_rounds.cc:107] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 14:53:11.042053  2606 raft_consensus.cc:2889] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:11.035816  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:11.039152  2760 rpcz_store.cc:269] 0709 14:53:09.151667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.376539 (+1224872us) service_pool.cc:224] Handling call
0709 14:53:11.018081 (+641542us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:11.018327 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663003008}
I20260709 14:53:11.046308  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1344) took 4955 ms. Trace:
I20260709 14:53:10.998538  6878 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:11.031410  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:11.039263  3545 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:53:11.048588  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 258151us
I20260709 14:53:11.003441  6971 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:10.927570  6613 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:11.050413  3572 debug-util.cc:398] Leaking SignalData structure 0x7b0800171ae0 after lost signal to thread 2637
I20260709 14:53:11.057811  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:11.050506  2760 rpcz_store.cc:276] 0709 14:53:06.091235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.044777 (+4953542us) service_pool.cc:224] Handling call
0709 14:53:11.045350 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.045652 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.045698 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:11.045776 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:11.045888 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.045976 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:11.046162 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:11.062071  3572 debug-util.cc:398] Leaking SignalData structure 0x7b08001610c0 after lost signal to thread 2710
I20260709 14:53:11.051828  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:11.051946  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.058907  3536 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:53:11.064762  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.064869  2755 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 9. Current term is 11. Ops: []
W20260709 14:53:11.065948  3572 debug-util.cc:398] Leaking SignalData structure 0x7b08003d2780 after lost signal to thread 2784
I20260709 14:53:11.052927  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.012492  6972 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.024147  6976 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.055357  2435 catalog_manager.cc:5697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 9, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
W20260709 14:53:11.072144  3572 debug-util.cc:398] Leaking SignalData structure 0x7b08003d2740 after lost signal to thread 2785
I20260709 14:53:11.072970  2752 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:11.063609  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:11.069500  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.030005  6797 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:11.008404  6467 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:11.032147  6826 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.068984  6977 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 14:53:11.041682  6675 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.077097  3542 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:11.074563  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1239) took 4975 ms. Trace:
W20260709 14:53:11.077023  3572 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9100 after lost signal to thread 3580
I20260709 14:53:11.078903  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:11.097007  3572 debug-util.cc:398] Leaking SignalData structure 0x7b080035e640 after lost signal to thread 3581
W20260709 14:53:11.089076  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.096861  2755 rpcz_store.cc:276] 0709 14:53:06.090781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.036034 (+4945253us) service_pool.cc:224] Handling call
0709 14:53:11.064590 (+ 28556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.065530 (+   940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.065811 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.089272  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:11.094656  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1350) took 4928 ms. Trace:
I20260709 14:53:11.098156  2435 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:11.049467  6978 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:11.100780  2435 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 10, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:11.099078  2752 rpcz_store.cc:276] 0709 14:53:06.163305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.051952 (+4888647us) service_pool.cc:224] Handling call
0709 14:53:11.072691 (+ 20739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.091059 (+ 18368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.091293 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.108593  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:11.109529  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.109886  2757 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:53:11.103945  2435 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 8, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:53:11.113463  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103825us
I20260709 14:53:11.114383  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.082408  6777 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:11.117456  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.096167  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:11.105350  6877 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.001s
I20260709 14:53:11.123430  2753 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:11.131778  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1238) took 5034 ms. Trace:
I20260709 14:53:11.133072  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1351) took 4969 ms. Trace:
W20260709 14:53:11.131353  3411 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 3.648s (SENT)
I20260709 14:53:11.132690  2757 rpcz_store.cc:276] 0709 14:53:06.090325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.036599 (+4946274us) service_pool.cc:224] Handling call
0709 14:53:11.106619 (+ 70020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:11.122214 (+ 15595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.122513 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.133695  2753 rpcz_store.cc:276] 0709 14:53:06.163680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.066516 (+4902836us) service_pool.cc:224] Handling call
0709 14:53:11.123226 (+ 56710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.132664 (+  9438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.133005 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.053540  6984 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:11.139766  2751 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 6.
I20260709 14:53:11.113914  6967 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:11.106961  6985 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.144037  2609 raft_consensus.cc:2410] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 14:53:11.116351  6864 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.002s
W20260709 14:53:11.146096  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.112991  6990 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:11.117163  6820 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:11.121241  6971 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:11.105626  6748 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.152143  6968 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:11.155252  6786 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:11.104562  6815 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 14:53:11.147645  2755 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 14:53:11.173683  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1241) took 4954 ms. Trace:
I20260709 14:53:11.140496  2435 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 8, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 8 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:11.123064  6989 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:11.177711  2753 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:11.178366  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 697) took 4927 ms. Trace:
I20260709 14:53:11.165953  6864 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:11.179610  6981 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 14:53:11.180917  6983 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 14:53:11.181269  6864 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.182857  6994 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:11.182847  2684 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:11.178701  2753 rpcz_store.cc:276] 0709 14:53:06.250675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.145432 (+4894757us) service_pool.cc:224] Handling call
0709 14:53:11.177399 (+ 31967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.178084 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.178302 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.186614  2753 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 14:53:11.187162  2753 pending_rounds.cc:85] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
I20260709 14:53:11.187459  2753 pending_rounds.cc:107] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 14:53:11.174690  6820 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:11.187673  6924 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.639s	user 0.014s	sys 0.000s
I20260709 14:53:11.188369  6989 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:11.176347  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55922.2us
I20260709 14:53:11.176753  2751 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:11.187879  2753 raft_consensus.cc:2889] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:11.174238  2755 rpcz_store.cc:276] 0709 14:53:06.219083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.097860 (+4878777us) service_pool.cc:224] Handling call
0709 14:53:11.147323 (+ 49463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:11.173262 (+ 25939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.173610 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:11.335021  6979 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.226s	user 0.004s	sys 0.000s
I20260709 14:53:11.335475  6626 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.160678  6985 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.280821  6906 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:11.188450  6924 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.257381  3542 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:11.210498  6820 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:11.232296  6740 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:11.237912  2608 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:11.209448  3411 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.662s (SENT)
I20260709 14:53:11.413098  2612 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:11.412998  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:11.227480  2435 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 7, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:11.207906  2757 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:11.214465  3547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.011s	sys 0.010s
W20260709 14:53:11.415354  3547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.015s	sys 0.014s
I20260709 14:53:11.263970  6895 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:11.174196  6990 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:11.266305  6862 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.002s
W20260709 14:53:11.298666  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.259306  2758 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 12. Ops: []
I20260709 14:53:11.426095  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.426748  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:11.426935  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 708) took 4935 ms. Trace:
I20260709 14:53:11.427345  2758 rpcz_store.cc:276] 0709 14:53:06.491562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.258415 (+4766853us) service_pool.cc:224] Handling call
0709 14:53:11.259025 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.426603 (+167578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.426871 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.428337  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.212620  6495 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:11.268714  6748 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.433885  3537 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:11.208076  6989 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.269737  6414 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:11.167223  6971 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:11.171999  6984 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:11.250615  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 214913us
I20260709 14:53:11.223343  6987 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:11.174621  6877 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.321301  6826 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:11.350051  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:11.457072  2609 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:11.353549  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:11.459332  2607 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:11.281160  6876 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:11.460575  6876 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:11.434923  6862 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:11.450179  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.465519  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:11.338969  6959 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.005s
I20260709 14:53:11.342803  6969 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:11.152238  6824 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:11.236636  6964 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:11.146876  6980 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:11.189519  6467 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:11.223508  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153685us
I20260709 14:53:11.243119  2605 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 14:53:11.406591  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 706) took 4942 ms. Trace:
W20260709 14:53:11.408701  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.015s	sys 0.027s
W20260709 14:53:11.476740  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.015s	sys 0.027s
I20260709 14:53:11.476534  2753 rpcz_store.cc:276] 0709 14:53:06.456499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.185457 (+4728958us) service_pool.cc:224] Handling call
0709 14:53:11.185891 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.398545 (+212654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.398780 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.409551  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1242) took 5017 ms. Trace:
W20260709 14:53:11.418740  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 300606us
W20260709 14:53:11.477653  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 612819us
W20260709 14:53:11.420517  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.421494  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 707) took 4940 ms. Trace:
I20260709 14:53:11.478148  2751 rpcz_store.cc:276] 0709 14:53:06.380136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.142936 (+4762800us) service_pool.cc:224] Handling call
0709 14:53:11.171951 (+ 29015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.396223 (+224272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.397536 (+  1313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.478526  2757 rpcz_store.cc:276] 0709 14:53:06.481403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.203624 (+4722221us) service_pool.cc:224] Handling call
0709 14:53:11.207644 (+  4020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.421069 (+213425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.421401 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:11.424084  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 325775us
I20260709 14:53:11.434808  2756 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:11.398964  6961 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.436954  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:11.488416  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:11.446255  6971 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:11.438105  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 329602us
W20260709 14:53:11.442616  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.443962  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.500775  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:11.501133  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57290.7us
I20260709 14:53:11.488736  3541 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
W20260709 14:53:11.444531  2603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.012s
W20260709 14:53:11.502081  2603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.012s
I20260709 14:53:11.501910  2609 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:11.444977  2435 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:11.451862  2758 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:11.503520  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 712) took 4937 ms. Trace:
I20260709 14:53:11.504102  2758 rpcz_store.cc:276] 0709 14:53:06.566636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.428043 (+4861407us) service_pool.cc:224] Handling call
0709 14:53:11.451720 (+ 23677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.503110 (+ 51390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.503429 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.395246  6957 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.453603  6990 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:11.454722  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.456066  2755 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:11.517297  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 709) took 5015 ms. Trace:
I20260709 14:53:11.517722  2755 rpcz_store.cc:276] 0709 14:53:06.502397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.420626 (+4918229us) service_pool.cc:224] Handling call
0709 14:53:11.455625 (+ 34999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.516768 (+ 61143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.517200 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.469662  2754 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:11.396994  6976 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.519735  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:11.419565   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:11.480201  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.482676  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:11.482532  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 273054us
I20260709 14:53:11.485669  2605 pending_rounds.cc:85] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 7
I20260709 14:53:11.464293  6626 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.537158  6969 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:11.538149  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:11.412837  6878 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:11.489902  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:11.540354  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50547us
I20260709 14:53:11.541205  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.541452  6878 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.534396  6967 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:11.543535  6967 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:11.544596  6967 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:11.545111  6967 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:11.545565  6967 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:11.546917  2758 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:11.547147  6824 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:11.547730  6824 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:11.548112  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1247) took 3629 ms. Trace:
I20260709 14:53:11.548707  2758 rpcz_store.cc:276] 0709 14:53:07.918604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.519179 (+3600575us) service_pool.cc:224] Handling call
0709 14:53:11.546624 (+ 27445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.547697 (+  1073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.548031 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.549758  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 757) took 1656 ms. Trace:
I20260709 14:53:11.550246  2678 rpcz_store.cc:276] 0709 14:53:09.893207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.353884 (+460677us) service_pool.cc:224] Handling call
0709 14:53:10.354357 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:10.631079 (+276722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:10.631120 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:10.664863 (+ 33743us) log.cc:844] Serialized 52 byte log entry
0709 14:53:10.819764 (+154901us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:10.837714 (+ 17950us) log.cc:844] Serialized 14 byte log entry
0709 14:53:10.838778 (+  1064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.005775 (+166997us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:11.005909 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:11.256551 (+250642us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:11.549368 (+292817us) raft_consensus.cc:1685] finished
0709 14:53:11.549395 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:11.549683 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14556,"prepare.queue_time_us":17131,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":17068,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":14254,"spinlock_wait_cycles":622061568,"thread_start_us":84400,"threads_started":1,"wal-append.queue_time_us":310467}
I20260709 14:53:11.552248  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 757) took 2444 ms. Trace:
I20260709 14:53:11.552896  2751 rpcz_store.cc:276] 0709 14:53:09.107627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.514357 (+2406730us) service_pool.cc:224] Handling call
0709 14:53:11.551236 (+ 36879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.551693 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:11.551739 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:11.551847 (+   108us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:11.551902 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.551986 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:11.552162 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.527312  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:11.555346  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:11.486155  6980 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:11.556875  6980 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:11.433686  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.008s
I20260709 14:53:11.447273  6747 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:11.462085  6876 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.507526  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.507877  6906 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:11.508831  2435 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:11.487556  6693 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:11.507185  6987 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:11.576084  2751 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 14:53:11.577105  2751 pending_rounds.cc:85] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 6
I20260709 14:53:11.577409  2751 pending_rounds.cc:107] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:11.577733  2751 raft_consensus.cc:2889] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:11.579449  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1257) took 1195 ms. Trace:
I20260709 14:53:11.521155  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:11.522207  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56784.8us
I20260709 14:53:11.466954  6826 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.527400  2605 pending_rounds.cc:107] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 14:53:11.585093  2605 raft_consensus.cc:2889] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:11.492599  6748 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 14:53:11.473336  6959 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:11.467478  6978 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:11.511574  6961 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:11.535823  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53293.8us
I20260709 14:53:11.535913  6977 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 14:53:11.537108  3411 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.904s (SENT)
W20260709 14:53:11.558326  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78385.2us
I20260709 14:53:11.480715  6414 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.600381  6905 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:11.558750  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.008s	sys 0.008s
I20260709 14:53:11.600965  6905 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:11.601274  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 729807us
I20260709 14:53:11.561568  2651 leader_election.cc:400] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
I20260709 14:53:11.602566  2651 leader_election.cc:403] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:11.603891  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:11.604044  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:11.481047  6968 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:11.605532  6826 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:11.605584  6968 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:11.607183  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70127.4us
W20260709 14:53:11.608668  3411 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.938s (SENT)
I20260709 14:53:11.559338  6764 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:11.580133  2756 rpcz_store.cc:276] 0709 14:53:10.384219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.746548 (+362329us) service_pool.cc:224] Handling call
0709 14:53:11.579142 (+832594us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b5400423bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:11.579345 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579139456}
I20260709 14:53:11.564918  6797 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:11.596160  2753 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 13. Ops: []
I20260709 14:53:11.660300  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 713) took 5060 ms. Trace:
I20260709 14:53:11.596509  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1347) took 5416 ms. Trace:
I20260709 14:53:11.470430  6877 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.599992  6414 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:11.601375  6905 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:11.606199  6826 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:11.474665  6964 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:11.586380  2607 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:11.752977  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100097us
I20260709 14:53:11.529918  6892 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.614277  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.548141  6824 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:11.411692  6972 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:11.595389  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:11.762584  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1403) took 1190 ms. Trace:
W20260709 14:53:11.760702  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 146576us
I20260709 14:53:11.765170  2605 rpcz_store.cc:276] 0709 14:53:10.561754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.583228 (+ 21474us) service_pool.cc:224] Handling call
0709 14:53:10.598447 (+ 15219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.598966 (+1000519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.739038 (+140072us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540016e4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:11.751967 (+ 12929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360865024}
I20260709 14:53:11.590191  6878 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.171s
I20260709 14:53:11.516115  6912 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 14:53:11.573284  6982 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:11.627278  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:11.617739  2435 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:11.779961  2760 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:11.662106  2754 rpcz_store.cc:276] 0709 14:53:06.161710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.050272 (+4888562us) service_pool.cc:224] Handling call
0709 14:53:11.080921 (+ 30649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.519067 (+438146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.577624 (+ 58557us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:11.577916 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893438336}
I20260709 14:53:11.739663  6862 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.663064  2753 rpcz_store.cc:276] 0709 14:53:06.599948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.490094 (+4890146us) service_pool.cc:224] Handling call
0709 14:53:11.595896 (+105802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.659949 (+ 64053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.660220 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:11.665314  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:11.773325  3411 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.046s (SENT)
W20260709 14:53:11.675709  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134566us
I20260709 14:53:11.740937  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:11.573861  6987 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:11.759034  6864 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:11.755651  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 758) took 2616 ms. Trace:
I20260709 14:53:11.649102  6986 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 14:53:11.617198  6764 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:11.769846  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 166033us
W20260709 14:53:11.635259  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.779158  6912 consensus_queue.cc:1243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:11.649416  6977 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:11.758522  2678 pending_rounds.cc:85] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 6
W20260709 14:53:11.624428  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.786989  2754 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:11.786309  2435 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 10, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:11.591033  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1376) took 1027 ms. Trace:
W20260709 14:53:11.794795  6675 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.690s	user 0.013s	sys 0.000s
I20260709 14:53:11.793955  2756 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:11.825865  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1380) took 1248 ms. Trace:
I20260709 14:53:11.826267  2756 rpcz_store.cc:276] 0709 14:53:10.577645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.773356 (+1195711us) service_pool.cc:224] Handling call
0709 14:53:11.793706 (+ 20350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.825572 (+ 31866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.825791 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.760107  6797 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:11.826712  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:11.827262  2751 rpcz_store.cc:276] 0709 14:53:10.563300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.554321 (+991021us) service_pool.cc:224] Handling call
0709 14:53:11.571063 (+ 16742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.578337 (+  7274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.590934 (+ 12597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.828634  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1262) took 1247 ms. Trace:
I20260709 14:53:11.828985  2754 rpcz_store.cc:276] 0709 14:53:10.581573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.781587 (+1200014us) service_pool.cc:224] Handling call
0709 14:53:11.785737 (+  4150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.828352 (+ 42615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.828573 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:11.830369  2678 pending_rounds.cc:107] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:11.830684  2678 raft_consensus.cc:2889] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:11.761783  6992 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:11.759472  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 252007us
I20260709 14:53:11.773830  6959 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:11.794934  2755 rpcz_store.cc:276] 0709 14:53:09.139103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.518529 (+2379426us) service_pool.cc:224] Handling call
0709 14:53:11.550566 (+ 32037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.755009 (+204443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.755063 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:11.755138 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:11.755230 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.755352 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:11.755560 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218446,"thread_start_us":193775,"threads_started":1}
I20260709 14:53:11.799227  2752 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
W20260709 14:53:11.800786  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:11.833189  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.802081  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:11.775029  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 179720us
I20260709 14:53:11.807875  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:11.809407  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:11.835637  3541 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.836037  3537 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.836117  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:11.838248  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:11.838099  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:11.837819  2435 catalog_manager.cc:5697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:11.839720  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:11.840628  3536 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:11.849103  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 696) took 5605 ms. Trace:
I20260709 14:53:11.849686  2752 rpcz_store.cc:276] 0709 14:53:06.243420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.103367 (+4859947us) service_pool.cc:224] Handling call
0709 14:53:11.104069 (+   702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.798966 (+694897us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400161d20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:53:11.842773 (+ 43807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.849017 (+  6244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259410816}
I20260709 14:53:11.821455  6764 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:11.855412  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92311.4us
W20260709 14:53:11.900063  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64074us
I20260709 14:53:11.901374  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:11.832754  6740 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:11.833211  6959 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:11.842411  7006 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.832396  6924 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:11.843546  6815 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:11.738521  6984 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:11.919080  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 81055.4us
W20260709 14:53:11.928602  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120808us
I20260709 14:53:11.850871  6985 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:11.922708  3411 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.142s (SENT)
I20260709 14:53:11.796880  7002 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 14:53:11.806471  7014 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 14:53:11.860311  6820 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:11.867282  6862 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:11.923913  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 145505us
I20260709 14:53:11.938786  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:11.939728  2685 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.915390  6959 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:11.929589  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:11.912355  6969 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:11.946241  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:11.965005  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:11.915599  6982 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:11.938786  6675 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:11.971532  7009 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 14:53:11.972340  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:11.976087  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:11.977435  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:11.926883  6748 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:11.929518  6967 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:11.980501  6961 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:11.924877  7006 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.868992  6826 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:11.978152  6797 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:11.991533  2682 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
W20260709 14:53:11.992430  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63000.7us
I20260709 14:53:12.025563  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1240) took 5810 ms. Trace:
W20260709 14:53:12.027853  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71319.5us
I20260709 14:53:11.994700  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.028015  7005 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:12.033946  6972 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.034260  6961 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.028692  6675 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:11.945143  7001 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 14:53:11.945959  6957 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:12.036419  2685 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:12.038736  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:12.042802  2760 rpcz_store.cc:276] 0709 14:53:06.215042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.092375 (+4877333us) service_pool.cc:224] Handling call
0709 14:53:11.113963 (+ 21588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.780464 (+666501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:12.025127 (+244663us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:12.025458 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531043200}
I20260709 14:53:11.945520  6985 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.058811  2756 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:11.850399  6992 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:12.027870  6998 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.378s	user 0.002s	sys 0.000s
I20260709 14:53:11.985046  6976 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:12.060153  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.077747  2753 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 14:53:11.854143  6987 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:12.023044  6969 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:12.037833  6997 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:12.060936  6675 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:11.859629  6964 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.074056  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:12.027922  7016 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:12.067540  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.103451  6982 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:12.105263  7006 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:12.107677  7012 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 14:53:12.066442  6892 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.061441  7005 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.064766  2682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100d7ba791034203ab40e147a55e2183"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:12.032058  6748 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.116143  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.064064  6978 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.144693  2760 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:12.149910  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.150424  2753 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 14:53:12.150938  2753 pending_rounds.cc:85] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
I20260709 14:53:12.151203  2753 pending_rounds.cc:107] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:53:12.151495  2753 raft_consensus.cc:2889] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:12.145972  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 231631us
I20260709 14:53:12.072489  6958 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:12.069340  6414 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:12.076663  6986 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:12.150681  2760 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:12.079924  6912 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 14:53:12.155898  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 254752us
I20260709 14:53:12.061146  6961 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.162642  2682 raft_consensus.cc:606] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:12.114101  7016 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:12.170750  2682 raft_consensus.cc:614] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:12.166560  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.078238  6985 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.171578  2756 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:53:12.158221  6983 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:53:12.087121  6997 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:12.172780  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1261) took 1783 ms. Trace:
W20260709 14:53:12.173923  3411 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.157s (SENT)
W20260709 14:53:12.159983  6993 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:53:12.154376  6864 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:12.181110  2758 rpcz_store.cc:276] 0709 14:53:10.416280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.549836 (+1133556us) service_pool.cc:224] Handling call
0709 14:53:11.568362 (+ 18526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:11.569018 (+   656us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:11.780556 (+211538us) log.cc:844] Serialized 14 byte log entry
0709 14:53:12.165065 (+384509us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:12.165122 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:12.172423 (+  7301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:12.172531 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:12.172705 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":367547,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":937,"raft.queue_time_us":2264,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":2719,"thread_start_us":484916,"threads_started":2}
I20260709 14:53:12.113760  6957 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 14:53:12.164065  6987 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:12.191105  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 765) took 1127 ms. Trace:
I20260709 14:53:12.189785  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1418) took 1278 ms. Trace:
I20260709 14:53:12.168285  6958 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:12.247212  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.248083  2753 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:12.039642  6977 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:12.241979  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 75162.8us
I20260709 14:53:12.180665  6912 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:12.188169  7005 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.244840  2677 rpcz_store.cc:276] 0709 14:53:11.063726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.063864 (+   138us) service_pool.cc:224] Handling call
0709 14:53:11.091434 (+ 27570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:11.108939 (+ 17505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.108990 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:11.121357 (+ 12367us) log.cc:844] Serialized 26 byte log entry
0709 14:53:11.396690 (+275333us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:11.396739 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.396822 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:12.190787 (+793965us) raft_consensus.cc:1685] finished
0709 14:53:12.190811 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:12.191034 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28411,"raft.queue_time_us":17748,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":22068,"thread_start_us":187064,"threads_started":1,"wal-append.queue_time_us":354636}
I20260709 14:53:12.244921  2606 rpcz_store.cc:276] 0709 14:53:10.911673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.912012 (+   339us) service_pool.cc:224] Handling call
0709 14:53:10.912589 (+   577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:11.396260 (+483671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.396301 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:11.404264 (+  7963us) log.cc:844] Serialized 52 byte log entry
0709 14:53:11.761149 (+356885us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:11.761232 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.761314 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:12.189481 (+428167us) raft_consensus.cc:1685] finished
0709 14:53:12.189508 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:12.189688 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17592,"raft.queue_time_us":110341,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":539471,"threads_started":2,"wal-append.queue_time_us":348721}
I20260709 14:53:12.187242  6964 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:12.247448  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.255664  2758 raft_consensus.cc:2368] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:12.257016  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.203527  2680 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:12.173782  7016 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:12.182200  6997 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:12.190636  6978 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:12.253925  3411 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.213s (SENT)
W20260709 14:53:12.288900  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.280824  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.290283  2753 raft_consensus.cc:2376] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14. Re-sending same reply.
W20260709 14:53:12.298421  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49179.4us
I20260709 14:53:12.308000  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.250092  6864 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:12.320853  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 419538us
W20260709 14:53:12.326402  7026 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 14:53:12.331753  6786 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:12.332352  6786 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:12.333523  6786 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:12.334753  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1204134279998925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2922s for a response
W20260709 14:53:12.336421  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0750891330000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31000000000000005s for a response
W20260709 14:53:12.336731  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0090915120000545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 14:53:12.341221  3537 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
I20260709 14:53:12.341804  3537 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:12.261670  6820 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:12.343159  2753 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:12.335654  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 766) took 1271 ms. Trace:
W20260709 14:53:12.355636  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:12.358908  3412 connection.cc:256] RPC call timeout handler was delayed by -1.7266973429999553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 14:53:12.254848  6969 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:12.392791  3541 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.000s
W20260709 14:53:12.395532  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.047s	sys 0.032s
I20260709 14:53:12.395987  3541 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:12.396251  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.047s	sys 0.032s
I20260709 14:53:12.337216  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:12.398020  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62451.9us
W20260709 14:53:12.400084  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:12.402015  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.403570  3541 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:12.403841  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:12.405315  3545 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:12.419201  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:12.420162  3536 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:12.420418  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:12.393735  2679 rpcz_store.cc:276] 0709 14:53:11.064507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.064942 (+   435us) service_pool.cc:224] Handling call
0709 14:53:11.090966 (+ 26024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:11.112789 (+ 21823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.112825 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:11.122891 (+ 10066us) log.cc:844] Serialized 26 byte log entry
0709 14:53:11.443261 (+320370us) raft_consensus.cc:1645] Marking committed up to 3
0709 14:53:11.443321 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.443418 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:12.335336 (+891918us) raft_consensus.cc:1685] finished
0709 14:53:12.335372 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:12.335576 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4320,"raft.queue_time_us":17777,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":17752,"thread_start_us":279857,"threads_started":1,"wal-append.queue_time_us":1062065}
I20260709 14:53:12.421449  3545 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:12.424728  3537 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:12.330197  6764 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.288336  6978 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.428126  6748 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:12.428947  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:12.429652  3411 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.620s (SENT)
I20260709 14:53:12.430712  3536 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 14:53:12.431432  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.430991  6912 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:12.431943  7019 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:12.432473  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:12.472203  3542 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:12.431982  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:12.434252  7011 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 14:53:12.434692  7030 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.434989  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:12.435864  7036 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 14:53:12.439874  6961 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:12.345726  6864 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:12.442550  6495 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.007s	user 0.011s	sys 0.002s
I20260709 14:53:12.457306  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:12.387929  6786 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:53:12.444293  7013 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.406s	user 0.001s	sys 0.000s
I20260709 14:53:12.392145  6967 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:12.471042  3545 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:12.478902  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46975.7us
I20260709 14:53:12.479936  6414 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.484652  2681 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:12.492414  6824 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.000s
W20260709 14:53:12.496232  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66630.2us
W20260709 14:53:12.497541  7035 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 14:53:12.506785  3543 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:12.470430  7023 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.209s	user 0.000s	sys 0.005s
I20260709 14:53:12.562738  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.516783  3542 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:12.476805  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:12.571552  3537 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:12.484475  6764 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:12.473018  6990 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:12.531993  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.547474  6626 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.540686  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.596527  2609 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:12.543731  2759 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:12.472136  7032 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:12.544150  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.546032  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:12.481549  6995 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:12.552197  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:12.552381  3411 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.778s (SENT)
I20260709 14:53:12.598562  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.505746  6992 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.561515  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.561830  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:12.581159  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.582376  3537 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:12.544255  6693 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:12.538527  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 201327us
I20260709 14:53:12.586663  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.591614  2678 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:12.585529  2758 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:12.593628  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.597438  2753 raft_consensus.cc:2376] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:12.598912  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:12.560446  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.610855  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.611755  2680 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:12.455961  6958 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.521471  7030 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.606987  2612 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:12.612188  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:12.604224  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:12.663103  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50958.2us
I20260709 14:53:12.600788  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.600203  6892 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.556484  6824 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:12.635879  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:12.593858  6980 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.036s	user 0.014s	sys 0.001s
I20260709 14:53:12.559429  6414 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.655069  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.655766  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:12.607033  2686 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:12.660735  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:12.606891  2679 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:12.669705  2609 raft_consensus.cc:2368] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
W20260709 14:53:12.671026  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 148518us
W20260709 14:53:12.665400  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.665119  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:12.668236  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.667621  2758 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:12.674535  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 122170us
I20260709 14:53:12.675149  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:12.636987  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.684051  2678 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:12.614557  6990 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:12.662585  6675 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.515s	user 0.015s	sys 0.001s
W20260709 14:53:12.686718  2756 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.002s
I20260709 14:53:12.689486  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1110) took 1263 ms. Trace:
I20260709 14:53:12.639004  6985 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:12.693548  2755 raft_consensus.cc:2368] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12.
I20260709 14:53:12.659464  6957 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:12.694002  3538 rpcz_store.cc:276] 0709 14:53:11.426252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.435344 (+  9092us) service_pool.cc:224] Handling call
0709 14:53:11.446534 (+ 11190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:11.463139 (+ 16605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.463197 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:11.464023 (+   826us) log.cc:844] Serialized 26 byte log entry
0709 14:53:11.756825 (+292802us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:11.756878 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:11.756955 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:12.689170 (+932215us) raft_consensus.cc:1685] finished
0709 14:53:12.689200 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:12.689368 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32291,"raft.queue_time_us":17736,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":10675,"thread_start_us":292257,"threads_started":1,"wal-append.queue_time_us":458061}
I20260709 14:53:12.603925  6626 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.699579  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.700467  2756 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:12.701625  2678 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 14:53:12.670543  6992 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:12.605831  7032 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.634982  6693 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.687412  6613 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:12.676465  6892 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.683784  6984 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:12.715687  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.728134  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.675982  6980 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.789963  2756 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10. Re-sending same reply.
I20260709 14:53:12.792596  2605 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 14:53:12.795250  2605 pending_rounds.cc:85] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 4
I20260709 14:53:12.788461  2686 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:12.718489  7038 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 14:53:12.786954  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55737.9us
I20260709 14:53:12.797092  2605 pending_rounds.cc:107] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 14:53:12.798205  2605 raft_consensus.cc:2889] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:12.799324  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.811518  6693 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:12.812400  7016 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 14:53:12.812927  7016 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.698820  6675 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.814296  2752 raft_consensus.cc:2368] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:12.815539  6675 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:12.815902  7030 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.813236  6824 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:12.816210  6626 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.718631  7041 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:12.785998  7018 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 14:53:12.788017  7046 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
W20260709 14:53:12.790454  6905 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.016s	user 0.011s	sys 0.004s
W20260709 14:53:12.796286  7036 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 14:53:12.825850  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.836834  6984 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:12.788582  6895 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.788318  6992 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:12.828840  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.878650  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.877849  2607 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:12.831449  7041 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:12.877322  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 169126us
I20260709 14:53:12.887517  2606 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 14:53:12.842387  7002 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 14:53:12.887758  2686 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 14:53:12.889981  2686 pending_rounds.cc:85] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 6
I20260709 14:53:12.890246  2686 pending_rounds.cc:107] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 14:53:12.890525  2686 raft_consensus.cc:2889] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:12.891034  7054 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:12.891764  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.892418  7054 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 14:53:12.881863  6964 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
I20260709 14:53:12.894007  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.894790  2756 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:12.894960  7041 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.895905  2678 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:12.888242  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.896636  7032 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.897408  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.878216  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.899225  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.899773  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.899979  2754 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:12.900983  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.901789  2683 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:12.904338  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.839133  7039 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 14:53:12.892884  7054 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:12.899434  7048 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 14:53:12.902915  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.904820  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.891763  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.840207  6905 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:12.807359  6987 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.557s	user 0.015s	sys 0.022s
W20260709 14:53:12.906168  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 483973us
I20260709 14:53:12.928066  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:12.928570  2680 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 15. Ops: [13.8-13.8]
I20260709 14:53:12.885128  6895 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.922410  2679 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:12.911867  6824 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:12.843482  7016 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:12.838701  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:12.923759  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 480541us
I20260709 14:53:12.795292  6957 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 14:53:12.900954  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:12.943717  2754 raft_consensus.cc:2368] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:53:12.944821  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:12.945039  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 106461us
I20260709 14:53:12.945670  2754 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:12.933235  2686 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:12.915491  7038 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 14:53:12.894990  6626 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 14:53:12.943477  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.953859  2612 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:12.817982  7005 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 14:53:12.954568  2681 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:12.948576  2684 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:12.956394  3411 leader_election.cc:400] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:12.928992  6978 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:12.931241  6864 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:12.947674  6495 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 14:53:12.966084  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:12.965876  7054 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:12.967433  6626 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:12.968012  6626 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:12.968953  6626 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.948200  6957 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:12.971025  7024 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:12.971585  7024 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:12.971946  7024 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:12.972060  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 765) took 1012 ms. Trace:
I20260709 14:53:12.972595  7005 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:12.972503  2752 rpcz_store.cc:276] 0709 14:53:11.959761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.975669 (+ 15908us) service_pool.cc:224] Handling call
0709 14:53:12.971765 (+996096us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400570154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:12.971984 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816105728}
I20260709 14:53:12.973173  7005 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:12.974104  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:12.975554  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1265) took 2392 ms (client timeout 2033 ms). Trace:
W20260709 14:53:12.975991  2759 rpcz_store.cc:269] 0709 14:53:10.582677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.629917 (+ 47240us) service_pool.cc:224] Handling call
0709 14:53:12.975316 (+2345399us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:12.975476 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4231810432}
I20260709 14:53:12.947269  6824 consensus_queue.cc:1243] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:12.977094  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:12.977232  6824 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:12.977804  2759 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:12.979476  7006 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:12.980013  7006 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:12.981089  7006 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.974306  7005 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:12.984136  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:12.984344  7005 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:12.885268  6992 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:12.970526  6626 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:12.963572  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:12.967532  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:12.992447  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:13.045360  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53056.7us
I20260709 14:53:13.048647  2683 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:12.994434  2756 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:13.051069  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 67059.4us
I20260709 14:53:12.978691  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:12.989452  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.053006  2760 raft_consensus.cc:2376] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:13.054077  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.056721  2754 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:13.057747  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.058389  2754 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:13.047250  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:13.051923  2606 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:12.982743  7006 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:13.074229  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:13.074950  6877 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:13.075805  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.076467  2753 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:13.077162  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.071926  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:13.080622  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:12.967485  7054 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:13.081915  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.082633  2752 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:13.082984  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.083909  2759 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:12.930186  6990 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:13.120980  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:13.053298  6985 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:13.122782  2754 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:13.070190  6978 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:13.127055  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:13.126861  2683 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:13.127806  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.130496  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.117586  6878 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.132434  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:13.136636  7049 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 14:53:13.136349  6878 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:13.137496  7053 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 14:53:13.138309  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.140340  2681 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 14:53:13.140509  7001 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.068s	user 0.013s	sys 0.000s
I20260709 14:53:13.141012  7001 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13.
I20260709 14:53:13.142724  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.136567  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.144544  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.145560  2607 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:13.116977  6414 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:13.147185  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.152079  6977 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.150727  2759 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:13.152809  6977 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:13.156188  6977 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:13.157488  6878 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:13.160770  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.086145  7041 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 14:53:13.164295  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:13.164572  7041 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:13.146520  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:13.165851  2607 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:13.166972  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 115103us
I20260709 14:53:13.168198  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:13.170608  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:13.181945  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.182544  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.182108  2678 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:13.198475  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.181471  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.125061  7052 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:13.202127  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.209359  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.214272  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:13.214658  2686 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:13.188571  2685 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:13.201493  2754 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:13.202286  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.216060  2759 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:13.216343  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.218030  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.219113  2754 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:53:13.220115  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.221879  7047 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.188364  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:13.225025  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.226361  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.227537  2753 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:13.227584  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:13.187314  6987 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:13.233871  2759 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:13.186115  7001 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.243498  2754 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:13.206278  7038 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 14:53:13.184341  6978 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:13.255823  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.257656  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:13.260941  2759 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:13.260988  6748 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 14:53:13.275385  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49256.7us
I20260709 14:53:13.222585  7047 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:13.246555  6980 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:13.319532  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1271) took 1518 ms. Trace:
I20260709 14:53:13.271467  6467 raft_consensus.cc:2764] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 14:53:13.277810  6977 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:13.280552  2607 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:13.280787  2686 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:13.310066  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:13.263269  6414 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:13.315141  2753 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:13.225154  7052 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:13.327457  3411 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.078s (SENT)
I20260709 14:53:13.278870  6905 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:13.329674  2759 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:13.316236  6964 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.309664  6748 consensus_queue.cc:1243] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 14:53:13.315166  6995 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.004s
I20260709 14:53:13.320205  3540 rpcz_store.cc:276] 0709 14:53:11.801234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.819820 (+ 18586us) service_pool.cc:224] Handling call
0709 14:53:11.842706 (+ 22886us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:11.853416 (+ 10710us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:12.115938 (+262522us) log.cc:844] Serialized 14 byte log entry
0709 14:53:12.279723 (+163785us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:12.335859 (+ 56136us) log.cc:844] Serialized 52 byte log entry
0709 14:53:12.336120 (+   261us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:12.336477 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 14:53:12.398054 (+ 61577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:12.398167 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:13.319157 (+920990us) raft_consensus.cc:1685] finished
0709 14:53:13.319193 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:13.319439 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145368,"prepare.queue_time_us":339294,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":10215,"raft.queue_time_us":297669,"raft.run_cpu_time_us":682,"raft.run_wall_time_us":63397,"thread_start_us":304983,"threads_started":2,"wal-append.queue_time_us":361604}
I20260709 14:53:13.385798  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:13.335266  2753 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:13.307142  6820 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:13.246886  6985 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:13.371785  2754 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 14:53:13.266417  7001 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:13.372203  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:13.305681  6987 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 14:53:13.380239  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:13.391660  2754 pending_rounds.cc:85] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:13.402002  2754 pending_rounds.cc:107] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 14:53:13.402311  2754 raft_consensus.cc:2889] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:13.368994  6986 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.199s	user 0.010s	sys 0.004s
I20260709 14:53:13.373764  6876 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:13.405382  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.335634  6957 raft_consensus.cc:2764] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 14:53:13.418503  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1245) took 5644 ms. Trace:
I20260709 14:53:13.318537  7041 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:13.374135  6895 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:13.440754  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.441673  7052 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:13.443780  7061 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 14:53:13.407802  6868 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 14:53:13.460405  7001 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:13.443588  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:13.437798  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:13.405977  6495 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.394675  6995 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:13.419304  2757 rpcz_store.cc:276] 0709 14:53:07.762631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.492309 (+3729678us) service_pool.cc:224] Handling call
0709 14:53:11.555373 (+ 63064us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:11.556101 (+   728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.556149 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:11.557056 (+   907us) log.cc:844] Serialized 52 byte log entry
0709 14:53:11.755858 (+198802us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:12.495546 (+739688us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18003ba100. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:13.401827 (+906281us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b18003ba100. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf
0709 14:53:13.402033 (+   206us) log.cc:844] Serialized 14 byte log entry
0709 14:53:13.406684 (+  4651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:13.406801 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:13.406907 (+   106us) raft_consensus.cc:1685] finished
0709 14:53:13.406927 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:13.407148 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7394,"prepare.queue_time_us":283060,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":5163,"raft.queue_time_us":221311,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11462,"spinlock_wait_cycles":45080320,"thread_start_us":229370,"threads_started":2,"wal-append.queue_time_us":511242}
I20260709 14:53:13.442610  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:13.506160  2607 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:13.403002  6985 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:13.367961  6613 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 14:53:13.407044  6905 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:13.367641  6414 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:13.368762  6977 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:13.464190  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84070.5us
W20260709 14:53:13.507094  6971 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.982s	user 0.009s	sys 0.010s
I20260709 14:53:13.387137  6964 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:13.508126  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 180697us
I20260709 14:53:13.435541  7039 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:13.422616  6986 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:13.439421  6992 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:13.478250  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:13.539459  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61303.1us
I20260709 14:53:13.399915  6820 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:13.468673  6797 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.521260  3412 leader_election.cc:400] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
W20260709 14:53:13.524061  6997 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.224s	user 0.012s	sys 0.003s
I20260709 14:53:13.548637  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:13.472827  2681 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:13.531813  3411 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.922s (SENT)
I20260709 14:53:13.551290  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:13.476912  6957 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:13.554668  6957 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:13.555517  6957 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:13.560647  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1253) took 1012 ms. Trace:
I20260709 14:53:13.561170  2685 rpcz_store.cc:276] 0709 14:53:12.548146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.548409 (+   263us) service_pool.cc:224] Handling call
0709 14:53:13.560424 (+1012015us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:13.560573 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332790528}
W20260709 14:53:13.536940  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 408942us
I20260709 14:53:13.568027  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:13.540676  6905 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:13.514289  6985 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:13.568874  2685 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:53:13.540760  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 130918us
I20260709 14:53:13.575879  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:13.581065  3538 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:13.462632  6978 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:13.570556  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:13.517549  6495 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:13.564113  6959 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.597s	user 0.015s	sys 0.003s
I20260709 14:53:13.546787  7001 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:13.587129  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.606961  2751 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:13.614265  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:13.617435  2607 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:13.619194  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:13.619900  2683 pending_rounds.cc:85] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 3
I20260709 14:53:13.620327  2683 pending_rounds.cc:107] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 14:53:13.620201  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:13.620818  2683 raft_consensus.cc:2889] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:13.624361  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 170827us
I20260709 14:53:13.566855  7039 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:13.539628  6971 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:13.628049  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57646.5us
I20260709 14:53:13.628119  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:13.627465  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52252.1us
I20260709 14:53:13.629272  2757 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 11 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:13.630609  3540 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:53:13.631091  3540 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 14:53:13.631403  3540 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:13.637305  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:13.638126  2753 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:13.646449  2753 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:13.647809  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:13.577976  6986 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.655778  6986 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:13.658295  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:13.629081  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:13.582607  6980 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:13.659824  2754 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:53:13.571815  6957 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:13.657514  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71432.6us
I20260709 14:53:13.666697  3538 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:13.660908  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:13.684351  2751 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:13.685360  3537 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:13.692718  2760 raft_consensus.cc:2368] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:53:13.560351  6997 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:13.563282  6797 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:13.647344  6748 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:13.708839  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44984 (request call id 16) took 1664 ms. Trace:
I20260709 14:53:13.709350  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.713253  2757 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:13.714893  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:13.712070  2682 rpcz_store.cc:276] 0709 14:53:12.035287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.064508 (+ 29221us) service_pool.cc:224] Handling call
0709 14:53:13.699137 (+1634629us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:13.699314 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116598656}
I20260709 14:53:13.716250  2754 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 14:53:13.699818  7059 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 14:53:13.623931  6959 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:13.699656  7015 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:53:13.617923  6864 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 14:53:13.760565  3537 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 14:53:13.773070  3537 pending_rounds.cc:85] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:53:13.773413  3537 pending_rounds.cc:107] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 14:53:13.773767  3537 raft_consensus.cc:2889] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:13.760843  2606 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:13.751809  2757 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 14:53:13.709415  7063 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.307s	user 0.002s	sys 0.000s
I20260709 14:53:13.705204  7077 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:13.699579  7078 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 14:53:13.779026  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 247238us
I20260709 14:53:13.647604  6990 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 14:53:13.718539  6748 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:13.695251  6980 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:13.845562  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 216537us
I20260709 14:53:13.846614  2577 leader_election.cc:400] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
I20260709 14:53:13.847016  2577 leader_election.cc:403] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:53:13.841823  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:13.854076  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 216918us
I20260709 14:53:13.704330  6992 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 14:53:13.759752  7083 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:13.699359  7072 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.186s
I20260709 14:53:13.833967  6748 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:13.859118  7006 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.004s
I20260709 14:53:13.885159  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:13.886665  3537 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:13.895027  7082 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:13.897881  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:13.903826  2752 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:53:13.904708  3537 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:13.843258  6972 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 14:53:13.821593  7081 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260709 14:53:13.844841  6905 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:13.926390  7082 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:13.908064  6964 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:13.945660  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:13.914018  6986 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:13.933772  7024 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.961s	user 0.014s	sys 0.002s
I20260709 14:53:13.947460  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:13.950174  6977 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:13.952160  7010 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.871s	user 0.004s	sys 0.025s
I20260709 14:53:13.952672  7069 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:13.954799  3537 raft_consensus.cc:2376] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13. Re-sending same reply.
I20260709 14:53:13.844587  7077 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:13.933310  7005 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.948s	user 0.016s	sys 0.000s
W20260709 14:53:13.961161  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105816us
I20260709 14:53:13.961189  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 751) took 1347 ms. Trace:
W20260709 14:53:13.945042  7066 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
I20260709 14:53:13.970378  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:13.958199  6626 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.005s
I20260709 14:53:13.922487  7001 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:13.945125  6414 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:13.995590  3537 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:13.954583  6820 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:13.953936  7081 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:14.031910  7081 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:13.989482  2611 rpcz_store.cc:276] 0709 14:53:12.614208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.696060 (+ 81852us) service_pool.cc:224] Handling call
0709 14:53:12.729354 (+ 33294us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:12.792167 (+ 62813us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:53:12.871770 (+ 79603us) log.cc:844] Serialized 14 byte log entry
0709 14:53:13.077228 (+205458us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:13.167747 (+ 90519us) log.cc:844] Serialized 52 byte log entry
0709 14:53:13.168168 (+   421us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:13.168210 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:13.168293 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:13.960920 (+792627us) raft_consensus.cc:1685] finished
0709 14:53:13.960945 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:13.961112 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52528,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":204,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":92599,"thread_start_us":141760,"threads_started":1,"wal-append.queue_time_us":335250}
W20260709 14:53:14.036872  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 376016us
I20260709 14:53:14.033077  7001 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:13.866856  7083 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:13.966032  6992 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:14.055233  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:13.967548  7069 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:13.989871  6895 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:14.081486  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:14.088433  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 398177us
I20260709 14:53:14.089190  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.035014  7005 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Leader election lost for term 10. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11.
W20260709 14:53:14.092276  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 472156us
I20260709 14:53:14.041725  6495 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:14.104261  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 257802us
I20260709 14:53:14.106366  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:13.966889  7024 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.109980  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:14.114883  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.115921  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.092717  7069 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:14.119277  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:14.119544  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.092041  6895 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:14.123970  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.125828  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.126999  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.129948  3541 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:14.132025  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:14.133693  3545 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:14.134526  6626 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:14.135290  6626 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:14.135990  7003 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.136407  6626 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.142304  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1254) took 3777 ms. Trace:
I20260709 14:53:14.143392  6980 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:14.144315  7089 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:14.145891  7108 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.146486  7108 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.148492  3542 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:14.136698  7003 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.137142  7083 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.091692  7085 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:14.107722  7076 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:14.142891  3547 rpcz_store.cc:276] 0709 14:53:10.355241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:10.385124 (+ 29883us) service_pool.cc:224] Handling call
0709 14:53:10.385712 (+   588us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:11.415964 (+1030252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:11.416021 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:11.431743 (+ 15722us) log.cc:844] Serialized 52 byte log entry
0709 14:53:11.805794 (+374051us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:11.805834 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:12.441026 (+635192us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54002bb454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:12.441086 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:13.872066 (+1430980us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:14.132355 (+260289us) raft_consensus.cc:1685] finished
0709 14:53:14.132383 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.132558 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8485,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":12819,"spinlock_wait_cycles":461408896,"thread_start_us":1154336,"threads_started":2,"wal-append.queue_time_us":510685}
I20260709 14:53:14.120101  6977 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:14.174299  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:14.184217  2757 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:14.117687  6797 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.095219  6986 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 14:53:14.134164  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:14.092978  6992 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.206324  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:14.132102  6895 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:14.193801  7054 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.112s	user 0.007s	sys 0.010s
I20260709 14:53:14.112557  6997 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.196674  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.196671  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.122555  7005 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 14:53:14.201290  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 67201.6us
I20260709 14:53:14.202600  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.203567  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:14.203876  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1257) took 1565 ms. Trace:
W20260709 14:53:14.186568  6786 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.637s	user 0.011s	sys 0.007s
I20260709 14:53:14.131244  7087 raft_consensus.cc:2764] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:53:14.209875  2757 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:14.220995  2680 rpcz_store.cc:276] 0709 14:53:12.639094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.668087 (+ 28993us) service_pool.cc:224] Handling call
0709 14:53:12.668409 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:12.929053 (+260644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.203557 (+1274504us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:14.203806 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597588224}
I20260709 14:53:14.221107  2754 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:14.231338  2612 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:14.206324  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.224016  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.234018  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.225039  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.235509  3545 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:14.225014  3541 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:14.180578  7089 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:14.239574  2758 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:14.180212  7085 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:14.169822  7099 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:14.244290  7090 raft_consensus.cc:2764] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:53:14.219726  6977 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:14.249650  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.173580  6877 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 14:53:14.184487  6980 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 14:53:14.262705  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:14.272389  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.001s	sys 0.024s
W20260709 14:53:14.272994  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.001s	sys 0.025s
W20260709 14:53:14.273403  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 654261us
I20260709 14:53:14.230388  6797 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:14.282366  3412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.860s (SENT) [suppressed 28 similar messages]
I20260709 14:53:14.284029  2754 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:14.290371  2605 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:14.258957  6786 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:53:14.274004  6876 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.855s	user 0.013s	sys 0.003s
I20260709 14:53:14.247839  7085 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:14.244997  7105 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:14.273674  6980 consensus_queue.cc:1243] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:14.305882  2677 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:14.128171  7082 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.309612  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:14.320686  2754 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:14.282223  6977 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:14.272622  7005 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 14:53:14.200719  7076 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:14.252559  6868 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 14:53:14.335985  6992 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:14.336663  7109 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.337303  7109 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:14.339044  7111 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 14:53:14.343160  7103 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
W20260709 14:53:14.349793  7079 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
W20260709 14:53:14.352077  7086 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.285s	user 0.000s	sys 0.002s
I20260709 14:53:14.295807  7106 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.326961  3537 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
W20260709 14:53:14.327212  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44955.8us
I20260709 14:53:14.379392  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.328058  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:14.380084  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.326254  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:14.343210  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1277) took 1412 ms. Trace:
I20260709 14:53:14.320119  7019 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.299834  7102 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:14.380837  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52921.9us
I20260709 14:53:14.381731  2759 rpcz_store.cc:276] 0709 14:53:12.931616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.369499 (+437883us) service_pool.cc:224] Handling call
0709 14:53:13.400665 (+ 31166us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:13.407555 (+  6890us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:13.577336 (+169781us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.154751 (+577415us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:14.188346 (+ 33595us) log.cc:844] Serialized 52 byte log entry
0709 14:53:14.188840 (+   494us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:14.189375 (+   535us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.197250 (+  7875us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.197487 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.197611 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:14.342764 (+145153us) raft_consensus.cc:1685] finished
0709 14:53:14.342808 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.343100 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43637,"prepare.queue_time_us":710795,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":126,"raft.queue_time_us":49704,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":11811,"spinlock_wait_cycles":167939968,"thread_start_us":588405,"threads_started":2,"wal-append.queue_time_us":674680}
I20260709 14:53:14.389956  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:14.387607  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:14.388067  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61937.9us
I20260709 14:53:14.391188  2680 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:14.291154  6797 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.381194  2754 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:14.392612  7105 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:14.381202  7106 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.394227  7024 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:14.395640  7108 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:14.396684  7100 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.397179  7106 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:14.397342  7100 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.390547  3537 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:14.393241  7105 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:14.393232  7109 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.415616  2759 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:14.405820  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.376317  7089 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:14.419456  7100 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:14.420727  7115 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:14.421924  7115 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:14.428821  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 775) took 1094 ms. Trace:
I20260709 14:53:14.429698  2681 rpcz_store.cc:276] 0709 14:53:13.334693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.578524 (+243831us) service_pool.cc:224] Handling call
0709 14:53:13.624676 (+ 46152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:13.624941 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:13.719352 (+ 94411us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.146280 (+426928us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b180038c1a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:14.428327 (+282047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.428397 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:14.428464 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.428543 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.428728 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127951,"prepare.queue_time_us":94111,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":513,"spinlock_wait_cycles":63520512,"thread_start_us":223542,"threads_started":1}
I20260709 14:53:14.433024  7115 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.433735  7076 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.424085  7106 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:14.435506  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.437281  2681 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:14.431077  6980 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 14:53:14.446085  7058 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.121s	user 0.003s	sys 0.000s
I20260709 14:53:14.446192  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.449682  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.450774  3537 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:14.455899  6996 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:14.456967  6996 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.460721  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.463011  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.463503  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1356) took 1511 ms. Trace:
I20260709 14:53:14.463912  2608 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:14.464000  2606 rpcz_store.cc:276] 0709 14:53:12.952489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.133513 (+181024us) service_pool.cc:224] Handling call
0709 14:53:14.463211 (+1329698us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54002cdb54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:14.463402 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387616512}
I20260709 14:53:14.465047  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.471563  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:14.475884  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.481949  2680 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:14.484364  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:14.476796  3537 raft_consensus.cc:2376] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13. Re-sending same reply.
I20260709 14:53:14.488191  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:14.489519  2680 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:53:14.490386  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.491469  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.492563  2608 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:14.493950  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.356740  7081 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.411144  7019 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:14.498771  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1425) took 1960 ms. Trace:
I20260709 14:53:14.402122  7102 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.483489  6984 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:14.491537  2606 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:14.499886  2603 rpcz_store.cc:276] 0709 14:53:12.538568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.538771 (+   203us) service_pool.cc:224] Handling call
0709 14:53:12.563873 (+ 25102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:12.564324 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:12.564355 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:12.564919 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 14:53:13.044802 (+479883us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:13.044868 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:13.045122 (+   254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:14.438251 (+1393129us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b2400467a38. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:53:14.488705 (+ 50454us) raft_consensus.cc:1685] finished
0709 14:53:14.488730 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.498688 (+  9958us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2474,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":16116,"spinlock_wait_cycles":37819648,"thread_start_us":389389,"threads_started":1,"wal-append.queue_time_us":608922}
W20260709 14:53:14.519003  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:14.501325  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.503168  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:14.523748  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.524672  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.525285  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.507526  7081 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:14.511734  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.497581  6815 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:14.539631  6815 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:14.515415  2681 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:14.451920  6977 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.497845  7024 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 14:53:14.542748  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:14.499168  7105 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:14.549494  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:14.550648  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.508355  7121 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:14.555148  3537 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:14.560920  7121 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:14.535018  7014 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:14.575829  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.578199  6996 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:14.578624  7014 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:14.579253  7014 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:14.580385  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1385) took 1683 ms. Trace:
I20260709 14:53:14.581113  2756 rpcz_store.cc:276] 0709 14:53:12.896767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.896928 (+   161us) service_pool.cc:224] Handling call
0709 14:53:12.994229 (+ 97301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:13.051216 (+ 56987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.580099 (+1528883us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b5400570150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:14.580304 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222854528}
I20260709 14:53:14.594502  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.517540  7069 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.685061  7069 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:14.692232  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:14.703552  2756 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:14.541239  6815 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.722059  2611 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:14.719249  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.729847  2608 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:14.751361  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:14.752799  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1366) took 1428 ms. Trace:
I20260709 14:53:14.753227  2603 rpcz_store.cc:276] 0709 14:53:13.324914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.596562 (+1271648us) service_pool.cc:224] Handling call
0709 14:53:14.752703 (+156141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:14.745285  6748 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.853s	user 0.015s	sys 0.000s
I20260709 14:53:14.764633  6797 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:14.765050  6748 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:14.767421  2607 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:14.770046  6748 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:14.775557  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:14.777757  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.692250  7109 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.717732  6989 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.746086  6786 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.807993  6786 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.813180  6967 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:14.813925  6967 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:14.815781  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1353) took 1864 ms (client timeout 1850 ms). Trace:
I20260709 14:53:14.816231  6967 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:14.816300  2612 rpcz_store.cc:269] 0709 14:53:12.951511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.066553 (+115042us) service_pool.cc:224] Handling call
0709 14:53:14.815538 (+1748985us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540042e0d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:14.815697 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2658593664}
I20260709 14:53:14.818013  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.824561  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.825932  3547 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:14.801589  7055 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.638s	user 0.010s	sys 0.006s
I20260709 14:53:14.745824  7100 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:14.807132  7067 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.356s	user 0.003s	sys 0.000s
I20260709 14:53:14.788165  6995 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:14.863119  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:14.875595  2608 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 14:53:14.849416  7069 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:14.857470  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.884676  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:14.848340  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:14.871064  6984 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:14.798614  7109 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:14.912735  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1216) took 1580 ms. Trace:
W20260709 14:53:14.838933  3411 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.932s (SENT)
I20260709 14:53:14.850533  6989 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.921336  7019 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:14.922041  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:14.939890  2753 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:14.923022  2608 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 14:53:14.913889  2686 rpcz_store.cc:276] 0709 14:53:13.333047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.333390 (+   343us) service_pool.cc:224] Handling call
0709 14:53:13.333766 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:13.569562 (+235796us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:13.587144 (+ 17582us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.259696 (+672552us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.259776 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:14.259835 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.912326 (+652491us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:14.912407 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.912642 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71224,"prepare.queue_time_us":88591,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":79,"raft.queue_time_us":285439,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":8956,"spinlock_wait_cycles":541618048,"thread_start_us":563809,"threads_started":2,"wal-append.queue_time_us":344683}
I20260709 14:53:14.926889  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:14.927567  6748 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 14:53:14.929059  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1428) took 1532 ms. Trace:
I20260709 14:53:14.952623  2603 rpcz_store.cc:276] 0709 14:53:13.397597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.894589 (+1496992us) service_pool.cc:224] Handling call
0709 14:53:14.928484 (+ 33895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.928690 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.928718 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.928773 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:14.928813 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.928866 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.928990 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:14.946137  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:14.939968  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:14.955724  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1360) took 1863 ms. Trace:
I20260709 14:53:14.956298  2608 rpcz_store.cc:276] 0709 14:53:13.091997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.882486 (+1790489us) service_pool.cc:224] Handling call
0709 14:53:14.922698 (+ 40212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:14.955395 (+ 32697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.955649 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:14.961351  2612 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:14.962055  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:14.964918  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:14.858255  7100 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:14.968230  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1430) took 1556 ms. Trace:
I20260709 14:53:14.975621  2603 rpcz_store.cc:276] 0709 14:53:13.398294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.953687 (+1555393us) service_pool.cc:224] Handling call
0709 14:53:14.954082 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.954302 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.954332 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.954378 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:14.954419 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.954473 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.954600 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:14.969961  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1354) took 2018 ms. Trace:
I20260709 14:53:14.972757  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1431) took 1574 ms. Trace:
I20260709 14:53:14.991708  2609 rpcz_store.cc:276] 0709 14:53:13.398592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.954165 (+1555573us) service_pool.cc:224] Handling call
0709 14:53:14.972031 (+ 17866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.972267 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.972303 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.972365 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:14.972428 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.972506 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:14.972673 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:14.994354  2612 rpcz_store.cc:276] 0709 14:53:12.951834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.891946 (+1940112us) service_pool.cc:224] Handling call
0709 14:53:14.943100 (+ 51154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.962462 (+ 19362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:14.969850 (+  7388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:14.957109  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:15.009524  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:14.999697  2603 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:15.004726  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:15.019752  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1432) took 1620 ms. Trace:
I20260709 14:53:15.021488  2603 rpcz_store.cc:276] 0709 14:53:13.398908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.977069 (+1578161us) service_pool.cc:224] Handling call
0709 14:53:14.977573 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.019349 (+ 41776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.019671 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:15.018435  3545 raft_consensus.cc:2376] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14. Re-sending same reply.
I20260709 14:53:15.024204  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:15.020289  2686 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:14.895717  7069 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:14.850566  7121 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:15.028373  2603 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:14.969105  6786 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:15.032207  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 193363us
I20260709 14:53:15.048146  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1388) took 2149 ms. Trace:
I20260709 14:53:15.145824  2752 rpcz_store.cc:276] 0709 14:53:12.898348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.179518 (+281170us) service_pool.cc:224] Handling call
0709 14:53:15.047704 (+1868186us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540040e7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.048032 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605166208}
I20260709 14:53:15.048772  2757 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 14:53:15.120846  6968 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:15.147308  2757 pending_rounds.cc:85] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 5
I20260709 14:53:15.147655  2757 pending_rounds.cc:107] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 14:53:15.148025  2757 raft_consensus.cc:2889] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:15.055655  7114 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:14.958900  7103 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 14:53:14.872752  6995 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:15.006647  6892 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:15.058370  6997 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:15.091197  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:15.116573  2683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.049s	sys 0.140s
W20260709 14:53:15.090664  6905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.070s	sys 0.111s
W20260709 14:53:15.121762  7085 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.790s	user 0.005s	sys 0.011s
W20260709 14:53:15.127806  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69211.2us
I20260709 14:53:15.176828  6997 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:15.177342  3536 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:15.178144  6997 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:15.179095  3536 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:15.125166  6989 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:15.147392  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:15.198232  2686 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:14.986176  7019 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:53:15.125119  7089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.009s	sys 0.057s
W20260709 14:53:15.130257  7082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.037s	sys 0.035s
W20260709 14:53:15.168530  2683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.049s	sys 0.143s
W20260709 14:53:15.175483  6905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.070s	sys 0.112s
I20260709 14:53:15.166113  7114 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:15.194564  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:15.169624  2753 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:15.127794  6986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.022s	sys 0.051s
W20260709 14:53:15.233472  6986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.022s	sys 0.052s
W20260709 14:53:15.180747  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 766) took 3220 ms (client timeout 1662 ms). Trace:
W20260709 14:53:15.234934  2755 rpcz_store.cc:269] 0709 14:53:11.960177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.977000 (+ 16823us) service_pool.cc:224] Handling call
0709 14:53:15.180431 (+3203431us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54002765d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.180655 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537957632}
W20260709 14:53:15.202353  7089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.009s	sys 0.058s
I20260709 14:53:15.180181  6747 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:15.202164  7019 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:15.202718  7082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.037s	sys 0.035s
W20260709 14:53:15.230993  6985 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.612s	user 0.017s	sys 0.002s
I20260709 14:53:15.231024  7114 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:15.247551  7069 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:15.245421  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59898.9us
I20260709 14:53:15.261966  2686 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:15.157224  6968 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:15.245154  7019 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:15.268555  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:15.290609  3536 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:15.197600  7085 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:15.269523  7025 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.244974  6747 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:15.268555  6986 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:15.270099  7041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.045s	sys 0.030s
I20260709 14:53:15.252652  6985 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:15.286953  6675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.013s	sys 0.014s
I20260709 14:53:15.285610  7109 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:15.333244  3538 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:15.325779  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:15.335031  6747 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:15.337453  6985 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:15.311645  7025 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:15.348330  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1196) took 3722 ms (client timeout 2865 ms). Trace:
W20260709 14:53:15.324014  7041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.896s	user 0.046s	sys 0.031s
I20260709 14:53:15.351462  7025 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:15.349049  2684 rpcz_store.cc:269] 0709 14:53:11.625801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:11.626703 (+   902us) service_pool.cc:224] Handling call
0709 14:53:15.347956 (+3721253us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002b3254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.348214 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2664375168}
I20260709 14:53:15.306468  7100 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:15.325892  6986 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:15.376809  7043 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.708s	user 0.013s	sys 0.012s
I20260709 14:53:15.377614  6986 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:15.381194  7065 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.925s	user 0.000s	sys 0.003s
I20260709 14:53:15.383136  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 769) took 2336 ms. Trace:
W20260709 14:53:15.383962  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1361) took 2249 ms (client timeout 2800 ms). Trace:
W20260709 14:53:15.384694  2605 rpcz_store.cc:269] 0709 14:53:13.134503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.163808 (+ 29305us) service_pool.cc:224] Handling call
0709 14:53:15.383677 (+2219869us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.383833 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1991464832}
I20260709 14:53:15.360703  2604 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 14:53:15.371098  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1252) took 2823 ms (client timeout 1705 ms). Trace:
W20260709 14:53:15.387746  2677 rpcz_store.cc:269] 0709 14:53:12.547733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.557829 (+ 10096us) service_pool.cc:224] Handling call
0709 14:53:15.370808 (+2812979us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.370997 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3902916480}
I20260709 14:53:15.386669  2605 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:15.383684  2679 rpcz_store.cc:276] 0709 14:53:13.046997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.047207 (+   210us) service_pool.cc:224] Handling call
0709 14:53:13.076294 (+ 29087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:13.119263 (+ 42969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:13.119299 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:13.124546 (+  5247us) log.cc:844] Serialized 26 byte log entry
0709 14:53:13.327408 (+202862us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:13.327464 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:13.327537 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:15.379528 (+2051991us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:15.382604 (+  3076us) raft_consensus.cc:1685] finished
0709 14:53:15.382643 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:15.383041 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41242,"raft.queue_time_us":43879,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":7839,"spinlock_wait_cycles":33803904,"thread_start_us":171555,"threads_started":1,"wal-append.queue_time_us":300390}
W20260709 14:53:15.329064  6675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.015s	sys 0.014s
W20260709 14:53:15.353461  7014 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.774s	user 0.015s	sys 0.000s
W20260709 14:53:15.397563  3540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.000s	sys 0.019s
I20260709 14:53:15.401160  3536 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:15.397774  7014 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:15.430068  3540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.000s	sys 0.020s
I20260709 14:53:15.353040  6995 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:15.362645  6959 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:15.418146  2726 connection.cc:256] RPC call timeout handler was delayed by -1.7556568800000605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 14:53:15.421330  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.015s	sys 0.005s
W20260709 14:53:15.417677  6895 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.186s	user 0.009s	sys 0.010s
I20260709 14:53:15.432628  7014 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:15.433395  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.016s	sys 0.005s
W20260709 14:53:15.432667  2726 connection.cc:256] RPC call timeout handler was delayed by -1.6512568800000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 14:53:15.434017  2726 connection.cc:256] RPC call timeout handler was delayed by -1.6251568800000769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 14:53:15.434612  2726 connection.cc:256] RPC call timeout handler was delayed by -1.5333568800000421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
W20260709 14:53:15.435195  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3866568800000323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 14:53:15.423563  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.004s	sys 0.016s
I20260709 14:53:15.419668  6824 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.420079  7100 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:15.436232  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.004s	sys 0.016s
W20260709 14:53:15.439786  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3812568800000236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 14:53:15.443852  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.023s	sys 0.070s
W20260709 14:53:15.443778  2726 connection.cc:256] RPC call timeout handler was delayed by -1.372256880000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 14:53:15.444267  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.023s	sys 0.070s
W20260709 14:53:15.444347  2726 connection.cc:256] RPC call timeout handler was delayed by -1.362356879999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 14:53:15.444662  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.03238e+06us
W20260709 14:53:15.444954  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1395627470000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 14:53:15.445690  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:15.423280  7085 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:15.436024  7115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.014s	sys 0.006s
I20260709 14:53:15.434504  7078 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:15.446334  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:15.451197  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:15.457783  7115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.014s	sys 0.006s
I20260709 14:53:15.480662  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:15.452769  6824 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:15.433039  6895 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:15.451752  7100 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:15.451064  6995 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 14:53:15.481590  3542 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.015s	sys 0.021s
W20260709 14:53:15.524679  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.013s	sys 0.014s
W20260709 14:53:15.572535  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.013s	sys 0.022s
W20260709 14:53:15.522732  3543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.004s	sys 0.027s
W20260709 14:53:15.577778  3543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.012s	sys 0.029s
W20260709 14:53:15.526355  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.003s	sys 0.016s
I20260709 14:53:15.581604  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1122) took 1432 ms. Trace:
I20260709 14:53:15.453939  6892 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:15.581723  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.005s	sys 0.017s
W20260709 14:53:15.526794  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.007s	sys 0.015s
W20260709 14:53:15.592512  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.010s	sys 0.015s
W20260709 14:53:15.526005  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.001s	sys 0.031s
W20260709 14:53:15.594508  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.002s	sys 0.036s
W20260709 14:53:15.526666  2651 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.011s	sys 0.008s
W20260709 14:53:15.601393  2651 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.013s	sys 0.010s
W20260709 14:53:15.592672  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 665836us
W20260709 14:53:15.602039  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 758639us
W20260709 14:53:15.572535  3542 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.018s	sys 0.021s
I20260709 14:53:15.582218  3537 rpcz_store.cc:276] 0709 14:53:14.149594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.559314 (+409720us) service_pool.cc:224] Handling call
0709 14:53:14.559810 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.560166 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:14.589203 (+ 29037us) log.cc:844] Serialized 14 byte log entry
0709 14:53:15.581083 (+991880us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:15.581182 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:15.581256 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.581334 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:15.581524 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29216,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":22995,"raft.queue_time_us":7761,"thread_start_us":983502,"threads_started":1}
W20260709 14:53:15.632432  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:15.637475  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:15.638692  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:15.615703  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1276) took 2041 ms. Trace:
I20260709 14:53:15.492542  6786 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:15.526532  7045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.011s	sys 0.015s
I20260709 14:53:15.491719  7014 raft_consensus.cc:2764] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 14:53:15.497586  6968 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:15.664054  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:15.596134  7039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.121s	user 0.016s	sys 0.049s
I20260709 14:53:15.674733  7072 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 14:53:15.681926  7131 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 14:53:15.683346  7072 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:53:15.683970  7129 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 14:53:15.686110  7127 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 14:53:15.687268  7100 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:15.688302  7045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.012s	sys 0.021s
W20260709 14:53:15.688609  7128 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 14:53:15.605952  7126 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:15.574721  7115 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.573671  7099 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.607084  6959 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.653908  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:15.641641  3546 rpcz_store.cc:276] 0709 14:53:13.574200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.581792 (+  7592us) service_pool.cc:224] Handling call
0709 14:53:13.627324 (+ 45532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:13.627581 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:13.678089 (+ 50508us) log.cc:844] Serialized 14 byte log entry
0709 14:53:14.832560 (+1154471us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18003a5520. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:15.615104 (+782544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:15.615192 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:15.615275 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.615402 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:15.615611 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20471,"prepare.queue_time_us":43922,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":5633,"raft.queue_time_us":20735,"spinlock_wait_cycles":7615232,"thread_start_us":761480,"threads_started":1}
I20260709 14:53:15.644896  6786 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:15.661594  6967 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:15.725384  7014 raft_consensus.cc:2764] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 14:53:15.725595  6967 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:15.677541  7039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.016s	sys 0.050s
I20260709 14:53:15.637306  6984 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:15.704943  7044 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 3.035s	user 0.005s	sys 0.017s
I20260709 14:53:15.705945  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 783) took 1302 ms. Trace:
I20260709 14:53:15.733909  2681 rpcz_store.cc:276] 0709 14:53:14.404141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.439569 (+ 35428us) service_pool.cc:224] Handling call
0709 14:53:15.705655 (+1266086us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.705850 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153517952}
I20260709 14:53:15.707472  7099 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:15.709898  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:15.746941  2752 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:15.672803  7089 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 14:53:15.717574  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:15.718721  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:15.719295  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:15.663272  7103 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:15.684033  7072 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:15.706529  7047 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:15.724875  6786 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:15.641309  7041 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 14:53:15.679783  6968 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:15.661330  7069 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 14:53:15.728175  6959 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:15.771557  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:15.772688  6990 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:15.773386  6990 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:15.773823  6990 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:15.774362  6959 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:15.753787  7002 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.763361  7039 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.772626  7041 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:15.751696  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:15.721786  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68037.4us
I20260709 14:53:15.794718  2686 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:15.796772  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:15.789310  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:15.799343  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:15.794052  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.007s	sys 0.020s
W20260709 14:53:15.749307  7124 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.307s	user 0.000s	sys 0.002s
I20260709 14:53:15.726784  6967 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:15.753844  7103 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:15.723186  7115 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:15.806557  7130 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:15.801088  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.007s	sys 0.020s
I20260709 14:53:15.721570  7126 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:15.810714  7130 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:15.812282  7113 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:15.812633  7130 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:15.800547  7105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.002s	sys 0.024s
I20260709 14:53:15.800192  7047 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:15.794282  7041 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:15.825579  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:15.809829  7103 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:15.835837  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:15.838213  6824 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:15.822952  7105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.002s	sys 0.024s
I20260709 14:53:15.755328  6984 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:15.843676  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:15.844767  7136 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 14:53:15.793248  7039 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:15.858192  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.005s	sys 0.021s
I20260709 14:53:15.829484  7047 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:15.844784  6985 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:15.839035  7115 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:15.862578  2606 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.002s	sys 0.028s
W20260709 14:53:15.930986  2606 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.003s	sys 0.028s
I20260709 14:53:15.793817  7002 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:15.938689  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.026s
W20260709 14:53:15.939401  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.000s	sys 0.026s
I20260709 14:53:15.864902  6895 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:15.861302  7025 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:15.861824  6815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.012s	sys 0.017s
W20260709 14:53:15.891022  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.005s	sys 0.022s
W20260709 14:53:15.908835  3411 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.589s (SENT)
I20260709 14:53:15.980037  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:15.981170  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 72424us
I20260709 14:53:15.858629  7085 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:15.920111  6764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.023s	sys 0.029s
W20260709 14:53:15.921108  2603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.017s	sys 0.036s
W20260709 14:53:15.986564  2603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.017s	sys 0.037s
I20260709 14:53:15.913679  7047 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:15.920598  6969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.486s	user 0.034s	sys 0.047s
W20260709 14:53:15.922851  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.021s	sys 0.041s
W20260709 14:53:15.994603  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.021s	sys 0.041s
W20260709 14:53:15.923466  2575 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.019s	sys 0.011s
W20260709 14:53:15.995384  2575 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.019s	sys 0.011s
W20260709 14:53:15.995723  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 880829us
W20260709 14:53:15.925053  7076 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.014s	sys 0.025s
W20260709 14:53:15.953106  2605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.014s	sys 0.014s
W20260709 14:53:15.957059  2754 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.008s	sys 0.011s
W20260709 14:53:15.958130  6675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.008s	sys 0.011s
W20260709 14:53:15.958446  7098 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.012s
I20260709 14:53:15.961169  6968 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:15.813016  7113 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:15.971330  2612 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.015s	sys 0.045s
W20260709 14:53:16.021133  2612 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.016s	sys 0.045s
W20260709 14:53:15.861124  7121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.005s	sys 0.022s
W20260709 14:53:15.976594  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 496029us
I20260709 14:53:15.874339  7105 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:15.990197  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 747) took 3406 ms (client timeout 1860 ms). Trace:
W20260709 14:53:16.008805  2605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.014s	sys 0.015s
W20260709 14:53:16.008881  2754 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.009s	sys 0.011s
W20260709 14:53:16.103426  2610 rpcz_store.cc:269] 0709 14:53:12.584296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.596551 (+ 12255us) service_pool.cc:224] Handling call
0709 14:53:15.989892 (+3393341us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b540050ffd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:15.990103 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368270464}
I20260709 14:53:15.998528  2686 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
W20260709 14:53:15.989461  6815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.013s	sys 0.017s
I20260709 14:53:15.984045  7025 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:15.986186  6764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.023s	sys 0.030s
W20260709 14:53:16.008667  7076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.014s	sys 0.026s
W20260709 14:53:16.009864  6969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.560s	user 0.035s	sys 0.047s
W20260709 14:53:16.030007  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.171s	user 0.025s	sys 0.017s
W20260709 14:53:16.116210  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.027s	sys 0.018s
I20260709 14:53:15.995990  6985 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:16.029469  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.006s	sys 0.032s
W20260709 14:53:16.117623  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.006s	sys 0.036s
I20260709 14:53:15.998149  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:15.993875  7085 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:16.027251  7121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.006s	sys 0.022s
W20260709 14:53:16.023334  6978 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.026s
I20260709 14:53:15.990744  7144 raft_consensus.cc:476] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Not starting pre-election -- already a leader
I20260709 14:53:15.876304  7041 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:16.010795  6675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.008s	sys 0.012s
W20260709 14:53:16.046905  7030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.879s	user 0.018s	sys 0.027s
I20260709 14:53:15.882663  7072 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:16.050897  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:16.108222  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1407) took 1762 ms. Trace:
I20260709 14:53:16.186736  2754 rpcz_store.cc:276] 0709 14:53:14.346274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.404785 (+ 58511us) service_pool.cc:224] Handling call
0709 14:53:14.406548 (+  1763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:14.424782 (+ 18234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.425561 (+   779us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:14.431108 (+  5547us) log.cc:844] Serialized 26 byte log entry
0709 14:53:15.434012 (+1002904us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:16.107433 (+673421us) log.cc:844] Serialized 14 byte log entry
0709 14:53:16.107742 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.107842 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.107904 (+    62us) raft_consensus.cc:1685] finished
0709 14:53:16.107926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.108139 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25039,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":22154,"thread_start_us":1608509,"threads_started":2,"wal-append.queue_time_us":1225959}
W20260709 14:53:16.014853  7098 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.007s	sys 0.013s
W20260709 14:53:16.106042  7135 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 14:53:16.051805  6959 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:16.076169  7133 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 14:53:16.137740  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139681us
I20260709 14:53:16.143733  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:16.120308  6824 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:16.161034  7140 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 14:53:16.026484  6968 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:16.162220  7134 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 14:53:16.175835  7041 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:16.114257  7025 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:16.148718  6978 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.006s	sys 0.026s
I20260709 14:53:16.192271  7170 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:16.192550  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 142117us
I20260709 14:53:16.138139  6957 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:16.227114  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 777) took 1902 ms. Trace:
I20260709 14:53:16.239243  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:16.147083  6864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.285s	user 0.003s	sys 0.041s
I20260709 14:53:16.182945  7005 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:16.262066  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:16.232664  7136 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 14:53:16.243005  2684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.005s	sys 0.036s
I20260709 14:53:16.117699  7099 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:16.249891  2610 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:16.177868  7030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.027s	user 0.019s	sys 0.027s
I20260709 14:53:16.213852  7144 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:16.283475  2684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.005s	sys 0.044s
W20260709 14:53:16.261745  7064 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.841s	user 0.006s	sys 0.000s
I20260709 14:53:16.177785  6969 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:16.242074  2677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.022s	sys 0.008s
I20260709 14:53:16.292801  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:16.229240  7085 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:16.288846  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:16.283180  6693 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:16.247900  6824 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:16.256520  2683 rpcz_store.cc:276] 0709 14:53:14.324456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.327074 (+  2618us) service_pool.cc:224] Handling call
0709 14:53:14.327485 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.203075 (+875590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.203125 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.204799 (+  1674us) log.cc:844] Serialized 26 byte log entry
0709 14:53:15.809672 (+604873us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:15.809741 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.809860 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.226719 (+416859us) raft_consensus.cc:1685] finished
0709 14:53:16.226743 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.227000 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34775,"raft.queue_time_us":76951,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":17551,"thread_start_us":1444596,"threads_started":2,"wal-append.queue_time_us":916156}
W20260709 14:53:16.308409  2677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.023s	sys 0.009s
I20260709 14:53:16.194639  7121 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 14:53:16.180492  7105 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:16.313411  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:16.320016  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:16.324430  2754 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:16.269693  6864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.418s	user 0.004s	sys 0.041s
I20260709 14:53:16.290457  7144 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:16.249423  6959 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:16.282855  7005 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:16.327481  6693 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:16.348301  7160 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 14:53:16.348788  6693 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:16.356112  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:16.268863  6957 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:16.319094  7039 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:16.316207  6969 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:16.353482  7144 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:16.365873  7145 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.251s	user 0.002s	sys 0.001s
I20260709 14:53:16.337384  7002 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:16.398638  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:16.401407  7142 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 14:53:16.449491  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1286) took 2464 ms (client timeout 2079 ms). Trace:
W20260709 14:53:16.458268  7164 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 14:53:16.458685  3538 rpcz_store.cc:269] 0709 14:53:13.946742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.947162 (+   420us) service_pool.cc:224] Handling call
0709 14:53:16.411189 (+2464027us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54004b5ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:16.411333 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159045376}
I20260709 14:53:16.471303  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 999) took 1652 ms. Trace:
I20260709 14:53:16.472097  3547 rpcz_store.cc:276] 0709 14:53:14.819391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.857123 (+ 37732us) service_pool.cc:224] Handling call
0709 14:53:14.885173 (+ 28050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:14.887829 (+  2656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.887882 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:14.918884 (+ 31002us) log.cc:844] Serialized 26 byte log entry
0709 14:53:15.595183 (+676299us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:15.595252 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.595344 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.471024 (+875680us) raft_consensus.cc:1685] finished
0709 14:53:16.471047 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.471225 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2110,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":9580,"thread_start_us":585569,"threads_started":1,"wal-append.queue_time_us":1048136}
W20260709 14:53:16.392243  7133 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:53:16.490654  6824 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:16.491580  6824 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:16.492153  6824 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:16.471230  7003 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:16.511157  2754 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:16.513041  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:16.372306  7014 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 14:53:16.557833  7105 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:16.559067  6786 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.794s	user 0.014s	sys 0.003s
I20260709 14:53:16.561373  7005 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:16.561724  7025 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:16.498869  7165 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 14:53:16.567315  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1273) took 3652 ms (client timeout 1952 ms). Trace:
W20260709 14:53:16.656263  2760 rpcz_store.cc:269] 0709 14:53:12.915256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.053697 (+138441us) service_pool.cc:224] Handling call
0709 14:53:16.566996 (+3513299us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:16.567208 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465708288}
W20260709 14:53:16.567972  7160 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 14:53:16.643857  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1275) took 3071 ms. Trace:
I20260709 14:53:16.571528  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 766) took 2110 ms. Trace:
W20260709 14:53:16.710316  2751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.010s	sys 0.034s
I20260709 14:53:16.669699  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:16.812928  2751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.011s	sys 0.035s
W20260709 14:53:16.703717  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:16.814563  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1219) took 3251 ms. Trace:
W20260709 14:53:16.822994  2606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.016s	sys 0.040s
W20260709 14:53:16.824455  2606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.016s	sys 0.040s
I20260709 14:53:16.859181  2682 rpcz_store.cc:276] 0709 14:53:13.562993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.832896 (+269903us) service_pool.cc:224] Handling call
0709 14:53:13.856249 (+ 23353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.108769 (+252520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.108822 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:14.108898 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:14.108965 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.814301 (+2705336us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:16.814357 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.814501 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579888896}
W20260709 14:53:16.861761  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:16.883101  2753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.009s	sys 0.042s
W20260709 14:53:16.694193  7057 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 3.552s	user 0.012s	sys 0.009s
W20260709 14:53:16.697566  6990 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.923s	user 0.013s	sys 0.004s
W20260709 14:53:16.695741  7103 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.847s	user 0.016s	sys 0.004s
W20260709 14:53:16.890466  2753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.009s	sys 0.042s
W20260709 14:53:16.894510  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1434) took 3465 ms (client timeout 1906 ms). Trace:
W20260709 14:53:16.965389  2607 rpcz_store.cc:269] 0709 14:53:13.417552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.618453 (+200901us) service_pool.cc:224] Handling call
0709 14:53:16.882440 (+3263987us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:16.882790 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556651264}
W20260709 14:53:16.765591  7019 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.442s	user 0.019s	sys 0.000s
I20260709 14:53:16.657533  6824 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:16.647970  7105 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:16.889170  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:17.056880  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.007s	sys 0.044s
W20260709 14:53:17.058090  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.010s	sys 0.042s
W20260709 14:53:17.059926  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.481s	user 0.030s	sys 0.072s
W20260709 14:53:17.060318  2603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.024s	sys 0.028s
W20260709 14:53:17.060715  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.014s	sys 0.032s
W20260709 14:53:17.086988  6764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.010s	sys 0.053s
W20260709 14:53:17.087563  6987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.646s	user 0.042s	sys 0.084s
W20260709 14:53:17.087852  2759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.603s	user 0.029s	sys 0.081s
W20260709 14:53:17.088611  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.018s	sys 0.059s
W20260709 14:53:17.089061  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.022s	sys 0.033s
I20260709 14:53:16.794139  6969 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:16.794559  6968 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:16.649577  6786 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:17.353864  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:17.538174  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.516s	user 0.033s	sys 0.037s
I20260709 14:53:17.133739  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:17.539737  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.539s	user 0.016s	sys 0.062s
W20260709 14:53:17.539835  6987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.099s	user 0.044s	sys 0.098s
W20260709 14:53:17.383189  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:17.540774  2759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.060s	user 0.032s	sys 0.093s
W20260709 14:53:17.540956  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.962s	user 0.032s	sys 0.095s
I20260709 14:53:17.541154  7025 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:17.542014  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1287) took 3215 ms. Trace:
W20260709 14:53:17.542260  6764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.380s	user 0.025s	sys 0.053s
I20260709 14:53:17.542368  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:17.543617  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 774) took 1547 ms. Trace:
I20260709 14:53:17.542528  3543 rpcz_store.cc:276] 0709 14:53:14.326397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.718534 (+392137us) service_pool.cc:224] Handling call
0709 14:53:14.760751 (+ 42217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.761157 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:15.578429 (+817272us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.541478 (+1963049us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:17.541554 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:17.541634 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.541738 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.541915 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2674,"prepare.queue_time_us":96425,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":413,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":2776506,"threads_started":2}
I20260709 14:53:17.544046  2607 rpcz_store.cc:276] 0709 14:53:15.996545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:16.966642 (+970097us) service_pool.cc:224] Handling call
0709 14:53:17.543541 (+576899us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:17.538268  2603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.551s	user 0.033s	sys 0.045s
I20260709 14:53:17.127182  6990 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:17.539125  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.512s	user 0.026s	sys 0.066s
I20260709 14:53:17.546495  2607 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 14:53:17.502203  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.398s	user 0.007s	sys 0.028s
W20260709 14:53:17.494458  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.177s	user 0.011s	sys 0.024s
W20260709 14:53:17.539124  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.525s	user 0.028s	sys 0.044s
W20260709 14:53:17.540158  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.5397e+06us
W20260709 14:53:17.599167  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.495s	user 0.007s	sys 0.030s
W20260709 14:53:17.602347  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:17.603346  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.013s	sys 0.025s
I20260709 14:53:17.608552  6987 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:17.492123  7175 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 14:53:17.550972  7025 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:17.609529  6987 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:17.609871  7025 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:17.608963  6824 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:17.611055  6824 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:17.612619  7025 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:17.613271  7025 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:53:17.613933  7025 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:17.540087  2760 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:17.612121  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1415) took 1904 ms. Trace:
I20260709 14:53:17.585887  6990 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:17.615475  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1298) took 1693 ms. Trace:
I20260709 14:53:17.615257  2755 rpcz_store.cc:276] 0709 14:53:15.708303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.710955 (+  2652us) service_pool.cc:224] Handling call
0709 14:53:15.711419 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.711916 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.711954 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.712595 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.097322 (+384727us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:16.097437 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.097634 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:17.611869 (+1514235us) raft_consensus.cc:1685] finished
0709 14:53:17.611894 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.612052 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":383637,"threads_started":1,"wal-append.queue_time_us":1460091}
I20260709 14:53:17.618654  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:17.619390  2755 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:17.619930  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1442) took 3129 ms. Trace:
I20260709 14:53:17.620534  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:17.621450  2755 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:17.621250  2607 rpcz_store.cc:276] 0709 14:53:14.490912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.545290 (+3054378us) service_pool.cc:224] Handling call
0709 14:53:17.546078 (+   788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:17.602282 (+ 56204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.619782 (+ 17500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:17.615633  6824 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:17.621657  2760 rpcz_store.cc:276] 0709 14:53:15.922128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:16.916582 (+994454us) service_pool.cc:224] Handling call
0709 14:53:17.615341 (+698759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:17.627504  2760 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:17.628319  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1288) took 1800 ms. Trace:
I20260709 14:53:17.628731  2760 rpcz_store.cc:276] 0709 14:53:15.827528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.626842 (+1799314us) service_pool.cc:224] Handling call
0709 14:53:17.627258 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:17.628021 (+   763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.628250 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:17.626052  6987 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:17.382058  6959 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:17.314357  7160 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.429s	user 0.000s	sys 0.000s
I20260709 14:53:17.644256  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1001) took 2796 ms. Trace:
I20260709 14:53:17.645810  6786 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:17.644716  3545 rpcz_store.cc:276] 0709 14:53:14.847578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.133505 (+285927us) service_pool.cc:224] Handling call
0709 14:53:15.134056 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.156434 (+ 22378us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:15.231370 (+ 74936us) log.cc:844] Serialized 14 byte log entry
0709 14:53:16.121695 (+890325us) log.cc:844] Serialized 14 byte log entry
0709 14:53:16.121819 (+   124us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:16.149603 (+ 27784us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.154301 (+  4698us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:16.154648 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 14:53:16.154930 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.155041 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:17.643969 (+1488928us) raft_consensus.cc:1685] finished
0709 14:53:17.643995 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.644182 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25639,"prepare.queue_time_us":108059,"prepare.run_cpu_time_us":190,"prepare.run_wall_time_us":195,"raft.queue_time_us":194929,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":19414,"thread_start_us":861666,"threads_started":1,"wal-append.queue_time_us":740742}
W20260709 14:53:17.653455  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.639s	user 0.014s	sys 0.046s
I20260709 14:53:17.654863  2754 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:53:17.655988  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1295) took 1737 ms (client timeout 2241 ms). Trace:
W20260709 14:53:17.656543  2754 rpcz_store.cc:269] 0709 14:53:15.918612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:16.658896 (+740284us) service_pool.cc:224] Handling call
0709 14:53:17.655874 (+996978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:17.658550  2754 pending_rounds.cc:85] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
W20260709 14:53:17.659034  2757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.455s	user 0.001s	sys 0.007s
W20260709 14:53:17.780061  2757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.590s	user 0.002s	sys 0.007s
W20260709 14:53:17.659446  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.006s	sys 0.005s
W20260709 14:53:17.781314  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.465s	user 0.006s	sys 0.005s
W20260709 14:53:17.662262  6977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.010s	sys 0.003s
I20260709 14:53:17.134310  7103 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:17.173069  7019 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:16.915755  7032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.606s	user 0.025s	sys 0.029s
I20260709 14:53:17.134681  6957 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:17.790331  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:17.805647  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:17.681226  7144 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.201s	user 0.008s	sys 0.005s
W20260709 14:53:17.690312  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 782) took 5170 ms (client timeout 1932 ms). Trace:
W20260709 14:53:17.812420  2758 rpcz_store.cc:269] 0709 14:53:12.519510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:12.699330 (+179820us) service_pool.cc:224] Handling call
0709 14:53:17.689899 (+4990569us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54000d0f54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:17.690157 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3533213568}
I20260709 14:53:17.812862  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 786) took 2095 ms. Trace:
I20260709 14:53:17.813679  2677 rpcz_store.cc:276] 0709 14:53:15.717435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.726695 (+  9260us) service_pool.cc:224] Handling call
0709 14:53:15.727126 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.727360 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:16.315569 (+588209us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.812234 (+1496665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:17.812362 (+   128us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:17.812465 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.812587 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.812783 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12873,"prepare.queue_time_us":594251,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"thread_start_us":2053721,"threads_started":2}
W20260709 14:53:17.697304  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.715s	user 0.036s	sys 0.036s
W20260709 14:53:17.815923  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.834s	user 0.037s	sys 0.037s
W20260709 14:53:17.816507  2606 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.032s	sys 0.019s
W20260709 14:53:17.817028  2606 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.032s	sys 0.019s
I20260709 14:53:17.817998  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1429) took 4420 ms. Trace:
W20260709 14:53:17.818048  3411 connection.cc:256] RPC call timeout handler was delayed by -1.8239099879999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2972s for a response
I20260709 14:53:16.660630  3540 rpcz_store.cc:276] 0709 14:53:13.573523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.582996 (+  9473us) service_pool.cc:224] Handling call
0709 14:53:13.630298 (+ 47302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:13.760113 (+129815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:13.760163 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:13.760869 (+   706us) log.cc:844] Serialized 52 byte log entry
0709 14:53:14.465184 (+704315us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b18002fcc80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:53:15.430601 (+965417us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:15.431072 (+   471us) log.cc:844] Serialized 14 byte log entry
0709 14:53:15.431535 (+   463us) log.cc:844] Serialized 14 byte log entry
0709 14:53:15.431719 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.431810 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.643427 (+1211617us) raft_consensus.cc:1685] finished
0709 14:53:16.643478 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.643754 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18779,"prepare.queue_time_us":342,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":153710,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":7679,"spinlock_wait_cycles":8945408,"thread_start_us":990997,"threads_started":2,"wal-append.queue_time_us":1851453,"wal-append.run_cpu_time_us":22468,"wal-append.run_wall_time_us":1872762}
I20260709 14:53:17.700845  2604 rpcz_store.cc:276] 0709 14:53:14.461316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.386591 (+925275us) service_pool.cc:224] Handling call
0709 14:53:15.399511 (+ 12920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.403854 (+  4343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.403918 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.404586 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 14:53:15.945694 (+541108us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:15.945762 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.945877 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.571065 (+625188us) raft_consensus.cc:1685] finished
0709 14:53:16.571103 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.571368 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":600,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":4547,"thread_start_us":523320,"threads_started":1,"wal-append.queue_time_us":881815,"wal-append.run_cpu_time_us":15644,"wal-append.run_wall_time_us":1252618}
W20260709 14:53:16.649761  7114 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.380s	user 0.007s	sys 0.026s
W20260709 14:53:17.705018  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.014s	sys 0.039s
W20260709 14:53:17.342799  7140 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.006s	sys 0.023s
I20260709 14:53:17.708415  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 762) took 3285 ms. Trace:
I20260709 14:53:17.126663  6996 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:16.811198  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1418) took 1022 ms. Trace:
I20260709 14:53:17.755524  7052 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 14:53:17.819906  6693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.017s	sys 0.034s
W20260709 14:53:17.761245  7041 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.514s	user 0.013s	sys 0.000s
I20260709 14:53:17.768518  7077 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:17.718696  7130 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.904s	user 0.013s	sys 0.016s
I20260709 14:53:17.782346  2754 pending_rounds.cc:107] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 9.5
W20260709 14:53:17.790169  3542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.762s	user 0.028s	sys 0.053s
I20260709 14:53:17.725752  6971 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:17.808871  2652 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.004s	sys 0.009s
W20260709 14:53:17.785954  6414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.484s	user 0.028s	sys 0.043s
I20260709 14:53:17.824131  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:17.786530  6977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.308s	user 0.010s	sys 0.003s
W20260709 14:53:17.828970  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.014s	sys 0.039s
I20260709 14:53:17.827643  6957 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:17.829604  2756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.604s	user 0.033s	sys 0.108s
W20260709 14:53:17.819970  3411 connection.cc:256] RPC call timeout handler was delayed by -1.3953877250000915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2782s for a response
W20260709 14:53:17.791559  7032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.484s	user 0.026s	sys 0.036s
I20260709 14:53:17.839594  2603 rpcz_store.cc:276] 0709 14:53:14.422847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.146871 (+724024us) service_pool.cc:224] Handling call
0709 14:53:15.147287 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.147554 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:15.987221 (+839667us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.707883 (+1720662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:17.707962 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:17.708043 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.708144 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.708335 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162628,"prepare.queue_time_us":598263,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":15871,"raft.queue_time_us":188071,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":40015,"thread_start_us":2368447,"threads_started":2}
I20260709 14:53:17.851472  2681 raft_consensus.cc:2368] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16.
I20260709 14:53:17.757087  6786 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:17.856057  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:17.863353  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:17.844532  2752 rpcz_store.cc:276] 0709 14:53:15.788857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.789276 (+   419us) service_pool.cc:224] Handling call
0709 14:53:15.792789 (+  3513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:15.793275 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.793306 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:15.808807 (+ 15501us) log.cc:844] Serialized 52 byte log entry
0709 14:53:15.958379 (+149572us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:15.958438 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.958526 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:16.566726 (+608200us) raft_consensus.cc:1685] finished
0709 14:53:16.566760 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:16.811085 (+244325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7084,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":4550,"thread_start_us":135603,"threads_started":1,"wal-append.queue_time_us":580301,"wal-append.run_cpu_time_us":15604,"wal-append.run_wall_time_us":1144887}
I20260709 14:53:17.789829  7089 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:17.818465  2606 rpcz_store.cc:276] 0709 14:53:13.397995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.931908 (+1533913us) service_pool.cc:224] Handling call
0709 14:53:14.973870 (+ 41962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.974134 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:15.931684 (+957550us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.817406 (+1885722us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.817613 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:17.817670 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:17.817723 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.817800 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.817935 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56472,"prepare.queue_time_us":717371,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":2926,"raft.queue_time_us":56828,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":210883,"thread_start_us":2743627,"threads_started":3}
W20260709 14:53:17.845903  7140 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.504s	user 0.011s	sys 0.029s
I20260709 14:53:17.822525  7144 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:17.837751  6820 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:17.921277  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1225) took 3029 ms. Trace:
W20260709 14:53:18.976750  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.000s	sys 0.009s
W20260709 14:53:18.560101  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.021s	sys 0.017s
W20260709 14:53:19.203630  2756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.979s	user 0.054s	sys 0.140s
W20260709 14:53:18.473239  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.003s	sys 0.022s
W20260709 14:53:19.204162  2652 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.145s	user 0.034s	sys 0.050s
W20260709 14:53:18.161047  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:18.101377  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:17.931880  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1220) took 4271 ms. Trace:
W20260709 14:53:18.482789  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.013s	sys 0.012s
W20260709 14:53:18.168742  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.018s	sys 0.022s
W20260709 14:53:18.754720  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.009s	sys 0.030s
W20260709 14:53:19.212929  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.433s	user 0.009s	sys 0.050s
I20260709 14:53:19.212296  2678 rpcz_store.cc:276] 0709 14:53:13.660359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:13.867148 (+206789us) service_pool.cc:224] Handling call
0709 14:53:15.371004 (+1503856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.371417 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.371447 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:15.371493 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:15.371530 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.371601 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.931605 (+2560004us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540012cd50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:17.931794 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":503,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":3372676480}
W20260709 14:53:17.932427  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1363) took 4700 ms (client timeout 2121 ms). Trace:
I20260709 14:53:19.203186  2754 raft_consensus.cc:2889] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:18.724793  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.003s	sys 0.036s
W20260709 14:53:19.215345  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.674s	user 0.005s	sys 0.038s
W20260709 14:53:18.779564  2604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.003s	sys 0.004s
W20260709 14:53:19.003715  2724 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.107s	user 0.005s	sys 0.005s
W20260709 14:53:19.215404  2611 rpcz_store.cc:269] 0709 14:53:13.231670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.036022 (+804352us) service_pool.cc:224] Handling call
0709 14:53:17.932179 (+3896157us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:17.932356 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379655552}
W20260709 14:53:19.216132  2724 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.008s	sys 0.008s
W20260709 14:53:19.203084  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.001s	sys 0.011s
W20260709 14:53:19.203217  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1955655190000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
I20260709 14:53:19.201807  2679 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:19.202306  2685 rpcz_store.cc:276] 0709 14:53:14.892396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.892605 (+   209us) service_pool.cc:224] Handling call
0709 14:53:15.707184 (+814579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.707631 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.707663 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:15.707715 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:15.707750 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:15.707818 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:17.920986 (+2213168us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400268c50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:17.921178 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":586,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":121,"spinlock_wait_cycles":1872191360}
W20260709 14:53:19.217038  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0216115580000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
I20260709 14:53:17.917652  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:19.207223  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.457s	user 0.021s	sys 0.041s
W20260709 14:53:19.216665  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.32085e+06us
I20260709 14:53:19.218143  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 801) took 1386 ms. Trace:
I20260709 14:53:19.218649  2682 rpcz_store.cc:276] 0709 14:53:17.831982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.874486 (+ 42504us) service_pool.cc:224] Handling call
0709 14:53:19.218065 (+1343579us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:19.219463  2604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.341s	user 0.006s	sys 0.005s
W20260709 14:53:18.720472  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.001s	sys 0.008s
W20260709 14:53:19.222417  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.513s	user 0.003s	sys 0.010s
W20260709 14:53:18.720943  2612 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.006s	user 0.005s	sys 0.003s
I20260709 14:53:19.203128  2686 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:19.212317  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.021s	sys 0.041s
W20260709 14:53:19.223178  2612 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.516s	user 0.010s	sys 0.003s
W20260709 14:53:19.224889  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:19.225461  2575 connection.cc:256] RPC call timeout handler was delayed by -1.1271865689999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 14:53:19.212823  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.588s	user 0.036s	sys 0.054s
W20260709 14:53:19.202317  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 2.90935e+06us
W20260709 14:53:19.203084  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.469s	user 0.037s	sys 0.033s
I20260709 14:53:19.209555  2677 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:19.227036  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:18.720769  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.003s	sys 0.006s
W20260709 14:53:19.206259  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 2.14749e+06us
I20260709 14:53:19.217396  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 800) took 1423 ms. Trace:
W20260709 14:53:18.720189  2605 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.000s	sys 0.008s
I20260709 14:53:19.227273  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:19.227957  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 797) took 1435 ms (client timeout 1525 ms). Trace:
W20260709 14:53:19.227428  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1235) took 1446 ms (client timeout 1696 ms). Trace:
W20260709 14:53:19.227069  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.49292e+06us
I20260709 14:53:19.220381  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1443) took 3223 ms. Trace:
W20260709 14:53:19.231887  3542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.209s	user 0.045s	sys 0.107s
I20260709 14:53:19.235735  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:17.869351  6693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.018s	sys 0.034s
W20260709 14:53:17.930780  7080 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 4.063s	user 0.019s	sys 0.022s
W20260709 14:53:18.983841  7197 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
W20260709 14:53:19.388182  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 138621us
W20260709 14:53:18.927068  7203 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.342s	user 0.001s	sys 0.000s
W20260709 14:53:19.387786  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 153983us
W20260709 14:53:19.364233  2723 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.523s (SENT) [suppressed 19 similar messages]
I20260709 14:53:19.398085  2679 rpcz_store.cc:276] 0709 14:53:17.793799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.823612 (+ 29813us) service_pool.cc:224] Handling call
0709 14:53:19.217325 (+1393713us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:19.209059  6997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.788s	user 0.011s	sys 0.014s
W20260709 14:53:19.442914  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 181088us
W20260709 14:53:18.961179  7180 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.678s	user 0.001s	sys 0.001s
W20260709 14:53:19.396862  2686 rpcz_store.cc:269] 0709 14:53:17.792760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.823791 (+ 31031us) service_pool.cc:224] Handling call
0709 14:53:19.227888 (+1404097us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:19.136606  6967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.164s	user 0.011s	sys 0.019s
W20260709 14:53:18.002157  7172 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.067s	user 0.002s	sys 0.001s
W20260709 14:53:18.975929  7202 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 14:53:19.409482  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:19.076994  7182 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.627s	user 0.001s	sys 0.001s
W20260709 14:53:19.033972  7190 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.684s	user 0.001s	sys 0.001s
W20260709 14:53:19.295045  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66146us
W20260709 14:53:17.879738  7072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.003s	sys 0.022s
W20260709 14:53:17.934124  7122 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 3.409s	user 0.007s	sys 0.009s
W20260709 14:53:18.330015  7169 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.005s	sys 0.002s
W20260709 14:53:17.997601  7176 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.061s	user 0.001s	sys 0.002s
W20260709 14:53:18.707298  7047 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 2.693s	user 0.011s	sys 0.017s
W20260709 14:53:19.099741  7098 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.014s
W20260709 14:53:18.982815  7193 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
W20260709 14:53:18.089319  7183 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.153s	user 0.003s	sys 0.001s
I20260709 14:53:17.938449  6972 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:19.201614  6414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.899s	user 0.053s	sys 0.093s
W20260709 14:53:18.861778  7168 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.833s	user 0.003s	sys 0.000s
I20260709 14:53:17.850711  7052 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:19.387154  6986 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.029s	sys 0.051s
I20260709 14:53:19.447700  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1123) took 5209 ms. Trace:
W20260709 14:53:19.396576  7186 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.560s	user 0.002s	sys 0.000s
I20260709 14:53:17.839833  7114 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:19.226212  6969 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:18.708791  7103 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:17.970330  7041 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:17.946764  7126 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:18.848146  7102 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:19.233866  7100 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:19.479631  7047 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.427423  7081 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:19.440514  7088 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.228772  7105 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:19.458928  7072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.820s	user 0.034s	sys 0.061s
I20260709 14:53:19.462523  7198 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:19.207870  7130 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.231726  7089 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:19.501663  7098 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.008s	sys 0.029s
W20260709 14:53:19.508463  7173 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.472s	user 0.001s	sys 0.001s
I20260709 14:53:19.532068  7105 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:19.536563  7096 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:19.537142  7096 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:19.537818  7130 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:19.527078  7100 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:19.607539  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.898s	user 0.015s	sys 0.016s
I20260709 14:53:19.253974  2603 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:19.607208  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 380766us
W20260709 14:53:19.396190  2605 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.007s	sys 0.024s
I20260709 14:53:19.530207  7088 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:19.395190  2677 rpcz_store.cc:269] 0709 14:53:17.780924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.815744 (+ 34820us) service_pool.cc:224] Handling call
0709 14:53:19.227360 (+1411616us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.736654  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:19.737334  7088 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.738004  2677 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:19.738253  7114 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:19.448113  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83959.1us
I20260709 14:53:19.628600  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1413) took 3954 ms. Trace:
I20260709 14:53:19.448661  2752 raft_consensus.cc:2368] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:53:19.411401  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1408) took 4830 ms. Trace:
I20260709 14:53:19.742859  2751 rpcz_store.cc:276] 0709 14:53:15.674424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.674670 (+   246us) service_pool.cc:224] Handling call
0709 14:53:15.675211 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.683881 (+  8670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.683932 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.684575 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.813563 (+1128988us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:16.813606 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.752864 (+939258us) spinlock_profiling.cc:243] Waited 8.35 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:17.752953 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.627506 (+1874553us) raft_consensus.cc:1685] finished
0709 14:53:19.627558 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.628240 (+   682us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19194112,"thread_start_us":1128475,"threads_started":1,"wal-append.queue_time_us":3294558}
I20260709 14:53:19.610669  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.753014  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.754087  7210 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.634874  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1414) took 3958 ms. Trace:
I20260709 14:53:19.612439  2610 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:19.755489  7105 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:19.612833  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 181117us
I20260709 14:53:19.635377  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1417) took 3921 ms. Trace:
I20260709 14:53:19.634968  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1419) took 3826 ms. Trace:
I20260709 14:53:19.611784  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.616873  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 785) took 3899 ms. Trace:
I20260709 14:53:19.621591  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:19.619669  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 177405us
I20260709 14:53:19.761380  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 795) took 1974 ms. Trace:
I20260709 14:53:19.761893  2678 rpcz_store.cc:276] 0709 14:53:17.787256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.620535 (+1833279us) service_pool.cc:224] Handling call
0709 14:53:19.761266 (+140731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.763365  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:19.622522  2680 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:19.608568  2604 rpcz_store.cc:276] 0709 14:53:15.997128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.820852 (+1823724us) service_pool.cc:224] Handling call
0709 14:53:17.863863 (+ 43011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.219962 (+1356099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:19.220012 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.220068 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:19.220104 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.220187 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.220323 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22785,"raft.queue_time_us":911292,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":43946,"thread_start_us":1341388,"threads_started":1}
I20260709 14:53:19.619853  2683 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:19.766183  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 796) took 1978 ms. Trace:
I20260709 14:53:19.510087  3542 rpcz_store.cc:276] 0709 14:53:14.187166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.591135 (+403969us) service_pool.cc:224] Handling call
0709 14:53:14.717765 (+126630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:14.718094 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:15.607509 (+889415us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.396132 (+3788623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.396210 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:19.396274 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.396435 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.396610 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65430,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":157144,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":28252,"thread_start_us":4632649,"threads_started":3,"wal-append.queue_time_us":622920,"wal-append.run_cpu_time_us":1531,"wal-append.run_wall_time_us":1044122}
I20260709 14:53:19.232417  6820 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:19.771090  2607 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:19.770784  7210 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.759333  2755 rpcz_store.cc:276] 0709 14:53:15.808607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.622517 (+1813910us) service_pool.cc:224] Handling call
0709 14:53:17.623671 (+  1154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:17.624073 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:17.624098 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:17.624651 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 14:53:19.226489 (+1601838us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:19.226524 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.226627 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.634347 (+407720us) raft_consensus.cc:1685] finished
0709 14:53:19.634393 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.634704 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1521,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":1601377,"threads_started":1,"wal-append.queue_time_us":1911353}
I20260709 14:53:19.773563  6820 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:19.230568  6957 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:19.464962  3411 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 4.223s (SENT)
I20260709 14:53:19.773661  7212 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:19.229648  6786 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:19.483435  6984 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:17.860095  6996 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:19.449816  6997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.031s	user 0.019s	sys 0.025s
I20260709 14:53:19.484689  7019 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:19.565227  7144 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:19.558156  7199 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.680s	user 0.000s	sys 0.020s
I20260709 14:53:19.776628  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:19.611889  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 790) took 3892 ms (client timeout 1843 ms). Trace:
I20260709 14:53:19.755039  2753 rpcz_store.cc:276] 0709 14:53:15.676739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.676988 (+   249us) service_pool.cc:224] Handling call
0709 14:53:15.681257 (+  4269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.681726 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.681760 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.682325 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.965507 (+1283182us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:16.965565 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.965660 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.634060 (+2668400us) raft_consensus.cc:1685] finished
0709 14:53:19.634113 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.634551 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3839,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":1282658,"threads_started":1,"wal-append.queue_time_us":2251835}
I20260709 14:53:19.614450  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 755) took 5250 ms. Trace:
W20260709 14:53:19.615077  2651 connection.cc:256] RPC call timeout handler was delayed by -1.1319707010000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 14:53:19.687549  7170 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.687723  6969 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.688766  7126 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:19.525177  7041 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:19.691300  7130 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:19.801687  7130 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:19.803911  6996 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:19.804844  7130 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.691824  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1291) took 3813 ms. Trace:
I20260709 14:53:19.805780  2758 rpcz_store.cc:276] 0709 14:53:15.878350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.813666 (+1935316us) service_pool.cc:224] Handling call
0709 14:53:17.814088 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:17.814510 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:17.850416 (+ 35906us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.690661 (+1840245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.691007 (+   346us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:19.691160 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.691287 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.691541 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12205,"prepare.queue_time_us":122881,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":70,"raft.queue_time_us":202638,"raft.run_cpu_time_us":480,"raft.run_wall_time_us":30439,"thread_start_us":1678807,"threads_started":2,"wal-append.queue_time_us":1615734}
I20260709 14:53:19.809309  6786 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.809980  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1420) took 2205 ms. Trace:
I20260709 14:53:19.794675  7072 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.810479  2758 rpcz_store.cc:276] 0709 14:53:17.604792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.808263 (+2203471us) service_pool.cc:224] Handling call
0709 14:53:19.808867 (+   604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.809449 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:19.809486 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.809585 (+    99us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:19.809673 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.809738 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.809872 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":822,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126}
I20260709 14:53:19.811239  7072 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.812862  7041 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.692577  7215 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:19.813424  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1421) took 2208 ms. Trace:
W20260709 14:53:19.777580  2681 rpcz_store.cc:269] 0709 14:53:15.722141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.750781 (+ 28640us) service_pool.cc:224] Handling call
0709 14:53:19.611008 (+3860227us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b540024ead4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:19.611455 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4700161408}
I20260709 14:53:19.813903  2758 rpcz_store.cc:276] 0709 14:53:17.605160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.812099 (+2206939us) service_pool.cc:224] Handling call
0709 14:53:19.812594 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.813010 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:19.813034 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.813091 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:19.813151 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.813213 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.813359 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.816026  6957 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:19.698697  7100 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:19.515395  6986 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.032s	sys 0.054s
I20260709 14:53:19.816975  2758 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:19.778043  2612 rpcz_store.cc:276] 0709 14:53:14.363781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.126845 (+763064us) service_pool.cc:224] Handling call
0709 14:53:15.158273 (+ 31428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.158598 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:16.021645 (+863047us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.223703 (+3202058us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.613150 (+389447us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.613344 (+   194us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:19.613590 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.613760 (+   170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.614104 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":443431,"prepare.queue_time_us":1630324,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":192,"raft.queue_time_us":633145,"raft.run_cpu_time_us":872,"raft.run_wall_time_us":18585,"thread_start_us":3841700,"threads_started":3,"wal-append.queue_time_us":2119521}
I20260709 14:53:19.817386  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.717579  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:19.718461  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1410) took 5243 ms. Trace:
W20260709 14:53:19.719080  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64612.5us
I20260709 14:53:19.509718  6972 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:19.721674  7052 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:19.677464  7113 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.694706  7047 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:19.729478  7032 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.730269  7188 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:19.731031  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.814441  7072 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:19.620685  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 230908us
I20260709 14:53:19.767117  7089 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.760761  2684 rpcz_store.cc:276] 0709 14:53:15.717082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.717252 (+   170us) service_pool.cc:224] Handling call
0709 14:53:15.725527 (+  8275us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:15.725975 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.726011 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:15.726750 (+   739us) log.cc:844] Serialized 52 byte log entry
0709 14:53:16.290931 (+564181us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:16.290981 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:16.291061 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.615878 (+3324817us) raft_consensus.cc:1685] finished
0709 14:53:19.615941 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.616490 (+   549us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":484,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"thread_start_us":563577,"threads_started":1,"wal-append.queue_time_us":2102978}
I20260709 14:53:19.768090  2611 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:19.750330  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:19.834606  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:19.454886  6967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.487s	user 0.014s	sys 0.024s
W20260709 14:53:19.758455  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1444) took 3761 ms (client timeout 2292 ms). Trace:
I20260709 14:53:19.769098  2678 rpcz_store.cc:276] 0709 14:53:17.787659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.764583 (+1976924us) service_pool.cc:224] Handling call
0709 14:53:19.765216 (+   633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.765649 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:19.765690 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.765777 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:19.765877 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.765959 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.766091 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.774501  7210 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:19.759193  2757 rpcz_store.cc:276] 0709 14:53:15.714000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.714230 (+   230us) service_pool.cc:224] Handling call
0709 14:53:15.714726 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:15.715312 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:15.715357 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:15.716045 (+   688us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.086892 (+370847us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:17.780731 (+1693839us) log.cc:844] Serialized 14 byte log entry
0709 14:53:17.785983 (+  5252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:17.786105 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.634686 (+1848581us) raft_consensus.cc:1685] finished
0709 14:53:19.634734 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.635131 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138552,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":12277,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":19342,"thread_start_us":1960814,"threads_started":2,"wal-append.queue_time_us":2220004}
I20260709 14:53:19.783702  2608 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
W20260709 14:53:19.784345  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 315320us
I20260709 14:53:19.784755  7231 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:19.784394  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:19.789388  7081 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:19.748986  2759 rpcz_store.cc:276] 0709 14:53:14.399113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.427471 (+ 28358us) service_pool.cc:224] Handling call
0709 14:53:14.448116 (+ 20645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:14.448564 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.448596 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:14.466860 (+ 18264us) log.cc:844] Serialized 26 byte log entry
0709 14:53:15.450485 (+983625us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:17.541284 (+2090799us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.229190 (+1687906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.229286 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.229332 (+    46us) raft_consensus.cc:1685] finished
0709 14:53:19.229345 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.229484 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13546,"prepare.queue_time_us":774753,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":13522,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":211310,"spinlock_wait_cycles":1712000,"thread_start_us":4679193,"threads_started":3,"wal-append.queue_time_us":3202627,"wal-append.run_cpu_time_us":25405,"wal-append.run_wall_time_us":1708897}
I20260709 14:53:19.791299  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1232) took 2014 ms. Trace:
I20260709 14:53:19.844388  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 771) took 3843 ms. Trace:
I20260709 14:53:19.774303  2755 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:19.796790  2651 connection.cc:256] RPC call timeout handler was delayed by -1.1216748660000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 14:53:19.797605  7170 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:19.538094  7096 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.810506  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 791) took 4044 ms. Trace:
I20260709 14:53:19.823231  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.818092  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1422) took 2212 ms. Trace:
I20260709 14:53:19.796880  7126 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:19.729565  2609 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:19.863709  7231 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:19.845917  2683 rpcz_store.cc:276] 0709 14:53:17.777349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.777517 (+   168us) service_pool.cc:224] Handling call
0709 14:53:17.777912 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.766633 (+1988721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.790847 (+ 24214us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:19.791205 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4234159360}
I20260709 14:53:19.865602  6972 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:19.831910  2685 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:19.836320  2751 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 14:53:19.835959  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.849037  2759 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 14:53:19.867785  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1425) took 2261 ms. Trace:
I20260709 14:53:19.868311  2751 rpcz_store.cc:276] 0709 14:53:17.606367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.834444 (+2228077us) service_pool.cc:224] Handling call
0709 14:53:19.835026 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.867411 (+ 32385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.867692 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.848901  2757 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:19.869143  2759 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:19.846123  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.818539  2756 rpcz_store.cc:276] 0709 14:53:14.475349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:14.704636 (+229287us) service_pool.cc:224] Handling call
0709 14:53:14.758032 (+ 53396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:14.790700 (+ 32668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:14.790757 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:14.791556 (+   799us) log.cc:844] Serialized 26 byte log entry
0709 14:53:16.116862 (+1325306us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:19.204365 (+3087503us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.717739 (+513374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.717865 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:19.717943 (+    78us) raft_consensus.cc:1685] finished
0709 14:53:19.717958 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.718231 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152987,"prepare.queue_time_us":108857,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":117112,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":308,"thread_start_us":4666484,"threads_started":3,"wal-append.queue_time_us":1720556,"wal-append.run_cpu_time_us":2064,"wal-append.run_wall_time_us":1049065}
I20260709 14:53:19.872059  2612 pending_rounds.cc:85] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 5
I20260709 14:53:19.872555  2612 pending_rounds.cc:107] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 14:53:19.872929  2612 raft_consensus.cc:2889] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:19.867321  2604 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:19.874882  6978 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:19.875816  6978 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:19.876385  7188 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:19.851294  2758 rpcz_store.cc:276] 0709 14:53:17.605479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.815686 (+2210207us) service_pool.cc:224] Handling call
0709 14:53:19.816185 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.817612 (+  1427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.817992 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.851408  7081 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:19.882953  2752 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:19.884128  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1292) took 4004 ms. Trace:
I20260709 14:53:19.884615  2752 rpcz_store.cc:276] 0709 14:53:15.879282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.767545 (+3888263us) service_pool.cc:224] Handling call
0709 14:53:19.768253 (+   708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.883670 (+115417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.883998 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:19.887324  7052 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:19.850912  2680 rpcz_store.cc:276] 0709 14:53:15.766614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.766817 (+   203us) service_pool.cc:224] Handling call
0709 14:53:15.767409 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.765125 (+3997716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.810172 (+ 45047us) spinlock_profiling.cc:243] Waited 3.85 s on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:19.810433 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8864026496}
I20260709 14:53:19.895244  2679 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:19.895453  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:19.896342  2758 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:19.897047  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 799) took 2103 ms. Trace:
I20260709 14:53:19.897486  2679 rpcz_store.cc:276] 0709 14:53:17.793501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.808392 (+2014891us) service_pool.cc:224] Handling call
0709 14:53:19.808941 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.895959 (+ 87018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.896694 (+   735us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:19.896955 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75762560}
I20260709 14:53:19.899421  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:19.900007  7032 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:19.839105  2610 rpcz_store.cc:269] 0709 14:53:15.997392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:16.370110 (+372718us) service_pool.cc:224] Handling call
0709 14:53:19.757929 (+3387819us) spinlock_profiling.cc:243] Waited 3.14 s on lock 0x7b54000ac154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:19.758321 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7224248064}
I20260709 14:53:19.904595  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:19.904943  7096 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:19.905525  2610 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:19.867319  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:19.861691  2603 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:19.846258  2607 rpcz_store.cc:276] 0709 14:53:15.995704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.624687 (+1628983us) service_pool.cc:224] Handling call
0709 14:53:19.770554 (+2145867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.771704 (+  1150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.828615 (+ 56911us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54000ac150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:19.839458 (+ 10843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4822737664}
I20260709 14:53:19.989248  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:19.989847  7170 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:19.849947  6957 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 14:53:19.996909  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 531922us
I20260709 14:53:20.001070  7231 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:20.002754  7032 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:19.997406  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 224228us
I20260709 14:53:20.008221  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.003830  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 754) took 5637 ms. Trace:
I20260709 14:53:19.994781  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:20.005959  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.012827  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.006119  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:19.836104  2611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "963a4155717b4274b0121341056b7cb8"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:20.012302  2609 rpcz_store.cc:276] 0709 14:53:14.358259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.027384 (+669125us) service_pool.cc:224] Handling call
0709 14:53:17.932370 (+2904986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.864631 (+1932261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.906742 (+ 42111us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:19.982600 (+ 75858us) spinlock_profiling.cc:243] Waited 2.9 s on lock 0x7b54004808d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:19.995786 (+ 13186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10801480320}
W20260709 14:53:20.024104  2723 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.906s (SENT)
W20260709 14:53:20.030926  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 803515us
I20260709 14:53:20.031894  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:20.032315  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53637.7us
I20260709 14:53:20.035944  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.037458  3545 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
I20260709 14:53:20.038888  7232 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:20.039889  7232 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:20.042961  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 769) took 5504 ms. Trace:
I20260709 14:53:20.043649  2605 rpcz_store.cc:276] 0709 14:53:14.462277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.395385 (+933108us) service_pool.cc:224] Handling call
0709 14:53:15.410898 (+ 15513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:15.411234 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:16.103682 (+692448us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.734248 (+3630566us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.824583 (+ 90335us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:19.828889 (+  4306us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:19.844710 (+ 15821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:19.850741 (+  6031us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:19.966681 (+115940us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":286893,"prepare.queue_time_us":442335,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":128,"raft.queue_time_us":451618,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":93557,"thread_start_us":4238274,"threads_started":4,"wal-append.queue_time_us":2432613,"wal-append.run_cpu_time_us":1597,"wal-append.run_wall_time_us":1009656}
I20260709 14:53:20.045106  7081 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:20.046039  6977 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:20.047122  6986 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:20.030571  2679 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:20.047904  6992 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 14:53:20.050216  6992 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:20.050710  6992 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:20.032464  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.052942  7237 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:20.053700  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:20.030584  3542 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:20.022083  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:20.057142  2606 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:20.058331  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.060011  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:20.060300  2683 raft_consensus.cc:2376] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13. Re-sending same reply.
I20260709 14:53:20.060442  7170 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:20.061462  7234 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:20.065022  7126 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:20.065534  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.060925  7231 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:20.047964  6986 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:20.070211  6986 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:20.054183  7237 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:20.073206  7237 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:20.058701  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54529us
I20260709 14:53:20.073668  2684 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:20.074519  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 232098us
I20260709 14:53:20.074859  3540 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:20.074990  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:20.075879  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.077129  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:20.078328  2679 raft_consensus.cc:2368] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16.
I20260709 14:53:20.078470  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.077149  6748 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:20.079656  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 199189us
I20260709 14:53:20.081126  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.081702  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:20.082104  2680 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:20.082576  7232 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:20.084969  7041 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:20.046572  6977 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:20.027910  2611 raft_consensus.cc:606] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.107821  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:20.107679  2611 raft_consensus.cc:614] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:20.060103  6969 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:20.122671  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.047963  7081 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:20.090219  7041 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:20.124611  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.120338  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.108628  6977 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:20.131244  2677 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:53:20.119436  2650 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.079s (SENT)
I20260709 14:53:20.135869  2681 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:20.136941  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:20.143280  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:20.135061  7237 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:20.147634  7237 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:20.148159  7237 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:20.151026  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:20.151643  7144 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:20.152315  2759 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:20.153927  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:20.155755  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98013.2us
I20260709 14:53:20.159210  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.163250  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:20.164296  2757 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:20.164834  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:20.168944  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1435) took 5869 ms. Trace:
I20260709 14:53:20.169551  2608 rpcz_store.cc:276] 0709 14:53:14.300087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:15.016808 (+716721us) service_pool.cc:224] Handling call
0709 14:53:15.076853 (+ 60045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.845100 (+4768247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.168480 (+323380us) spinlock_profiling.cc:243] Waited 4.71 s on lock 0x7b54002cdb54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:20.168859 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10823150336}
I20260709 14:53:20.137634  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:20.183761  7233 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 14:53:20.133363  7041 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:20.198740  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.199402  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:20.199856  2758 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:20.216367  7170 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:20.218055  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:20.219667  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 141441us
W20260709 14:53:20.219949  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 154711us
I20260709 14:53:20.221442  2751 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:20.216503  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.226146  2681 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:20.241600  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:20.241914  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.238550  7232 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:20.243925  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.244275  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:20.242576  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.201071  7234 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:20.242578  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.254138  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:20.260195  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.010s	sys 0.028s
W20260709 14:53:20.260783  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.010s	sys 0.028s
W20260709 14:53:20.261191  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.03958e+06us
I20260709 14:53:20.261305  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.261667  2609 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:20.263017  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:20.263381  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.263940  2682 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:20.264421  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.264922  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.266319  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.268694  3409 leader_election.cc:400] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16.
I20260709 14:53:20.269186  3409 leader_election.cc:403] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 14:53:20.273389  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:20.274049  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 785) took 1053 ms. Trace:
I20260709 14:53:20.263278  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:20.274796  2651 leader_election.cc:400] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:53:20.282027  2651 leader_election.cc:403] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 14:53:20.284303  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:20.285388  6995 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 10. Current term is 11.
I20260709 14:53:20.286324  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:20.290143  7144 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:20.295904  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.275949  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:20.312851  2755 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 14:53:20.341768  2755 pending_rounds.cc:85] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:20.277736  7052 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:20.278486  7126 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:20.319847  7242 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.163s	user 0.002s	sys 0.003s
I20260709 14:53:20.320801  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.279103  2680 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:20.277541  6986 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 14:53:20.279498  2677 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:20.352344  2610 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:20.355258  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.357571  3545 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:20.359484  2685 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:20.275408  6967 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:20.274497  2604 rpcz_store.cc:276] 0709 14:53:19.221042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.874544 (+653502us) service_pool.cc:224] Handling call
0709 14:53:19.875600 (+  1056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.084096 (+208496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.084139 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.084847 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 14:53:20.093530 (+  8683us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:20.094107 (+   577us) log.cc:844] Serialized 14 byte log entry
0709 14:53:20.273296 (+179189us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b1800397e40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:53:20.273531 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.273626 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:20.273668 (+    42us) raft_consensus.cc:1685] finished
0709 14:53:20.273808 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:20.273954 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":408669,"prepare.queue_time_us":301,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":404676,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":16298,"spinlock_wait_cycles":10403584,"thread_start_us":8088,"threads_started":1,"wal-append.queue_time_us":8633}
I20260709 14:53:20.324162  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:20.327042  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 210786us
I20260709 14:53:20.256969  6992 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:20.335495  6997 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:20.336725  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.270707  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:20.361251  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.322600  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.363651  2759 pending_rounds.cc:85] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 6
I20260709 14:53:20.382186  2759 pending_rounds.cc:107] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 14:53:20.382673  2759 raft_consensus.cc:2889] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:20.383488  2611 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:20.363862  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.364332  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.364949  7089 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:20.348874  7126 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:20.367760  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.368723  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:20.369488  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.370018  7237 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:20.372700  6986 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:20.374104  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.375010  2755 pending_rounds.cc:107] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 13.8
W20260709 14:53:20.379848  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 798) took 2567 ms (client timeout 2997 ms). Trace:
I20260709 14:53:20.276373  7114 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.231s
I20260709 14:53:20.505723  2751 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 14:53:20.470253  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120432us
I20260709 14:53:20.713024  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:20.507130  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:20.384984  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:20.425465  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 154905us
W20260709 14:53:20.735530  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:20.437700  2678 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:53:20.467547  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.435997  3538 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:20.712232  3545 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:20.446339  3543 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:20.739713  2755 raft_consensus.cc:2889] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:20.512163  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:20.716313  2685 rpcz_store.cc:269] 0709 14:53:17.793161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.639579 (+1846418us) service_pool.cc:224] Handling call
0709 14:53:20.360338 (+720759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:20.742406  3536 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:20.753496  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111535us
I20260709 14:53:20.716813  3541 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 9.
W20260709 14:53:20.791116  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.012s
W20260709 14:53:20.791853  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.001s	sys 0.012s
I20260709 14:53:20.695990  2603 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:20.796759  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:20.573848  7231 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:20.566766  2752 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 14:53:20.377351  6967 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:20.804876  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:20.638103  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.007s
W20260709 14:53:20.810438  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 375092us
W20260709 14:53:20.810943  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:20.816747  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.004s	sys 0.008s
I20260709 14:53:20.737263  6992 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:20.715083  6986 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:20.819979  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65312.1us
I20260709 14:53:20.808311  2683 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 14:53:20.846367  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:20.695365  7047 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:20.468676  7242 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 14:53:20.827917  2609 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:20.831817  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:20.832185  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.832631  2608 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:20.819339  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.380954  6997 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:20.733559  7126 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:20.842535  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.860684  2611 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:20.853645  3536 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:20.871840  2681 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:20.877394  2678 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:20.818384  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.881830  7188 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:20.842553  7231 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:20.883936  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.883572  6997 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:20.887365  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:20.888123  6978 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:20.889696  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.889837  7047 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:20.891463  2756 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: [7.4-10.5]
I20260709 14:53:20.821142  6967 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:20.891474  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.894025  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1297) took 4973 ms. Trace:
I20260709 14:53:20.897541  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1294) took 4980 ms. Trace:
I20260709 14:53:20.920657  2755 rpcz_store.cc:276] 0709 14:53:15.917473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.773255 (+3855782us) service_pool.cc:224] Handling call
0709 14:53:19.773922 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:20.897198 (+1123276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.897467 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:20.900199  7244 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.383s	user 0.003s	sys 0.002s
I20260709 14:53:20.900609  7239 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:20.901139  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:20.850548  6977 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.002s
I20260709 14:53:20.505882  7096 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.358s
I20260709 14:53:20.897384  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:20.858408  7072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.036s	sys 0.083s
W20260709 14:53:20.843780  7210 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.041s	sys 0.049s
I20260709 14:53:20.919384  7098 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:20.922668  2751 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:53:20.924664  2678 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:20.924221  3545 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:20.930598  3543 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:20.944998  2677 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:20.920585  2753 rpcz_store.cc:276] 0709 14:53:15.921109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.787586 (+3866477us) service_pool.cc:224] Handling call
0709 14:53:19.788334 (+   748us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:19.862562 (+ 74228us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:19.863036 (+   474us) log.cc:844] Serialized 14 byte log entry
0709 14:53:19.997018 (+133982us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:20.021480 (+ 24462us) log.cc:844] Serialized 52 byte log entry
0709 14:53:20.021690 (+   210us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:20.021736 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.296315 (+274579us) spinlock_profiling.cc:243] Waited 86.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:20.432745 (+136430us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:20.432819 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:20.893734 (+460915us) raft_consensus.cc:1685] finished
0709 14:53:20.893763 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:20.893947 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24555,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":85,"raft.queue_time_us":5970,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":292,"spinlock_wait_cycles":368694272,"thread_start_us":130258,"threads_started":1,"wal-append.queue_time_us":292699}
I20260709 14:53:20.957262  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.943135  6977 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:20.945221  7210 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.042s	sys 0.049s
W20260709 14:53:20.931385  7041 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.005s
I20260709 14:53:20.961628  6977 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:20.962944  7237 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:20.968294  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:20.968756  6997 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:20.969301  2606 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:20.973523  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.022s	sys 0.047s
I20260709 14:53:20.973701  7096 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 14:53:20.938233  7072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.037s	sys 0.083s
W20260709 14:53:20.977084  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 690913us
I20260709 14:53:20.957079  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1289) took 5122 ms. Trace:
W20260709 14:53:20.979883  2682 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.004s
I20260709 14:53:20.979702  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:20.997829  2682 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:20.977123  2752 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [10.7-13.8]
W20260709 14:53:20.987529  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.022s	sys 0.047s
I20260709 14:53:20.950124  7098 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:20.994570  7238 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.774s	user 0.001s	sys 0.003s
I20260709 14:53:20.996174  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:21.006551  7098 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:21.010947  7032 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:21.011636  7032 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:21.012099  7032 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.017036  6978 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:21.019223  7188 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:21.021591  7072 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:21.021711  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:21.027460  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.020s	sys 0.041s
W20260709 14:53:21.028088  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.020s	sys 0.041s
I20260709 14:53:21.048178  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.022104  7072 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:21.072928  3541 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 14:53:21.080381  7103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.020s	sys 0.051s
W20260709 14:53:21.083267  7103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.021s	sys 0.052s
I20260709 14:53:21.086930  7234 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:21.088253  7234 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.091476  7234 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:20.988682  7041 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.070089  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:20.996556  2760 rpcz_store.cc:276] 0709 14:53:15.834615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.629645 (+1795030us) service_pool.cc:224] Handling call
0709 14:53:17.630002 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.488167 (+1858165us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:19.683286 (+195119us) log.cc:844] Serialized 14 byte log entry
0709 14:53:20.131183 (+447897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:20.137101 (+  5918us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:20.191078 (+ 53977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.870056 (+678978us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18004439e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419
0709 14:53:20.948612 (+ 78556us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:20.948668 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:20.956834 (+  8166us) inbound_call.cc:177] Queueing success response
0709 14:53:20.980422 (+ 23588us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
Metrics: {"prepare.queue_time_us":194998,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":130,"raft.queue_time_us":8215,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":28178,"spinlock_wait_cycles":4330109056,"thread_start_us":620379,"threads_started":2,"wal-append.queue_time_us":458724}
I20260709 14:53:21.123816  3545 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:53:21.060141  7244 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 14:53:21.127879  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.128728  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.133301  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1449) took 1382 ms. Trace:
I20260709 14:53:21.008337  7096 consensus_queue.cc:1243] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:21.137143  3543 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:21.146883  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.119899  7216 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:21.022190  7098 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:21.158447  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.135139  6748 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:21.138198  2607 rpcz_store.cc:276] 0709 14:53:19.794157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.969257 (+175100us) service_pool.cc:224] Handling call
0709 14:53:19.992612 (+ 23355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:20.336285 (+343673us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:20.336897 (+   612us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.070399 (+733502us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:21.070482 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:21.126033 (+ 55551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.126135 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.126310 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":263197,"prepare.queue_time_us":25451,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":41,"raft.queue_time_us":325702,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":5608,"thread_start_us":677839,"threads_started":1}
I20260709 14:53:20.996661  6986 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:21.179138  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1297) took 1121 ms. Trace:
I20260709 14:53:21.183207  2610 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 13. Ops: []
I20260709 14:53:21.183600  3536 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:21.185333  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.185601  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.187575  2603 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 14:53:21.195194  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 199122us
I20260709 14:53:21.198356  3545 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 14:53:21.174861  7231 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:21.179826  3546 rpcz_store.cc:276] 0709 14:53:20.051583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.051829 (+   246us) service_pool.cc:224] Handling call
0709 14:53:20.052562 (+   733us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.053499 (+   937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.053561 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.054436 (+   875us) log.cc:844] Serialized 26 byte log entry
0709 14:53:20.817734 (+763298us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:20.817802 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.817907 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.170775 (+352868us) raft_consensus.cc:1685] finished
0709 14:53:21.170809 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.172817 (+  2008us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6148,"raft.queue_time_us":1906,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":39501,"thread_start_us":762616,"threads_started":1,"wal-append.queue_time_us":1103623}
W20260709 14:53:21.237175  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:21.245519  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:21.201567  7089 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:21.243547  2611 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:21.258740  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.207368  7126 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.269569  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:21.175401  7072 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:21.271461  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:21.269325  2723 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.523s (SENT)
I20260709 14:53:21.275642  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:21.260542  7246 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 14:53:21.254637  7047 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:21.282488  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 487972us
I20260709 14:53:21.276718  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.280436  3545 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 14:53:21.294663  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:21.296170  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:21.302469  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.318323  2609 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:21.313120  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:21.319309  2605 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:21.276191  7251 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 14:53:21.325598  3536 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:21.280977  7041 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:21.329640  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65760us
I20260709 14:53:21.330295  2611 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:21.336886  7098 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:21.337596  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:21.337826  7052 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.989s	user 0.015s	sys 0.000s
I20260709 14:53:21.338524  7052 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16.
I20260709 14:53:21.339432  7052 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:21.339758  2684 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:21.340307  2684 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:21.341048  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.014s	sys 0.003s
W20260709 14:53:21.341516  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.014s	sys 0.003s
I20260709 14:53:21.341377  2684 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:21.343700  7254 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 14:53:21.301474  7231 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:21.346016  7243 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.190s	user 0.009s	sys 0.006s
I20260709 14:53:21.341130  7052 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 14:53:21.347420  7052 consensus_queue.cc:1243] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 14:53:21.347765  7250 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 14:53:21.348172  2684 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:21.349468  7252 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:53:21.354177  7237 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:21.361224  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1467) took 1027 ms. Trace:
I20260709 14:53:21.361943  2603 rpcz_store.cc:276] 0709 14:53:20.333705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.820894 (+487189us) service_pool.cc:224] Handling call
0709 14:53:20.863718 (+ 42824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:21.192952 (+329234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.360771 (+167819us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400128254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:21.361087 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720803712}
I20260709 14:53:21.363353  3542 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:21.365304  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1302) took 1073 ms. Trace:
I20260709 14:53:21.369136  6957 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:21.369838  6957 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:21.372014  7245 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 14:53:21.374192  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.528s (SENT)
I20260709 14:53:21.365799  3542 rpcz_store.cc:276] 0709 14:53:20.291741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.320298 (+ 28557us) service_pool.cc:224] Handling call
0709 14:53:21.365022 (+1044724us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54004b5ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:21.365211 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334314752}
W20260709 14:53:21.375223  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133998us
W20260709 14:53:21.379174  2679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.018s	sys 0.033s
I20260709 14:53:21.380438  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:21.381595  2608 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:21.380892  2679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.018s	sys 0.033s
I20260709 14:53:21.386699  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:21.390959  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 807) took 1307 ms. Trace:
W20260709 14:53:21.395743  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58265.6us
I20260709 14:53:21.354707  7237 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:21.402493  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1240) took 1272 ms. Trace:
I20260709 14:53:21.403040  2684 rpcz_store.cc:276] 0709 14:53:20.123843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.809688 (+685845us) service_pool.cc:224] Handling call
0709 14:53:20.831462 (+ 21774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:21.339606 (+508144us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000c5e20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:53:21.390279 (+ 50673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:21.390332 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:21.390404 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:21.395803 (+  5399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.395999 (+   196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.396161 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14601,"spinlock_wait_cycles":331672960}
I20260709 14:53:21.393916  2679 rpcz_store.cc:276] 0709 14:53:20.083313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.091167 (+  7854us) service_pool.cc:224] Handling call
0709 14:53:20.112592 (+ 21425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:20.185959 (+ 73367us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:20.186361 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.383281 (+1196920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:21.383373 (+    92us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:21.390535 (+  7162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.390654 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.390873 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41962,"prepare.queue_time_us":905,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":175679,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":12414,"thread_start_us":1137672,"threads_started":2,"wal-append.queue_time_us":791635}
I20260709 14:53:21.404915  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:21.391762  7096 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:21.412457  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:21.444906  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 428863us
W20260709 14:53:21.401976  7113 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 14:53:21.462144  7216 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:21.390683  7047 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 14:53:21.459137  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1300) took 1323 ms. Trace:
I20260709 14:53:21.496235  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:21.514889  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106177us
W20260709 14:53:21.516259  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113935us
I20260709 14:53:21.517199  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1290) took 5517 ms. Trace:
W20260709 14:53:21.518887  7257 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 14:53:21.527643  7262 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:21.542476  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 783) took 2322 ms. Trace:
I20260709 14:53:21.468658  3540 rpcz_store.cc:276] 0709 14:53:20.140973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.147540 (+  6567us) service_pool.cc:224] Handling call
0709 14:53:20.215140 (+ 67600us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:20.215736 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.215769 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:20.216710 (+   941us) log.cc:844] Serialized 52 byte log entry
0709 14:53:20.678547 (+461837us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:20.678668 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.723003 (+ 44335us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.458769 (+735766us) raft_consensus.cc:1685] finished
0709 14:53:21.458807 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.459042 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5206,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"thread_start_us":460812,"threads_started":1,"wal-append.queue_time_us":963874}
I20260709 14:53:21.551812  3542 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
W20260709 14:53:21.445888  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 566810us
W20260709 14:53:21.548460  2723 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.601s (SENT)
W20260709 14:53:21.541838  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.664s (SENT)
W20260709 14:53:21.549870  6992 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.006s
I20260709 14:53:21.577423  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:21.573154  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:21.580675  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:21.506086  7089 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.584271  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:21.591459  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 198512us
I20260709 14:53:21.595121  7126 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:21.586866  7256 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.215s	user 0.003s	sys 0.000s
I20260709 14:53:21.541968  2754 rpcz_store.cc:276] 0709 14:53:15.876972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:17.657557 (+1780585us) service_pool.cc:224] Handling call
0709 14:53:17.658187 (+   630us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:19.743240 (+2085053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:19.743300 (+    60us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:19.844283 (+100983us) log.cc:844] Serialized 78 byte log entry
0709 14:53:20.363260 (+518977us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:20.433799 (+ 70539us) log.cc:844] Serialized 14 byte log entry
0709 14:53:20.446207 (+ 12408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:20.506626 (+ 60419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.379719 (+873093us) raft_consensus.cc:1685] finished
0709 14:53:21.379749 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.393833 (+ 14084us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55838,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":92269,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":48351,"thread_start_us":690797,"threads_started":2,"wal-append.queue_time_us":914063}
I20260709 14:53:21.580044  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:21.556447  2612 rpcz_store.cc:276] 0709 14:53:19.220517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.870686 (+650169us) service_pool.cc:224] Handling call
0709 14:53:19.871420 (+   734us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:20.026436 (+155016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.026490 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:20.089926 (+ 63436us) log.cc:844] Serialized 78 byte log entry
0709 14:53:20.566185 (+476259us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:20.730092 (+163907us) log.cc:844] Serialized 14 byte log entry
0709 14:53:20.983094 (+253002us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b180033b4a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:53:20.984502 (+  1408us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.130253 (+145751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.131031 (+   778us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.477825 (+346794us) raft_consensus.cc:1685] finished
0709 14:53:21.477872 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.542366 (+ 64494us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":421878,"prepare.queue_time_us":166291,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":41475,"raft.queue_time_us":559300,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":23549,"spinlock_wait_cycles":60613888,"thread_start_us":401000,"threads_started":1,"wal-append.queue_time_us":1105126}
I20260709 14:53:21.580708  7096 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 14:53:21.596645  2604 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:21.622663  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.626750  3536 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 14:53:21.450796  6957 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:21.634107  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78185.3us
I20260709 14:53:21.634999  3542 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:21.639379  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61733us
W20260709 14:53:21.644311  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:21.620568  7126 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:21.647742  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:21.637970  6997 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.639s	user 0.015s	sys 0.004s
W20260709 14:53:21.636166  6967 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.008s
I20260709 14:53:21.552915  7237 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:21.655333  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:21.652350  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.651403  3536 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
W20260709 14:53:21.653744  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 469688us
I20260709 14:53:21.656858  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:21.656706  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 347235us
W20260709 14:53:21.665652  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45745.5us
I20260709 14:53:21.664741  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:21.660203  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:21.668715  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:21.641786  6992 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 14:53:21.653049  6996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.018s	sys 0.034s
I20260709 14:53:21.646086  7114 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:21.692297  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:21.695148  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.694320  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1295) took 1643 ms. Trace:
I20260709 14:53:21.707278  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:21.692924  6996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.019s	sys 0.034s
I20260709 14:53:21.697058  3547 rpcz_store.cc:276] 0709 14:53:20.050489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.059341 (+  8852us) service_pool.cc:224] Handling call
0709 14:53:20.291985 (+232644us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:53:20.341449 (+ 49464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:20.383796 (+ 42347us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:20.384201 (+   405us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.678186 (+1293985us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.691163 (+ 12977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:21.691262 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:21.691339 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.691449 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.694242 (+  2793us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12508,"prepare.queue_time_us":39161,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":26513,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":212,"spinlock_wait_cycles":53582720,"thread_start_us":1292505,"threads_started":2,"wal-append.queue_time_us":1271176}
I20260709 14:53:21.709080  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.638293  7271 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.063s
W20260709 14:53:21.710258  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:21.776626  2723 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.786s (SENT)
I20260709 14:53:21.694644  6957 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:21.694285  7184 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:21.695966  6992 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:21.733111  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.694473  7210 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 14:53:21.734648  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83702.6us
I20260709 14:53:21.736117  7273 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:21.706241  7216 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 14:53:21.704499  7114 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:21.777443  7237 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:21.755228  7032 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.011s
W20260709 14:53:21.756152  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:21.706132  6977 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:21.759063  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:21.759440  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.760473  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.760739  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.761328  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:21.761590  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:21.762435  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:21.811884  2753 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:21.763195  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.763585  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.764299  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.812459  3540 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:21.764473  2751 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 14:53:21.765542  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58357.3us
I20260709 14:53:21.766218  6997 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:21.768942  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1245) took 1566 ms. Trace:
I20260709 14:53:21.783874  7259 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:21.796262  7216 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:21.852893  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.778982  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:21.780664  2723 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.807s (SENT)
W20260709 14:53:21.783034  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 211382us
W20260709 14:53:21.855216  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74585.1us
I20260709 14:53:21.787567  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:21.856896  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.789134  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:21.792037  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.859292  2679 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:21.859624  3541 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 14:53:21.860967  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.792243  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:21.664136  6967 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.795233  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.864472  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.865401  6967 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:21.795702  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:21.865854  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.866432  2677 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11. Re-sending same reply.
I20260709 14:53:21.803771  7265 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:21.814496  3546 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:21.815625  2685 rpcz_store.cc:276] 0709 14:53:20.202516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.887079 (+684563us) service_pool.cc:224] Handling call
0709 14:53:20.887514 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.888053 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.888088 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.888736 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 14:53:21.226230 (+337494us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:21.226341 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.226943 (+   602us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.768668 (+541725us) raft_consensus.cc:1685] finished
0709 14:53:21.768693 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.768872 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":312147,"threads_started":1,"wal-append.queue_time_us":800554}
I20260709 14:53:21.771721  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 789) took 1395 ms. Trace:
I20260709 14:53:21.813010  3545 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:21.827250  2752 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:21.763195  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.826409  3542 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:21.813417  2751 pending_rounds.cc:85] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
I20260709 14:53:21.830984  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.893219  2758 rpcz_store.cc:276] 0709 14:53:20.376481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.376695 (+   214us) service_pool.cc:224] Handling call
0709 14:53:20.677761 (+301066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.752930 (+ 75169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.752978 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.864630 (+111652us) log.cc:844] Serialized 26 byte log entry
0709 14:53:21.297290 (+432660us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:21.297332 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.297422 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.771459 (+474037us) raft_consensus.cc:1685] finished
0709 14:53:21.771484 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.771651 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":360909,"threads_started":1,"wal-append.queue_time_us":777259}
I20260709 14:53:21.886054  2682 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:21.793609  7273 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:21.810460  7032 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:21.834501  7269 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 14:53:21.843528  2755 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:21.845983  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 791) took 1219 ms. Trace:
I20260709 14:53:21.846968  7047 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:21.848243  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.850443  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:21.851094  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:21.809998  6977 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:21.855388  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96474us
I20260709 14:53:21.791992  2678 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:21.866602  2756 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:21.800767  2754 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:21.870955  7239 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:21.854535  2753 raft_consensus.cc:2410] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:21.858551  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.832211  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:21.895110  2751 pending_rounds.cc:107] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 9.5
W20260709 14:53:21.884874  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 183600us
I20260709 14:53:21.898291  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:21.894914  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 193229us
I20260709 14:53:21.899345  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.903295  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:21.915234  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:21.902230  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98520.9us
I20260709 14:53:21.904315  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.916672  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:21.910917  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:21.916929  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.917584  2760 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.904774  2757 rpcz_store.cc:276] 0709 14:53:20.626397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.626576 (+   179us) service_pool.cc:224] Handling call
0709 14:53:20.805170 (+178594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:21.209588 (+404418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:21.209626 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:21.216280 (+  6654us) log.cc:844] Serialized 26 byte log entry
0709 14:53:21.651633 (+435353us) raft_consensus.cc:1645] Marking committed up to 4
0709 14:53:21.651688 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.651776 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:21.845645 (+193869us) raft_consensus.cc:1685] finished
0709 14:53:21.845676 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:21.845901 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88471,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":689031,"threads_started":2,"wal-append.queue_time_us":573873}
I20260709 14:53:21.916621  7113 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:21.918181  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:21.918733  7263 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:21.919428  7113 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:21.914558  2751 raft_consensus.cc:2889] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:21.919588  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:21.919902  7113 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:21.920389  7113 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:21.920728  7263 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:21.921182  7263 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:21.921692  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:21.921618  7263 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:21.922655  6967 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.923311  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:21.924374  3411 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.681s (SENT)
I20260709 14:53:21.924906  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:21.924994  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:21.926087  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:21.926147  3411 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.684s (SENT)
I20260709 14:53:21.926707  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:21.927453  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.928792  2685 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:21.930188  3547 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:21.943959  6967 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:21.946264  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.874693  7072 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.914006  3543 raft_consensus.cc:2376] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11. Re-sending same reply.
W20260709 14:53:21.960188  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 101618us
I20260709 14:53:21.951983  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:21.953416  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 139350us
I20260709 14:53:21.954780  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:21.952952  7261 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:21.949992  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.900810  7273 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:21.911290  6977 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:21.961501  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:21.965252  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 108661us
I20260709 14:53:21.961804  2756 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:21.928814  7239 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:21.927482  7279 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:21.973882  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:21.968772  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.976171  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.976321  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:21.976923  2756 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:21.976866  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:21.977238  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.946482  7210 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:21.978154  2678 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:21.983330  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133074us
I20260709 14:53:21.976972  2754 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:21.893642  7259 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:21.986471  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:21.964181  7216 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:21.994139  2753 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:21.995555  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:21.996070  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:21.962766  6967 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:21.898294  6992 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.012410  2760 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:22.014235  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:22.015522  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.015903  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:21.979933  7096 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:21.971347  7287 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:22.019587  2678 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:22.020783  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58061.3us
I20260709 14:53:21.991995  6748 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:21.965399  6997 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 14:53:22.031850  2754 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.997792  7272 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:22.015149  6967 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:22.023942  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.029326  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:22.019058  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 73867.2us
I20260709 14:53:22.044917  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1247) took 1721 ms. Trace:
I20260709 14:53:22.044027  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:22.038057  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.045696  2681 rpcz_store.cc:276] 0709 14:53:20.323380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.900092 (+576712us) service_pool.cc:224] Handling call
0709 14:53:20.900574 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.900924 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.900956 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.901600 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 14:53:21.269820 (+368220us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:21.269885 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.269972 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:22.044451 (+774479us) raft_consensus.cc:1685] finished
0709 14:53:22.044482 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:22.044825 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":312040,"threads_started":1,"wal-append.queue_time_us":714210}
W20260709 14:53:22.047281  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59864.1us
I20260709 14:53:22.020063  7096 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:22.061138  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.061259  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:22.047631  6967 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:22.063027  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:22.027374  2760 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:22.065482  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.067407  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.020133  7263 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.038971  7272 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:22.076829  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.077711  2753 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:22.064548  7096 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:21.999953  7279 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.067166  7287 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:22.071367  7216 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:22.087726  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.069366  7271 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 14:53:22.111140  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 184990us
I20260709 14:53:22.117009  2756 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:22.097631  2760 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:22.100667  2679 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:22.118541  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.163431  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 107351us
W20260709 14:53:22.164736  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.165377  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.162858  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.071288  7296 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:22.177373  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.178601  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.179430  2756 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:22.179535  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.180480  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.180884  2760 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:22.182072  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.108296  7285 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 14:53:22.079224  7272 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:22.163547  3411 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.998s (SENT)
I20260709 14:53:22.182209  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.163462  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 246976us
I20260709 14:53:22.111768  6997 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:22.106866  7286 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:22.164284  7271 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:22.189966  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:22.114748  7265 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:22.195770  2753 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:22.196573  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:22.185295  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.200249  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:22.201263  2605 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:22.198859  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.212834  2756 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 14:53:22.213029  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 286971us
W20260709 14:53:22.207472  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 247856us
W20260709 14:53:22.214037  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51331.2us
I20260709 14:53:22.214515  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.215909  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.216555  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.221390  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 204083us
I20260709 14:53:22.106791  6748 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 14:53:22.193394  7287 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:22.240713  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.195739  7032 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:22.248622  2756 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:22.251299  2760 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:22.226562  6996 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:22.291684  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 266894us
W20260709 14:53:22.292346  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:22.294416  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 79730.8us
I20260709 14:53:22.294857  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.294150  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:22.299631  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51559.1us
I20260709 14:53:22.238873  7265 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:22.301782  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.302265  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.247612  7287 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:22.310235  7287 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:22.311906  7259 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:22.313215  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.314316  2607 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:22.300813  6996 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:22.316033  6996 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.316561  7265 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.303408  7032 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:53:22.330241  7258 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.872s	user 0.005s	sys 0.002s
I20260709 14:53:22.305087  7292 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:22.305635  7295 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.218s
I20260709 14:53:22.308552  7279 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:22.292433  7293 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.262s
W20260709 14:53:22.313215  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63976.2us
I20260709 14:53:22.335263  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.303503  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.347288  7292 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:22.347877  7291 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 14:53:22.292657  7286 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:22.349668  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1462) took 2017 ms. Trace:
I20260709 14:53:22.361639  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.363768  2756 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:22.364010  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1463) took 2021 ms (client timeout 1778 ms). Trace:
W20260709 14:53:22.379205  2604 rpcz_store.cc:269] 0709 14:53:20.332309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.429727 (+ 97418us) service_pool.cc:224] Handling call
0709 14:53:22.353384 (+1923657us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:22.353633 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924231552}
I20260709 14:53:22.365149  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:22.371623  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 254586us
I20260709 14:53:22.372675  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 794) took 4591 ms. Trace:
I20260709 14:53:22.370499  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.378302  7286 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:22.385178  2760 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
W20260709 14:53:22.376708  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 306012us
I20260709 14:53:22.378463  2610 rpcz_store.cc:276] 0709 14:53:20.331985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.811317 (+479332us) service_pool.cc:224] Handling call
0709 14:53:20.842938 (+ 31621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:21.183967 (+341029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:22.349307 (+1165340us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:22.349589 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781882112}
I20260709 14:53:22.390908  7286 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:22.367662  7259 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:22.380888  2686 rpcz_store.cc:276] 0709 14:53:17.781247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:19.641314 (+1860067us) service_pool.cc:224] Handling call
0709 14:53:19.642567 (+  1253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:19.785391 (+142824us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:20.120150 (+334759us) log.cc:844] Serialized 14 byte log entry
0709 14:53:21.006994 (+886844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:21.007102 (+   108us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:21.010677 (+  3575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:22.372319 (+1361642us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400269654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:22.372402 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:22.372596 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":178427,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":67,"raft.queue_time_us":302109,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":12179,"spinlock_wait_cycles":310020480,"thread_start_us":1125779,"threads_started":2,"wal-append.queue_time_us":975035,"wal-append.run_cpu_time_us":2983,"wal-append.run_wall_time_us":1171890}
I20260709 14:53:22.392149  6995 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:53:22.403293  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1454) took 2278 ms (client timeout 3021 ms). Trace:
I20260709 14:53:22.403956  7293 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:22.403857  2606 rpcz_store.cc:269] 0709 14:53:20.129877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.136656 (+  6779us) service_pool.cc:224] Handling call
0709 14:53:22.403021 (+2266365us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:22.403217 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860553600}
I20260709 14:53:22.404726  7293 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:22.405431  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.407032  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:22.407697  2756 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:22.392735  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:22.409725  7237 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.600s	user 0.005s	sys 0.007s
I20260709 14:53:22.331944  7216 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:22.411350  7308 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:22.396883  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:22.413192  3411 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.862s (SENT)
I20260709 14:53:22.413650  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:22.415022  7299 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 14:53:22.415966  7237 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:22.410693  7300 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:22.420045  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:22.420259  7300 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.380410  7290 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:22.422338  7309 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:22.422534  7263 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:22.423566  6995 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:22.424330  7188 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:53:22.424508  2726 connection.cc:256] RPC call timeout handler was delayed by -5.3399206799999774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 14:53:22.425201  7301 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:22.425860  7301 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:22.415215  3411 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.865s (SENT)
I20260709 14:53:22.427089  7301 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.335299  7271 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.430548  7271 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:22.436441  7290 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:22.437306  6995 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.437875  7290 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.423369  7307 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 14:53:22.414551  7297 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:22.412303  7308 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:22.427680  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:22.440057  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78535.6us
I20260709 14:53:22.455251  2756 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:22.499063  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.424871  7188 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:22.451337  2752 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.002s
I20260709 14:53:22.521859  2752 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:22.523164  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.524091  2752 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:22.391335  7294 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:22.453389  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 788) took 2115 ms. Trace:
I20260709 14:53:22.525487  2759 rpcz_store.cc:276] 0709 14:53:20.338156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:20.338489 (+   333us) service_pool.cc:224] Handling call
0709 14:53:20.355074 (+ 16585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:20.384213 (+ 29139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:20.384248 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:20.384847 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 14:53:21.134163 (+749316us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:21.134209 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:21.134287 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:22.329226 (+1194939us) spinlock_profiling.cc:243] Waited 9.73 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:22.440878 (+111652us) raft_consensus.cc:1685] finished
0709 14:53:22.440919 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:22.453281 (+ 12362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22378240,"thread_start_us":486893,"threads_started":1,"wal-append.queue_time_us":894307}
I20260709 14:53:22.528450  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.529276  2759 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:22.472311  2726 connection.cc:256] RPC call timeout handler was delayed by -5.10592067999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 14:53:22.531318  2726 connection.cc:256] RPC call timeout handler was delayed by -4.7963206799998943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
W20260709 14:53:22.531677  2726 connection.cc:256] RPC call timeout handler was delayed by -4.6064206799999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23410000000000003s for a response
W20260709 14:53:22.532109  2726 connection.cc:256] RPC call timeout handler was delayed by -3.8405206799999405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31920000000000004s for a response
W20260709 14:53:22.532500  2726 connection.cc:256] RPC call timeout handler was delayed by -2.1863206799998807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.503s for a response
I20260709 14:53:22.535275  7188 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:22.494006  2753 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:22.536419  7298 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:22.536943  7298 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:22.452512  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.538988  7263 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:22.537930  7298 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.508339  2754 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:22.514472  2679 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:22.540612  7298 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:22.541796  2754 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 14:53:22.543298  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90857.9us
I20260709 14:53:22.537532  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.514509  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.516993  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.549458  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.550367  7309 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:22.550940  2755 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 14:53:22.551386  7309 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:22.447052  7300 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:22.551981  7309 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:22.547809  7294 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:22.504645  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.491110  7274 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:22.551411  2755 pending_rounds.cc:85] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 6
I20260709 14:53:22.563644  2755 pending_rounds.cc:107] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 14:53:22.563956  2755 raft_consensus.cc:2889] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:22.553507  2756 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:22.551816  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.504323  7293 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:22.560081  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 144914us
I20260709 14:53:22.530494  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:22.566164  2752 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 13. Ops: [11.9-11.9]
W20260709 14:53:22.556478  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 255363us
W20260709 14:53:22.576633  3411 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.097s (SENT)
I20260709 14:53:22.581578  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:22.575338  2759 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:22.584772  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90093us
I20260709 14:53:22.593279  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.599197  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:22.588789  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 294697us
I20260709 14:53:22.603103  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.558580  7259 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:22.561787  7309 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:22.607621  2754 pending_rounds.cc:85] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 5
W20260709 14:53:22.610219  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.658898  2754 pending_rounds.cc:107] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 14:53:22.657238  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.612573  6986 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:22.613415  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:22.614189  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:22.616978  6996 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:53:22.618281  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1249) took 1614 ms (client timeout 1718 ms). Trace:
I20260709 14:53:22.576139  7300 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:22.627336  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:22.559371  7306 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 14:53:22.633329  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:22.633661  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.634155  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:22.675915  3540 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:22.634717  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.640971  7047 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:22.637938  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:22.656562  7314 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 14:53:22.656955  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.657218  7259 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:22.657578  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.659320  2754 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:22.659762  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:22.641956  6967 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.000s
W20260709 14:53:22.646384  7113 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.725s	user 0.011s	sys 0.004s
W20260709 14:53:22.663137  2682 rpcz_store.cc:269] 0709 14:53:21.003903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:21.778324 (+774421us) service_pool.cc:224] Handling call
0709 14:53:22.618062 (+839738us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:22.618206 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51707264}
W20260709 14:53:22.667393  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 228656us
I20260709 14:53:22.635097  7294 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:22.653396  7072 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.002s
I20260709 14:53:22.683024  2604 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 14:53:22.683640  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90502.3us
I20260709 14:53:22.684332  2606 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:22.687127  2603 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:22.695046  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:22.723666  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 120719us
I20260709 14:53:22.725687  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:22.728166  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101014us
I20260709 14:53:22.725069  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:22.680002  7292 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:22.735213  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.630621  7286 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.748916  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.753464  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:22.757679  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:22.748054  6977 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.765s	user 0.016s	sys 0.000s
W20260709 14:53:22.760254  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 190446us
I20260709 14:53:22.759692  6977 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:22.761212  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:22.695493  7318 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:22.704998  7210 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:22.724087  7274 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:22.767076  2605 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
W20260709 14:53:22.772814  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:22.761432  3411 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.189s (SENT)
I20260709 14:53:22.727672  6992 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:22.775913  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:22.776244  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:22.733994  7089 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:22.730305  7047 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:22.779328  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:22.795877  2679 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:22.793678  2607 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 14:53:22.796661  2607 pending_rounds.cc:85] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:53:22.797009  2607 pending_rounds.cc:107] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 14:53:22.797339  2607 raft_consensus.cc:2889] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:22.704742  6967 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:22.804308  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.774070  7261 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:22.723703  7113 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:22.787061  7052 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:22.773703  6992 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:22.772333  7210 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:22.801934  6957 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.009s	user 0.017s	sys 0.000s
W20260709 14:53:22.883352  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:22.852617  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76771.6us
W20260709 14:53:22.874202  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112778us
I20260709 14:53:22.881904  2677 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:22.781760  7307 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 14:53:22.832576  7293 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:22.808506  7318 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:22.897992  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.929642  2681 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:22.898872  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:22.920394  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 368615us
I20260709 14:53:22.864153  7089 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:22.928083  2723 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.213s (SENT)
I20260709 14:53:22.945137  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:22.883953  6992 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:22.942667  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43918us
I20260709 14:53:22.881522  7210 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:22.954304  3411 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.196s (SENT)
I20260709 14:53:22.957177  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.911391  7052 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:22.973455  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:22.897914  7323 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:22.922092  7293 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:23.005735  7293 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:22.954347  7265 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:23.009734  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.011288  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.928028  6957 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:23.013017  7287 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.003s
I20260709 14:53:23.012914  6957 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.013646  7287 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:22.977070  6977 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:23.014967  7047 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.015455  7309 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:22.988186  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47077us
I20260709 14:53:22.991403  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.991874  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.032804  2611 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:22.992066  7313 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:22.992820  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64834.6us
I20260709 14:53:22.993525  7292 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:22.997897  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:23.019609  7319 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 14:53:23.020303  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:23.047425  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 128610us
W20260709 14:53:23.048468  3411 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.259s (SENT)
I20260709 14:53:23.048974  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:23.020766  7089 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:23.051316  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.052508  2604 raft_consensus.cc:2368] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:23.021370  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1316) took 1223 ms. Trace:
I20260709 14:53:23.022748  7286 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:23.022637  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:22.984664  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.053958  2607 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:22.991461  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.053200  2756 rpcz_store.cc:276] 0709 14:53:21.797909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.564985 (+767076us) service_pool.cc:224] Handling call
0709 14:53:22.592510 (+ 27525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:22.640501 (+ 47991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:22.640551 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:22.665968 (+ 25417us) log.cc:844] Serialized 26 byte log entry
0709 14:53:22.826809 (+160841us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:22.827998 (+  1189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:22.828129 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:23.021112 (+192983us) raft_consensus.cc:1685] finished
0709 14:53:23.021135 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.021289 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43499,"raft.queue_time_us":59967,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":20465,"thread_start_us":98458,"threads_started":1,"wal-append.queue_time_us":327416}
W20260709 14:53:23.035953  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56739.6us
I20260709 14:53:23.031810  2612 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:23.046728  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.057047  2610 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:23.030121  3540 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:23.061797  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:23.063093  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:23.074000  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1326) took 1033 ms. Trace:
I20260709 14:53:23.074751  2756 rpcz_store.cc:276] 0709 14:53:22.040291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.072875 (+1032584us) service_pool.cc:224] Handling call
0709 14:53:23.073304 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.073561 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:23.073588 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:23.073644 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:23.073695 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.073763 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.073919 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:23.085822  2723 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.341s (SENT)
I20260709 14:53:23.111631  3540 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 14:53:23.113083  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.114132  3540 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:23.115043  7273 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.131s	user 0.014s	sys 0.006s
I20260709 14:53:23.115280  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.115671  7273 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.031687  7317 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:23.118772  7317 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.121546  7317 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.063827  7279 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 14:53:23.122973  7188 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.575s	user 0.006s	sys 0.008s
I20260709 14:53:23.123486  7113 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:23.123639  7188 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:23.124560  7239 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.138s	user 0.012s	sys 0.002s
I20260709 14:53:23.125309  2603 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:23.125043  7239 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.122959  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.128124  2685 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:22.941582  7318 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.130960  3541 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:23.114934  7306 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:23.131772  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.138768  7309 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:23.141731  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.142262  6992 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:23.143628  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.143841  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1315) took 1356 ms. Trace:
I20260709 14:53:23.144542  2604 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:23.144984  6992 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:23.141628  7293 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.106101  7274 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.147080  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.148870  2608 raft_consensus.cc:2368] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:23.149060  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.149420  7263 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:23.149914  7263 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:23.142294  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.151127  7052 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.142748  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:23.153270  7321 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:53:23.144299  2753 rpcz_store.cc:276] 0709 14:53:21.787506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.557483 (+769977us) service_pool.cc:224] Handling call
0709 14:53:22.557951 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:22.558225 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:22.558244 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:22.570071 (+ 11827us) log.cc:844] Serialized 26 byte log entry
0709 14:53:22.735750 (+165679us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:22.735809 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:22.735919 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:23.143570 (+407651us) raft_consensus.cc:1685] finished
0709 14:53:23.143594 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.143737 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":150998,"threads_started":1,"wal-append.queue_time_us":454901}
I20260709 14:53:23.155486  7237 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:23.123080  7297 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 14:53:23.146957  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.156282  7237 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.157020  2751 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:23.157173  7263 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:23.158772  2751 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:23.157035  2606 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:23.159551  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1323) took 1189 ms. Trace:
I20260709 14:53:23.113050  6967 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.137861  7188 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:23.176213  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.146937  7306 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:23.180998  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.182356  2604 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:23.186148  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:23.169823  7337 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 14:53:23.191550  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54784.1us
I20260709 14:53:23.192027  2606 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:23.158946  7263 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:23.192772  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.195973  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.155471  7114 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.189756  3540 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:23.211977  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.173727  6957 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:23.168040  7113 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 14:53:23.229677  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 508064us
I20260709 14:53:23.189736  7273 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:23.240036  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44266.4us
I20260709 14:53:23.239842  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.245409  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.256701  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.257740  3547 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:23.211045  7293 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:23.226075  7096 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.111s	user 0.013s	sys 0.003s
W20260709 14:53:23.263960  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:23.273311  3542 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:23.198148  7317 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 14:53:23.279800  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a57a97528f31400dbc0900e76292b021"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
W20260709 14:53:23.224862  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 499836us
W20260709 14:53:23.259948  7272 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.046s	user 0.012s	sys 0.004s
I20260709 14:53:23.285701  7292 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:23.286628  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:23.289155  7326 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 14:53:23.289691  7302 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.293567  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.315339  3545 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:23.294566  6992 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 14:53:23.296229  2604 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 14:53:23.257890  7188 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:23.299352  7239 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:23.210604  7266 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 14:53:23.302048  7341 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:23.310214  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:23.303224  7311 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:23.298133  7315 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:23.327086  7292 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.324113  7188 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.329942  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:23.331970  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45479.1us
W20260709 14:53:23.342370  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71541.9us
I20260709 14:53:23.327869  7266 consensus_queue.cc:1243] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:23.335737  7311 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:23.276944  7096 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.320622  7300 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:23.359870  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:23.355255  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45148.6us
I20260709 14:53:23.331173  7287 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:23.311335  7272 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.387264  2682 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:23.331573  7216 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 14:53:23.357679  6995 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:23.385113  7271 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.954s	user 0.017s	sys 0.000s
W20260709 14:53:23.396772  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73337us
I20260709 14:53:23.410096  3537 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:23.410581  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:23.324954  7306 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.396775  7317 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:23.434350  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:23.431293  7298 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.890s	user 0.016s	sys 0.000s
I20260709 14:53:23.460392  3543 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:53:23.429612  7311 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:23.468616  2607 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 14:53:23.472566  7348 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 14:53:23.474174  7237 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:23.474644  7351 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 14:53:23.436736  7274 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.475128  6996 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:23.475754  6996 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.412711  6995 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.479455  6995 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.484108  7096 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.486806  7188 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:23.491700  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 69102.8us
I20260709 14:53:23.452518  7318 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:23.493341  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:23.495518  3536 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:23.501571  3543 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:23.507884  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.490996  7272 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:23.501117  7210 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.003s
I20260709 14:53:23.511411  7239 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:53:23.500844  7266 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:23.517192  7210 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.518369  2606 raft_consensus.cc:606] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.518816  2606 raft_consensus.cc:614] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:23.521209  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.529009  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:23.530057  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1321) took 1622 ms. Trace:
I20260709 14:53:23.532099  6978 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:23.532341  7237 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:23.532670  6978 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:23.532805  7237 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:23.533279  7237 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:23.533731  6978 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.521288  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1257) took 1439 ms. Trace:
I20260709 14:53:23.537300  2681 rpcz_store.cc:276] 0709 14:53:22.082120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.082258 (+   138us) service_pool.cc:224] Handling call
0709 14:53:22.107451 (+ 25193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:22.930450 (+822999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.520986 (+590536us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:23.521214 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890049920}
I20260709 14:53:23.540695  7311 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:23.530670  2759 rpcz_store.cc:276] 0709 14:53:21.907659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.584077 (+676418us) service_pool.cc:224] Handling call
0709 14:53:22.648804 (+ 64727us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:22.697098 (+ 48294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:22.697135 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:22.725655 (+ 28520us) log.cc:844] Serialized 52 byte log entry
0709 14:53:23.060928 (+335273us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b180022b6a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:53:23.355373 (+294445us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:23.355494 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.355654 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:23.529598 (+173944us) raft_consensus.cc:1685] finished
0709 14:53:23.529625 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.529930 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70237,"raft.queue_time_us":79241,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":16481,"spinlock_wait_cycles":8535936,"thread_start_us":285272,"threads_started":1,"wal-append.queue_time_us":750299}
I20260709 14:53:23.547748  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:23.506893  7032 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:23.553198  3537 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:23.560731  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:23.562815  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:23.565119  7320 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.682s	user 0.003s	sys 0.000s
I20260709 14:53:23.551785  7032 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:23.560052  7311 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.561007  6978 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.575696  7317 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:23.576363  7317 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.576285  7298 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:23.567543  7341 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:23.577535  7216 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:23.579043  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1318) took 1750 ms. Trace:
I20260709 14:53:23.579568  7032 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.580838  7032 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.579468  2754 rpcz_store.cc:276] 0709 14:53:21.829239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.570262 (+741023us) service_pool.cc:224] Handling call
0709 14:53:22.607207 (+ 36945us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:22.775954 (+168747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:22.776006 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:22.776847 (+   841us) log.cc:844] Serialized 78 byte log entry
0709 14:53:23.177192 (+400345us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:23.221910 (+ 44718us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.222462 (+   552us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.222653 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.222772 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:23.477691 (+254919us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:23.578723 (+101032us) raft_consensus.cc:1685] finished
0709 14:53:23.578750 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.578967 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100054,"prepare.queue_time_us":121340,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":8633,"raft.queue_time_us":119521,"raft.run_cpu_time_us":658,"raft.run_wall_time_us":68499,"spinlock_wait_cycles":11192192,"thread_start_us":285456,"threads_started":1,"wal-append.queue_time_us":550270}
I20260709 14:53:23.582373  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:23.577952  7341 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:23.591516  7126 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:23.592239  7126 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.592742  7290 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:23.593305  7290 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:23.595727  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:23.596673  7263 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:53:23.596915  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:23.598687  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.601441  2683 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:23.600203  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.603223  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.603864  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.606258  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.606413  2685 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:23.607405  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.164959  2760 rpcz_store.cc:276] 0709 14:53:21.969808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.622377 (+652569us) service_pool.cc:224] Handling call
0709 14:53:22.622862 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:22.623150 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:22.869847 (+246697us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.158953 (+289106us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.159189 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:23.159247 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:23.159303 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.159368 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.159490 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101234,"prepare.queue_time_us":494029,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":2354,"raft.queue_time_us":6731,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":13238,"thread_start_us":422592,"threads_started":2}
I20260709 14:53:23.607704  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.616778  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.618079  2685 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:23.621513  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.622922  2607 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:23.624020  7272 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:23.627650  7303 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:23.628115  2604 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 14:53:23.628378  7303 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.629964  2608 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:23.630318  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.633677  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.636227  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.639839  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:23.588671  7341 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.642633  3546 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:23.650076  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:23.650884  3547 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.650964  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:23.650445  2609 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 14:53:23.659480  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.594319  7290 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.657279  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:23.667399  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.669248  7303 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:23.673545  2681 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:53:23.685144  2685 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:23.687342  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.688479  7216 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:23.644176  7126 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:23.690145  7114 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:23.631230  7301 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:23.628216  7272 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.693430  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.695254  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.728751  2677 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:23.624949  7298 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:23.729681  7298 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:23.729750  7272 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:23.730744  7298 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.726096  7301 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:23.732442  7298 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:23.703298  2611 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:23.734225  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.735383  2611 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:23.734543  7301 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:23.736961  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.742730  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:23.702940  2754 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:23.719349  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:23.631731  7210 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:23.723181  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.732544  2605 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:23.746966  3543 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:23.752311  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.749783  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.645022  7263 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:23.746941  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:23.756063  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:23.758443  2607 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:23.710194  2603 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:23.759353  2604 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:23.752193  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:23.774232  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:23.773253  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 74494us
I20260709 14:53:23.783059  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.778242  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:23.763010  7296 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:23.800891  2605 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:23.741542  7096 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 14:53:23.773857  7052 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:23.753702  7265 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 14:53:23.816397  2683 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:23.814954  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.819152  2684 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:23.827356  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.761260  6967 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 14:53:23.811411  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1444) took 1498 ms. Trace:
I20260709 14:53:23.831489  2754 rpcz_store.cc:276] 0709 14:53:22.312526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.790146 (+1477620us) service_pool.cc:224] Handling call
0709 14:53:23.810869 (+ 20723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.811068 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:23.811093 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:23.811139 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:23.811180 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.811223 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:23.811342 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:23.832175  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:23.791165  7273 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:23.818616  6995 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:23.849133  7296 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:23.809693  7052 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:23.868119  2754 pending_rounds.cc:85] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 3
I20260709 14:53:23.873490  2754 pending_rounds.cc:107] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 14:53:23.874553  2754 raft_consensus.cc:2889] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:23.875412  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:23.876256  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.879591  2686 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:23.878058  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.880517  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:23.881811  2680 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:23.818121  7237 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:23.838690  7273 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:23.897696  7273 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:23.905413  7296 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:23.861210  6995 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:23.913184  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:23.878147  7052 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:23.980834  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:23.985272  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74038.9us
I20260709 14:53:23.989699  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.997212  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.999804  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:23.993947  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:24.000140  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:24.005951  2683 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:24.005231  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:24.008734  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:23.994786  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 387451us
I20260709 14:53:24.020964  7126 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:24.032895  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:24.009434  6977 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:24.044346  6977 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:24.045629  7293 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:24.046214  7293 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:24.047583  7216 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:24.047343  7293 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:24.048085  7216 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:24.048507  7216 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:23.867074  7263 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.038128  2760 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:24.062578  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1447) took 1749 ms. Trace:
I20260709 14:53:24.063766  2760 rpcz_store.cc:276] 0709 14:53:22.313520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.037384 (+1723864us) service_pool.cc:224] Handling call
0709 14:53:24.037845 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:24.062086 (+ 24241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.062473 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:24.069744  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:24.071215  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 644957us
I20260709 14:53:24.072791  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:24.076960  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1319) took 2183 ms. Trace:
I20260709 14:53:24.077685  2752 rpcz_store.cc:276] 0709 14:53:21.893983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.573635 (+679652us) service_pool.cc:224] Handling call
0709 14:53:22.574172 (+   537us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:22.587027 (+ 12855us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:22.768132 (+181105us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.280386 (+512254us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.325397 (+ 45011us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:23.331851 (+  6454us) log.cc:844] Serialized 78 byte log entry
0709 14:53:23.332062 (+   211us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:23.339357 (+  7295us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.340921 (+  1564us) log.cc:844] Serialized 14 byte log entry
0709 14:53:24.044601 (+703680us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b1800253540. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:53:24.076388 (+ 31787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.076539 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:24.076619 (+    80us) raft_consensus.cc:1685] finished
0709 14:53:24.076643 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.076874 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":978974,"prepare.queue_time_us":400629,"prepare.run_cpu_time_us":210,"prepare.run_wall_time_us":218,"raft.queue_time_us":772303,"raft.run_cpu_time_us":767,"raft.run_wall_time_us":66033,"spinlock_wait_cycles":37626240,"thread_start_us":500932,"threads_started":2,"wal-append.queue_time_us":560464}
I20260709 14:53:23.994730  6992 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 14:53:24.097707  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 746600us
I20260709 14:53:24.093616  7265 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:24.101490  7265 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:24.101953  7265 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:24.108242  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1439) took 1862 ms. Trace:
I20260709 14:53:24.108997  2758 rpcz_store.cc:276] 0709 14:53:22.245933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.417358 (+1171425us) service_pool.cc:224] Handling call
0709 14:53:23.431513 (+ 14155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.436211 (+  4698us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:23.467013 (+ 30802us) log.cc:844] Serialized 14 byte log entry
0709 14:53:24.107679 (+640666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:24.107761 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:24.107832 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.107926 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.108148 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":551263,"prepare.queue_time_us":30745,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"thread_start_us":78107,"threads_started":1,"wal-append.queue_time_us":245206}
I20260709 14:53:24.116053  2609 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:24.052527  7272 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:24.122555  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:24.124372  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 732868us
I20260709 14:53:24.130379  2605 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:24.132853  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:24.134055  2752 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:24.009893  7301 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 14:53:24.095849  7302 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:24.163666  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:24.093962  7290 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:24.155287  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:24.164330  2612 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 14:53:24.159374  7372 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 14:53:24.159936  7373 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 14:53:24.164502  2758 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:24.160329  7367 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 14:53:24.160710  7376 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 14:53:24.161084  7371 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 14:53:24.161485  7380 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 14:53:24.161866  7310 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 14:53:24.257707  7388 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 14:53:24.158789  7370 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 14:53:24.200167  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45023us
I20260709 14:53:24.259357  2726 leader_election.cc:400] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:24.259795  2726 leader_election.cc:403] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 14:53:24.168255  7032 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.002s
W20260709 14:53:24.192806  7354 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 14:53:24.094658  7318 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 14:53:24.045722  6977 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:24.167480  7308 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 14:53:24.145402  7271 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 14:53:24.291884  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1434) took 2051 ms. Trace:
I20260709 14:53:24.233237  7305 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:24.164273  7302 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:24.285907  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:24.135191  7265 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:24.195210  7358 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.279592  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1438) took 2036 ms. Trace:
W20260709 14:53:24.311946  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:24.280369  7259 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 14:53:24.292570  2756 rpcz_store.cc:276] 0709 14:53:22.240674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.076908 (+836234us) service_pool.cc:224] Handling call
0709 14:53:23.131433 (+ 54525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.131674 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:23.206714 (+ 75040us) log.cc:844] Serialized 14 byte log entry
0709 14:53:24.291275 (+1084561us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:24.291367 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:24.291482 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.291608 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.291782 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":715846,"prepare.queue_time_us":61030,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":5385,"raft.queue_time_us":652915,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9972,"thread_start_us":400806,"threads_started":2,"wal-append.queue_time_us":283202}
W20260709 14:53:24.313201  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 266583us
I20260709 14:53:24.315375  2751 rpcz_store.cc:276] 0709 14:53:22.243290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.188146 (+944856us) service_pool.cc:224] Handling call
0709 14:53:23.188554 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.188816 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:23.331347 (+142531us) log.cc:844] Serialized 14 byte log entry
0709 14:53:24.265022 (+933675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:24.279030 (+ 14008us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:24.279117 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.279241 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.279479 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":778626,"prepare.queue_time_us":146648,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":658189,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":17341,"thread_start_us":286225,"threads_started":2,"wal-append.queue_time_us":224685}
I20260709 14:53:24.312824  6992 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:24.331956  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:24.300318  7379 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 14:53:24.331446  6992 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:24.335855  7382 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 14:53:24.337334  2681 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 14:53:24.337924  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:24.338899  2756 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:24.330003  3547 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.000s
I20260709 14:53:24.342820  3547 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:24.310966  7302 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:53:24.377810  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:24.380496  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1322) took 2424 ms. Trace:
I20260709 14:53:24.383301  2756 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:24.383924  7237 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:24.385068  7295 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.385027  7358 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.386240  7295 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:24.288910  7297 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11.
I20260709 14:53:24.391173  7295 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:24.395618  7265 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:24.381220  2755 rpcz_store.cc:276] 0709 14:53:21.955815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.614681 (+658866us) service_pool.cc:224] Handling call
0709 14:53:22.615120 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:22.661809 (+ 46689us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:22.912731 (+250922us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.361575 (+448844us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.387010 (+ 25435us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.604571 (+217561us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:53:24.071252 (+466681us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:24.075940 (+  4688us) log.cc:844] Serialized 26 byte log entry
0709 14:53:24.076176 (+   236us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:24.076222 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.102196 (+ 25974us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:24.380120 (+277924us) raft_consensus.cc:1685] finished
0709 14:53:24.380158 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.380391 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1243079,"prepare.queue_time_us":396574,"prepare.run_cpu_time_us":201,"prepare.run_wall_time_us":2425,"raft.queue_time_us":1441481,"raft.run_cpu_time_us":4982,"raft.run_wall_time_us":974267,"spinlock_wait_cycles":29576960,"thread_start_us":546854,"threads_started":2,"wal-append.queue_time_us":483582}
I20260709 14:53:24.407761  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:24.409314  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:24.410396  2755 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 14:53:24.397311  6978 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.810s	user 0.014s	sys 0.000s
I20260709 14:53:24.403491  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:24.410390  2756 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:24.423786  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 164650us
I20260709 14:53:24.424783  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:24.458660  2686 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
I20260709 14:53:24.459399  2686 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:24.460357  2678 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:24.463558  2680 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:24.464902  2679 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [9.6-13.8]
I20260709 14:53:24.468690  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:24.473031  2755 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 14:53:24.474253  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1328) took 1053 ms. Trace:
I20260709 14:53:24.476292  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:24.478380  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 484517us
I20260709 14:53:24.390486  7358 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:24.480301  2752 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:24.474781  2755 rpcz_store.cc:276] 0709 14:53:23.420664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.412182 (+991518us) service_pool.cc:224] Handling call
0709 14:53:24.472657 (+ 60475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:24.473849 (+  1192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.474170 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:24.483624  2756 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:24.484243  2603 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:24.412313  7301 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:24.487212  2603 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:24.485770  2758 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 14:53:24.418761  7296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.015s	sys 0.023s
I20260709 14:53:24.490489  2752 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:24.470600  7292 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:24.478209  7295 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:24.497125  7295 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:24.497429  2610 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:24.497745  7295 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:24.491551  2603 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 14:53:24.493472  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:24.503139  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 450915us
I20260709 14:53:24.477986  7297 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:24.501550  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:24.513047  2755 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:24.537498  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1332) took 1115 ms. Trace:
I20260709 14:53:24.538282  2755 rpcz_store.cc:276] 0709 14:53:23.422453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.482959 (+1060506us) service_pool.cc:224] Handling call
0709 14:53:24.512745 (+ 29786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:24.537132 (+ 24387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.537412 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:24.477406  7318 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.335835  6992 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:24.494410  7296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.016s	sys 0.023s
I20260709 14:53:24.500792  7358 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:24.585989  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1324) took 2600 ms. Trace:
I20260709 14:53:24.566643  7309 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:24.586673  2757 rpcz_store.cc:276] 0709 14:53:21.985999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.048645 (+1062646us) service_pool.cc:224] Handling call
0709 14:53:23.049045 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.049337 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:23.066199 (+ 16862us) log.cc:844] Serialized 14 byte log entry
0709 14:53:23.899063 (+832864us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b1800119fe0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:24.553087 (+654024us) log.cc:844] Serialized 14 byte log entry
0709 14:53:24.585371 (+ 32284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:24.585470 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:24.585557 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.585678 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.585886 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91670,"prepare.queue_time_us":14995,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":68229,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":31278,"spinlock_wait_cycles":50764288,"thread_start_us":595728,"threads_started":2,"wal-append.queue_time_us":1132316}
I20260709 14:53:24.591658  2680 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:24.583158  2610 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 14:53:24.518966  7396 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:24.568720  7294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.019s	sys 0.046s
W20260709 14:53:24.603389  7294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.020s	sys 0.046s
I20260709 14:53:24.604463  3545 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:24.604736  2757 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 14:53:24.602435  2610 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 14:53:24.606041  7216 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.007s
I20260709 14:53:24.610070  7305 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:24.600569  7309 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:24.617888  2757 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:24.499908  7301 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:24.622004  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 145827us
W20260709 14:53:24.623646  7273 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.725s	user 0.014s	sys 0.001s
I20260709 14:53:24.640851  7273 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.641583  7096 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:24.642307  7096 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:24.645061  7293 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:24.634021  2726 leader_election.cc:400] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:24.649688  2726 leader_election.cc:403] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:53:24.639155  7052 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.002s
W20260709 14:53:24.626219  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.000s	sys 0.027s
I20260709 14:53:24.658524  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:24.659588  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.000s	sys 0.028s
W20260709 14:53:24.660606  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 747460us
W20260709 14:53:24.626572  7317 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.049s	user 0.008s	sys 0.008s
I20260709 14:53:24.666998  2686 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:24.646246  7309 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:24.521191  2758 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:24.587466  6992 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 14:53:24.702593  2680 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
W20260709 14:53:24.702773  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 372897us
I20260709 14:53:24.701735  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:24.707007  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:24.678190  2757 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 14:53:24.711899  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.031s	sys 0.070s
W20260709 14:53:24.714233  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.031s	sys 0.070s
I20260709 14:53:24.554574  7318 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:24.672430  7358 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 13
I20260709 14:53:24.718089  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:24.625178  7216 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.701897  7296 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:24.701706  7311 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.707023  7305 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:24.776391  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43912.1us
I20260709 14:53:24.669389  7052 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.719275  7309 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:24.617022  7396 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:24.789601  7341 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.088s	user 0.012s	sys 0.005s
W20260709 14:53:24.733016  7298 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.978s	user 0.013s	sys 0.003s
W20260709 14:53:24.786077  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.019s	sys 0.049s
W20260709 14:53:24.809535  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.019s	sys 0.050s
I20260709 14:53:24.695371  7317 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:24.827049  7391 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 14:53:24.827983  7395 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.830070  2757 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:24.830500  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:24.873121  2758 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:24.832891  7302 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:24.844537  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1307) took 2088 ms. Trace:
I20260709 14:53:24.877319  2682 rpcz_store.cc:276] 0709 14:53:22.755889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:22.757536 (+  1647us) service_pool.cc:224] Handling call
0709 14:53:22.758093 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:23.388192 (+630099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.844031 (+1455839us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:24.844427 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445926656}
I20260709 14:53:24.848429  6748 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 14:53:24.878938  3409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.156s (SENT) [suppressed 50 similar messages]
I20260709 14:53:24.848800  7300 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:24.851009  7216 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:24.851555  7381 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.052s	user 0.003s	sys 0.002s
I20260709 14:53:24.852464  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:24.875123  2612 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:24.907722  2685 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:24.848711  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 190317us
I20260709 14:53:24.908913  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:24.848464  7239 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:24.873528  7309 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:24.880085  6748 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:24.901327  7404 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 14:53:24.901886  7403 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 14:53:24.903925  7300 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:24.731166  7096 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:24.787518  7296 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:24.787976  7311 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:24.925657  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 408777us
I20260709 14:53:24.925261  2685 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:24.951452  2726 leader_election.cc:400] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:24.951995  2726 leader_election.cc:403] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:24.800943  7032 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:24.949651  7305 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:24.964301  6748 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:24.959977  7300 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:24.965636  6748 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:24.966399  7300 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:24.972146  7361 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.496s	user 0.004s	sys 0.007s
I20260709 14:53:24.973150  7311 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:24.973620  7096 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:24.973771  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1435) took 2732 ms. Trace:
I20260709 14:53:24.974326  2753 rpcz_store.cc:276] 0709 14:53:22.241002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.155044 (+914042us) service_pool.cc:224] Handling call
0709 14:53:23.155546 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:23.159088 (+  3542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:23.159116 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:23.159638 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 14:53:23.393030 (+233392us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:23.393102 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:23.393219 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:24.971235 (+1578016us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:24.973390 (+  2155us) raft_consensus.cc:1685] finished
0709 14:53:24.973418 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:24.973695 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26756352,"thread_start_us":232847,"threads_started":1,"wal-append.queue_time_us":280679}
I20260709 14:53:24.982658  2607 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:24.984486  7305 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:24.990206  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:24.990705  2610 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:24.991006  2609 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:24.991777  7400 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:24.992419  7400 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:24.993369  6995 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:24.997061  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:24.997915  2685 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:25.004913  2609 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:24.989946  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:25.008679  2605 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:24.980171  7290 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.784s	user 0.005s	sys 0.010s
W20260709 14:53:25.043136  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43850.2us
I20260709 14:53:24.961748  7273 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.076704  2677 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:24.876765  7395 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:25.048501  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 97265.1us
I20260709 14:53:24.921499  7295 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:25.077924  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:25.049355  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.075758  7403 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
W20260709 14:53:25.086018  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.091809  7318 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.094631  7392 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.095141  6995 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:25.095625  7392 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.048156  7032 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.047029  7378 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 14:53:25.099748  7032 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.100473  7378 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:25.100940  7378 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:25.101523  7402 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 14:53:25.101927  7378 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.110313  7378 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:25.111804  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:25.102641  7032 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:24.920486  7239 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.117142  7113 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:25.103638  6996 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.115732  2605 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:25.104501  7305 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:25.105245  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.126777  7047 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.153309  2753 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:25.105667  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.076159  7311 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:24.875646  7341 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.103889  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.154984  7311 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:25.156009  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52208.3us
I20260709 14:53:25.156713  7341 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.117738  7113 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:25.157404  7311 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.157625  7113 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:25.105300  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:25.128197  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46410.3us
I20260709 14:53:25.157701  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:25.163332  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.128882  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1153) took 3315 ms. Trace:
I20260709 14:53:25.132210  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:25.138240  2759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.076s	sys 0.100s
W20260709 14:53:25.167551  2759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.077s	sys 0.101s
I20260709 14:53:25.118235  7114 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:25.169770  7089 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:25.170363  7089 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:25.158866  7318 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 14:53:25.171558  7341 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:25.171590  7089 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.172312  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:25.172263  7341 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.173149  7089 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:25.173893  7292 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:25.174444  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:25.152586  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.176560  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.114562  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:25.171758  7318 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:25.177889  7318 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:25.178393  7114 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:25.179093  7114 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:25.179152  7301 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.180408  7114 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.181150  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1448) took 2864 ms. Trace:
I20260709 14:53:25.181908  7341 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.182034  7309 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.184444  7113 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:25.177912  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63432.7us
I20260709 14:53:25.187530  7113 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.188946  7405 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.189662  7405 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.191879  7296 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.048403  7408 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.163s
I20260709 14:53:25.193293  7405 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.193737  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1331) took 1771 ms. Trace:
I20260709 14:53:25.195540  7396 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.146922  7239 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.199025  7239 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:25.199972  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.201977  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:25.203605  7317 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.204385  7317 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:25.205854  7408 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:25.205988  7274 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:25.206662  7408 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:25.207002  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.208011  7408 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.208755  6996 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:25.208994  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.210114  7052 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.211145  7325 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:25.211956  7325 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:25.204880  7405 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:25.212459  7325 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:25.212826  7341 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:25.213310  7341 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.200127  7239 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:25.213548  7325 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.214413  7239 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.218235  7301 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:25.217795  7239 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.214269  7341 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.194188  2752 rpcz_store.cc:276] 0709 14:53:23.422064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.481391 (+1059327us) service_pool.cc:224] Handling call
0709 14:53:24.490205 (+  8814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:24.495215 (+  5010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.193492 (+698277us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b54003363d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:25.193665 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19155840}
I20260709 14:53:25.222543  7047 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.223485  7239 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:25.223587  7341 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:25.225715  7387 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:25.226802  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.227633  7387 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 14:53:25.229225  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:25.230223  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:25.231114  7294 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:25.231511  7294 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.232363  7294 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.232908  7052 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:25.233518  7294 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:25.233771  7341 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:25.217749  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:25.233788  7052 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.235817  7052 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.234423  7341 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.238947  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:25.181614  2760 rpcz_store.cc:276] 0709 14:53:22.317026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.065923 (+1748897us) service_pool.cc:224] Handling call
0709 14:53:24.066495 (+   572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:24.066864 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:24.066906 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:24.097305 (+ 30399us) log.cc:844] Serialized 26 byte log entry
0709 14:53:24.461853 (+364548us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:24.461920 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:24.462049 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:25.100681 (+638632us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:25.153545 (+ 52864us) raft_consensus.cc:1685] finished
0709 14:53:25.153570 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:25.181074 (+ 27504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11250944,"thread_start_us":314704,"threads_started":1,"wal-append.queue_time_us":577603}
I20260709 14:53:25.241528  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.242336  2753 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:25.243507  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83471.8us
I20260709 14:53:25.242367  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.244649  2752 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:25.214936  7308 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:25.166225  3541 rpcz_store.cc:276] 0709 14:53:21.761073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:21.861933 (+100860us) service_pool.cc:224] Handling call
0709 14:53:21.862267 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:23.134727 (+1272460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.076401 (+1941674us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:25.076711 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2911036672}
W20260709 14:53:25.218503  7384 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.423s	user 0.000s	sys 0.008s
I20260709 14:53:25.248808  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:25.136202  7273 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:25.282428  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51605.4us
I20260709 14:53:25.262045  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.293704  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:25.251137  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 88669.6us
I20260709 14:53:25.291815  3540 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 14:53:25.292246  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53076.3us
I20260709 14:53:25.296314  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.300658  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.301411  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:25.300122  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.157464  7297 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:25.152818  7271 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.306272  7271 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.308763  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.310439  6996 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:25.311332  7416 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:25.312064  7411 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:25.308812  7271 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.312700  7411 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.315805  7271 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:25.303157  3540 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:25.301291  3538 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [8.6-13.8]
I20260709 14:53:25.317457  7411 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.317627  7413 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:25.318377  7413 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:25.318625  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.319404  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:25.319630  3538 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:25.320134  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 8.
I20260709 14:53:25.319523  7413 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.320958  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.321525  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:25.322285  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.304382  7126 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:25.205755  7302 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.305593  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.302155  6978 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:25.304678  7306 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:25.326020  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.345283  3542 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:25.284183  7311 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.144874  7032 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.302479  7396 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:25.329031  3536 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:25.324573  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.314905  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:25.322742  7188 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:25.348244  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.348686  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1445) took 3035 ms. Trace:
I20260709 14:53:25.342715  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:25.354401  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.286090  7308 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:25.330058  3538 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:25.419679  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.420848  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.422592  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:25.424810  7188 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:25.426090  7188 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
W20260709 14:53:25.424906  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.005s	sys 0.021s
W20260709 14:53:25.432426  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.021s
I20260709 14:53:25.303020  7273 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.422004  7113 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:25.432888  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 553953us
I20260709 14:53:25.424976  2754 rpcz_store.cc:276] 0709 14:53:22.312829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.832582 (+1519753us) service_pool.cc:224] Handling call
0709 14:53:23.867671 (+ 35089us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:24.111787 (+244116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:24.111839 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:24.112875 (+  1036us) log.cc:844] Serialized 78 byte log entry
0709 14:53:24.811250 (+698375us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:25.335748 (+524498us) log.cc:844] Serialized 14 byte log entry
0709 14:53:25.345338 (+  9590us) log.cc:844] Serialized 14 byte log entry
0709 14:53:25.348248 (+  2910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.348340 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:25.348397 (+    57us) raft_consensus.cc:1685] finished
0709 14:53:25.348409 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:25.348615 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181135,"prepare.queue_time_us":178269,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":144595,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":31053,"spinlock_wait_cycles":162479104,"thread_start_us":1195322,"threads_started":2,"wal-append.queue_time_us":1038266}
I20260709 14:53:25.462494  3542 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:25.463511  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:25.418774  3541 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:25.455888  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.467777  3538 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:25.433460  7216 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.358731  7316 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 14:53:25.472782  7216 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.474145  7395 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.352540  7290 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 14:53:25.362176  6992 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.419757  7396 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:25.362684  6978 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:25.418923  7392 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.432104  7403 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.479009  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.354403  7309 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 14:53:25.481010  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:25.483393  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.463675  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.486886  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:25.488873  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:25.476863  7293 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.831s	user 0.019s	sys 0.000s
I20260709 14:53:25.421705  7032 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.433527  7298 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:25.511603  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:25.498270  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:25.507984  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:25.494257  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.456856  7113 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.493422  7308 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.549403  7308 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:25.515095  7300 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.004s
W20260709 14:53:25.550945  7315 consensus_queue.cc:1175] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c1b0e692e4294ee3b8694bbada174ba1" responder_term: 12 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:53:25.516002  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.360049  7311 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.567472  2604 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:25.567298  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:25.567682  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.557885  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:25.545066  2754 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:25.544199  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:25.482005  7396 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:25.554847  3538 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:25.529223  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.591385  6978 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:25.593202  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 81689.7us
I20260709 14:53:25.594409  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:25.596269  7317 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:25.596861  7317 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.597254  7317 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:25.516836  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 13) took 2228 ms. Trace:
I20260709 14:53:25.570734  2753 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:25.603799  2686 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:25.604130  7239 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.575901  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:25.577590  7405 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:25.605101  2606 rpcz_store.cc:276] 0709 14:53:23.279100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.279604 (+   504us) service_pool.cc:224] Handling call
0709 14:53:25.507646 (+2228042us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540042dbd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:25.507805 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545959680}
I20260709 14:53:25.579192  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:25.506150  7403 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.607021  7239 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.608850  7113 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.584594  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.585639  3546 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:25.609979  3541 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:25.547215  7399 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.835s	user 0.012s	sys 0.002s
I20260709 14:53:25.425115  7341 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.504534  7416 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:25.592386  7292 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:25.603166  7287 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.554479  7266 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:25.481940  6992 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.605724  7405 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:25.544266  7406 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 14:53:25.612429  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:25.596227  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:25.521536  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:25.522289  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.574081  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:25.554301  7389 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:25.624177  3543 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:25.641441  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:25.656891  3541 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:25.658595  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.617985  3537 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:25.644845  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:25.639813  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.009s	sys 0.024s
I20260709 14:53:25.649289  2752 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:25.649899  2760 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:25.642195  2606 raft_consensus.cc:606] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:25.643781  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47634.3us
I20260709 14:53:25.665809  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:25.667605  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:25.650156  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76161.1us
W20260709 14:53:25.669477  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.009s	sys 0.024s
I20260709 14:53:25.614090  7416 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:25.553443  7263 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 14:53:25.554891  7302 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.086s
W20260709 14:53:25.680151  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.017s
W20260709 14:53:25.684991  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.002s	sys 0.017s
I20260709 14:53:25.567540  7395 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
W20260709 14:53:25.679157  3411 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.110s (SENT)
I20260709 14:53:25.667100  7389 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:25.699393  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:25.665606  7392 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:25.712476  3536 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:25.618716  7293 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:25.666726  7047 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.669817  7274 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 14:53:25.716599  2751 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 14:53:25.697872  7096 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.001s
I20260709 14:53:25.721741  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:25.677222  7292 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:25.723225  7292 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:25.722791  7293 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.723661  7292 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:25.725513  7293 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.639171  7266 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.727918  2751 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:25.729125  7266 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.729254  7305 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.730193  7305 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.731204  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.732872  7305 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.733060  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.733731  7305 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:25.734313  7305 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:25.734491  7239 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.737025  7305 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.679723  7416 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:25.686058  7378 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:25.745036  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.006s	sys 0.027s
I20260709 14:53:25.627238  7287 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:25.749094  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.007s	sys 0.027s
I20260709 14:53:25.750757  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:25.717420  7389 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:25.754432  2755 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:25.729908  7266 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:25.698107  7216 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.782478  3536 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:25.817987  2751 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:25.748946  7392 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:25.827491  2752 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:25.832644  2754 raft_consensus.cc:1217] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[6.4-10.6]   Dedup: 6.4->[8.5-10.6]
I20260709 14:53:25.833333  2754 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
I20260709 14:53:25.836676  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:25.836849  7292 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.843940  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:25.845086  7396 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:25.853562  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1488) took 1310 ms. Trace:
I20260709 14:53:25.838677  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.859793  3536 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:25.861013  2752 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 14:53:25.861546  2752 pending_rounds.cc:85] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 6
I20260709 14:53:25.861857  2752 pending_rounds.cc:107] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 14:53:25.862191  2752 raft_consensus.cc:2889] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:25.864406  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:25.865337  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.868088  7292 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:25.881513  7396 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.885062  7403 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:25.810875  7266 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.878328  2760 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:25.890292  7266 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:25.891440  7410 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:53:25.892908  7311 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:25.896132  7318 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:25.778555  7378 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.898799  6992 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:25.900499  7275 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.854457  2603 rpcz_store.cc:276] 0709 14:53:24.543688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.544162 (+   474us) service_pool.cc:224] Handling call
0709 14:53:24.544894 (+   732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:24.944776 (+399882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:24.944825 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:24.945435 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 14:53:25.364059 (+418624us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:25.364127 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.364255 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:25.853158 (+488903us) raft_consensus.cc:1685] finished
0709 14:53:25.853198 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:25.853424 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9284,"raft.queue_time_us":242138,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":36722,"thread_start_us":789156,"threads_started":2,"wal-append.queue_time_us":721943}
I20260709 14:53:25.831423  7405 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:25.906162  7392 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:25.908613  7403 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:25.878152  7287 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:25.911676  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 596830us
W20260709 14:53:25.955569  7325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.021s	sys 0.040s
W20260709 14:53:25.918118  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.934875  3541 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:53:26.000124  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:25.962265  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.001726  3537 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:25.966022  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:25.932217  7358 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.197s	user 0.017s	sys 0.000s
W20260709 14:53:25.987579  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 817) took 2236 ms (client timeout 2156 ms). Trace:
I20260709 14:53:25.989487  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.992332  3536 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:25.995396  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:26.041466  3541 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:26.016546  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:26.038640  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:26.042668  2677 rpcz_store.cc:269] 0709 14:53:23.751667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.760511 (+  8844us) service_pool.cc:224] Handling call
0709 14:53:25.987180 (+2226669us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:25.987457 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2971655936}
I20260709 14:53:26.051194  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.052184  2752 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 6.
W20260709 14:53:26.014194  7325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.022s	sys 0.040s
I20260709 14:53:26.055593  7306 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:26.001902  7318 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:26.058979  3541 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:26.059365  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.002s	sys 0.015s
I20260709 14:53:25.969766  7317 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:26.065102  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.070438  3543 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:26.071215  2760 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 14:53:26.070510  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.002s	sys 0.015s
W20260709 14:53:26.070567  3547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.004s	sys 0.014s
W20260709 14:53:26.054811  7188 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.003s
I20260709 14:53:26.119457  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:26.123234  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:26.124573  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:26.125342  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 457876us
I20260709 14:53:26.126154  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:26.128515  3547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.004s	sys 0.016s
I20260709 14:53:26.131346  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:26.131997  2609 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 14:53:26.135339  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:26.049619  7298 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:26.135334  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.017s	sys 0.046s
I20260709 14:53:26.138505  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:26.049734  7358 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13.
I20260709 14:53:26.140106  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1335) took 2547 ms. Trace:
W20260709 14:53:26.161689  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.018s	sys 0.047s
W20260709 14:53:26.161234  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.018s	sys 0.047s
I20260709 14:53:26.163156  2603 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
W20260709 14:53:26.244278  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.019s	sys 0.048s
I20260709 14:53:26.041154  7403 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:26.058555  7216 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:26.169668  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43618.7us
I20260709 14:53:26.084646  7113 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:26.129370  7408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.001s	sys 0.018s
I20260709 14:53:26.169219  2608 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:26.237959  2757 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:26.243381  2756 rpcz_store.cc:276] 0709 14:53:23.592228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.486372 (+894144us) service_pool.cc:224] Handling call
0709 14:53:24.521615 (+ 35243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:24.521938 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:24.981878 (+459940us) log.cc:844] Serialized 14 byte log entry
0709 14:53:26.139314 (+1157436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:26.139419 (+   105us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:26.139652 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.139839 (+   187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:26.140013 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12696,"prepare.queue_time_us":456396,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":74,"raft.queue_time_us":65148,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":119,"thread_start_us":1597914,"threads_started":3,"wal-append.queue_time_us":1014350}
I20260709 14:53:26.248816  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:26.261673  2603 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:26.252755  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:26.263198  7408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.002s	sys 0.021s
I20260709 14:53:26.264154  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.135360  7052 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:26.171663  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:26.138397  7305 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:26.272148  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:26.272883  2760 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:26.087090  7311 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:26.066316  7263 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:26.084012  7287 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.165s
W20260709 14:53:26.249882  7302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.019s	sys 0.049s
I20260709 14:53:26.169533  7309 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
W20260709 14:53:26.298193  2759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.006s	sys 0.015s
W20260709 14:53:26.339993  2759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.006s	sys 0.015s
I20260709 14:53:26.169095  7047 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 14:53:26.248742  7292 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:26.259955  7216 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:26.136065  7266 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.134s
I20260709 14:53:26.084385  7306 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:26.262573  7439 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 14:53:26.141404  7188 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:26.347748  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:25.996642  7275 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:26.269488  7113 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:26.296424  6995 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
I20260709 14:53:26.314186  7305 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:26.575074  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1345) took 1054 ms. Trace:
W20260709 14:53:26.445019  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.689s (SENT)
W20260709 14:53:26.429005  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:26.648267  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.649421  2756 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 14:53:26.649464  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:26.379077  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1162) took 1117 ms. Trace:
I20260709 14:53:26.650482  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:26.362869  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:26.642068  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 503622us
W20260709 14:53:26.338961  7302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.019s	sys 0.050s
I20260709 14:53:26.442140  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:26.357021  7426 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 14:53:26.604300  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.021s	sys 0.022s
W20260709 14:53:26.710845  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.031s	sys 0.025s
I20260709 14:53:26.678290  2756 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:26.672412  3543 rpcz_store.cc:276] 0709 14:53:25.261424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.261652 (+   228us) service_pool.cc:224] Handling call
0709 14:53:26.378637 (+1116985us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002017d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:26.378919 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758909696}
W20260709 14:53:26.567852  7429 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 14:53:26.564117  7436 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 14:53:26.341440  7052 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:26.709465  2606 raft_consensus.cc:606] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:26.572032  7431 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 14:53:26.380266  7422 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.313s	user 0.000s	sys 0.000s
I20260709 14:53:26.443625  7114 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:26.714953  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.507561  7441 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:26.677628  7308 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.127s	user 0.013s	sys 0.002s
I20260709 14:53:26.713469  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:26.765844  3543 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
W20260709 14:53:26.379611  7425 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 14:53:26.554963  7317 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:26.359146  7411 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:26.533372  7266 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:26.343614  6992 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:26.381247  7216 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:26.328611  7311 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:26.646730  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:26.742477  2606 raft_consensus.cc:614] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:26.745527  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:26.820914  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.009s	sys 0.032s
I20260709 14:53:26.750080  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:26.821806  2756 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:26.770903  2753 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
W20260709 14:53:26.759064  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.040s	sys 0.058s
W20260709 14:53:26.764063  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.016s
I20260709 14:53:26.344393  7292 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:26.823745  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.041s	sys 0.059s
W20260709 14:53:26.823920  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.009s	sys 0.016s
I20260709 14:53:26.824126  7292 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:26.357652  7287 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.387s
I20260709 14:53:26.330020  7263 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:26.750734  7114 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:26.817278  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 481484us
W20260709 14:53:26.821298  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.032s
I20260709 14:53:26.823020  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:26.774142  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:26.827879  2756 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:26.673877  7416 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.897s	user 0.014s	sys 0.000s
W20260709 14:53:26.828840  3536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.002s	sys 0.032s
W20260709 14:53:26.829546  3536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.003s	sys 0.032s
W20260709 14:53:26.830605  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 670073us
W20260709 14:53:26.832394  7300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.036s
I20260709 14:53:26.739524  7302 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:26.821554  7294 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.588s	user 0.017s	sys 0.002s
I20260709 14:53:26.811329  7403 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:26.837458  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 392485us
I20260709 14:53:26.841337  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:26.811959  7411 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:26.811345  7266 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:26.843564  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1283) took 1541 ms. Trace:
I20260709 14:53:26.847923  2684 rpcz_store.cc:276] 0709 14:53:25.301954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.302197 (+   243us) service_pool.cc:224] Handling call
0709 14:53:25.322712 (+ 20515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.338000 (+ 15288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.338056 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.345915 (+  7859us) log.cc:844] Serialized 26 byte log entry
0709 14:53:25.824672 (+478757us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:25.824730 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.824858 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:26.843262 (+1018404us) raft_consensus.cc:1685] finished
0709 14:53:26.843288 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:26.843476 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":8667,"thread_start_us":391069,"threads_started":1,"wal-append.queue_time_us":775868}
W20260709 14:53:26.845806  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:26.851545  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:26.859371  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:26.770447  7442 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:26.836721  7416 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:26.820181  6992 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:26.862341  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:26.892783  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 822) took 1664 ms. Trace:
I20260709 14:53:26.910764  2605 rpcz_store.cc:276] 0709 14:53:25.227988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.228402 (+   414us) service_pool.cc:224] Handling call
0709 14:53:25.228902 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.235169 (+  6267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.235207 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.235821 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 14:53:25.752287 (+516466us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:25.752358 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:25.752470 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:26.892507 (+1140037us) raft_consensus.cc:1685] finished
0709 14:53:26.892533 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:26.892707 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5812,"raft.queue_time_us":20605,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":28434,"thread_start_us":515077,"threads_started":1,"wal-append.queue_time_us":820624}
I20260709 14:53:26.871479  3540 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:26.846776  3543 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:26.830688  7439 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:26.916865  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54617.5us
I20260709 14:53:26.828141  7089 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:26.836251  7300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.036s
I20260709 14:53:26.940217  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1313) took 2855 ms. Trace:
I20260709 14:53:26.842780  7433 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:26.930459  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:26.901124  7435 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 14:53:26.902611  7444 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 14:53:27.020648  3543 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:53:26.843915  7274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.124s	user 0.010s	sys 0.015s
W20260709 14:53:26.911682  7408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.022s	sys 0.040s
W20260709 14:53:26.901743  7445 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 14:53:26.960273  2679 rpcz_store.cc:276] 0709 14:53:24.085223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.085573 (+   350us) service_pool.cc:224] Handling call
0709 14:53:24.464584 (+379011us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:24.465329 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.939877 (+2474548us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400377150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:26.940116 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851371264}
I20260709 14:53:26.840165  7302 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:27.040493  7274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.013s	sys 0.018s
I20260709 14:53:26.944398  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:26.872705  7047 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 14:53:26.944226  7089 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:26.968019  7306 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:26.970341  7263 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:26.872339  7309 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 14:53:26.944832  7439 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:26.931568  7442 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:26.976714  7440 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.455s	user 0.000s	sys 0.002s
W20260709 14:53:26.977986  7449 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 14:53:26.983565  7188 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:26.983880  7403 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:26.985457  2611 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:26.986975  7416 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:26.988595  7311 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:26.989677  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1038) took 1649 ms. Trace:
I20260709 14:53:26.989902  7275 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:26.991552  7305 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:26.992600  7455 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:26.993800  7113 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:26.994946  7453 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:26.999922  7411 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.099717  7358 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.004514  6992 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:26.840332  7451 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:26.873229  7096 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:27.007843  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:27.013509  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:26.892755  7422 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 14:53:27.016712  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1016) took 3903 ms. Trace:
W20260709 14:53:26.900494  7438 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 14:53:27.042587  7454 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.038543  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.116734  7311 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:26.986554  7389 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.179s	user 0.015s	sys 0.000s
I20260709 14:53:26.925081  2756 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:27.137777  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 124339us
I20260709 14:53:27.138177  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 846) took 1318 ms. Trace:
I20260709 14:53:27.138370  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1468) took 1008 ms. Trace:
I20260709 14:53:27.138792  2756 rpcz_store.cc:276] 0709 14:53:26.130294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.822658 (+692364us) service_pool.cc:224] Handling call
0709 14:53:27.138298 (+315640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:27.037424  7317 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:27.058741  7408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.029s	sys 0.043s
I20260709 14:53:27.037938  7315 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:27.017319  7308 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:27.072319  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 843) took 1319 ms. Trace:
I20260709 14:53:27.044107  7287 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.138628  2751 rpcz_store.cc:276] 0709 14:53:25.819838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.820017 (+   179us) service_pool.cc:224] Handling call
0709 14:53:25.820462 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.821016 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.821071 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.860337 (+ 39266us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.645313 (+784976us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:27.098523 (+453210us) log.cc:844] Serialized 14 byte log entry
0709 14:53:27.099897 (+  1374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.100021 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.137916 (+ 37895us) raft_consensus.cc:1685] finished
0709 14:53:27.137940 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.138102 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1013,"prepare.queue_time_us":259678,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":67458,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":6028,"thread_start_us":799674,"threads_started":2,"wal-append.queue_time_us":968997}
I20260709 14:53:26.903374  7114 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.037053  7433 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:27.103394  2759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.011s	sys 0.007s
I20260709 14:53:27.104615  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:27.075145  7306 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:26.947866  7266 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.092029  7263 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:27.104972  3536 rpcz_store.cc:276] 0709 14:53:25.340630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.049039 (+708409us) service_pool.cc:224] Handling call
0709 14:53:26.104770 (+ 55731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:26.105160 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:26.105203 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:26.159332 (+ 54129us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.829970 (+670638us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:26.830024 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.830116 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:26.989267 (+159151us) raft_consensus.cc:1685] finished
0709 14:53:26.989312 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:26.989578 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":522545,"threads_started":1,"wal-append.queue_time_us":721710}
I20260709 14:53:27.001227  7301 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.105207  7446 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:27.119917  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 189598us
W20260709 14:53:27.122750  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.017s	sys 0.012s
W20260709 14:53:27.195750  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.017s	sys 0.012s
I20260709 14:53:27.126063  2752 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:27.197427  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:27.127043  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 821) took 1900 ms (client timeout 1891 ms). Trace:
I20260709 14:53:27.109944  7455 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:27.131842  2607 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:27.128816  7047 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.127552  3545 rpcz_store.cc:276] 0709 14:53:23.113105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:23.333183 (+220078us) service_pool.cc:224] Handling call
0709 14:53:23.333647 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:24.604937 (+1271290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.016280 (+2411343us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54004b41d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:27.016600 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2922070272}
I20260709 14:53:27.106662  7403 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.132609  7462 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:26.900077  7437 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 14:53:27.133816  3543 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:27.111544  7460 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.143808  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 832) took 1658 ms. Trace:
I20260709 14:53:27.150302  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 824) took 1660 ms. Trace:
W20260709 14:53:27.154202  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:27.126766  7411 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:27.132077  7464 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.126600  7096 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:27.093035  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:27.178646  2759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.011s	sys 0.007s
I20260709 14:53:27.179673  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:27.175936  2755 rpcz_store.cc:276] 0709 14:53:25.815899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.827745 (+ 11846us) service_pool.cc:224] Handling call
0709 14:53:25.847315 (+ 19570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.875614 (+ 28299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.875670 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.876449 (+   779us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.711347 (+834898us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:27.070606 (+359259us) log.cc:844] Serialized 14 byte log entry
0709 14:53:27.071747 (+  1141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.071914 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.071983 (+    69us) raft_consensus.cc:1685] finished
0709 14:53:27.072002 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.072227 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27623,"prepare.queue_time_us":266939,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":28374,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":14651,"thread_start_us":1148891,"threads_started":2,"wal-append.queue_time_us":1002049}
I20260709 14:53:27.131922  7443 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:27.129693  7113 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:27.179913  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:27.124802  7358 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.189718  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:27.173372  7089 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:27.198853  2604 rpcz_store.cc:269] 0709 14:53:25.226314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.226475 (+   161us) service_pool.cc:224] Handling call
0709 14:53:27.124636 (+1898161us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540050ffd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.126950 (+  2314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751570176}
I20260709 14:53:27.119253  7453 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.175387  7451 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:27.176402  7305 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.145426  7311 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.152748  7317 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.136898  7454 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.149966  7389 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.187191  7308 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.311594  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.176023  7413 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 14:53:27.200672  7295 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.387624  7442 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:27.328287  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 842) took 1587 ms. Trace:
I20260709 14:53:27.420904  2754 rpcz_store.cc:276] 0709 14:53:25.741125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.743725 (+  2600us) service_pool.cc:224] Handling call
0709 14:53:25.832368 (+ 88643us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:25.833750 (+  1382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.306645 (+1472895us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b540040ecd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.328152 (+ 21507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100181248}
I20260709 14:53:27.361989  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1398) took 1943 ms. Trace:
W20260709 14:53:27.310786  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:27.422470  2612 rpcz_store.cc:276] 0709 14:53:25.418727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.422175 (+  3448us) service_pool.cc:224] Handling call
0709 14:53:25.464305 (+ 42130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.608427 (+144122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.608460 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.609055 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.243834 (+634779us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:26.243904 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.361519 (+1117615us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:27.361577 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.361629 (+    52us) raft_consensus.cc:1685] finished
0709 14:53:27.361650 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.361911 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6928,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":2947,"spinlock_wait_cycles":330350848,"thread_start_us":634253,"threads_started":1,"wal-append.queue_time_us":945962}
W20260709 14:53:27.427976  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:27.429164  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.430168  2612 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:53:27.308485  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1155) took 2143 ms (client timeout 1689 ms). Trace:
W20260709 14:53:27.307700  3540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1034) took 1982 ms (client timeout 1634 ms). Trace:
W20260709 14:53:27.431191  3540 rpcz_store.cc:269] 0709 14:53:25.325413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.326019 (+   606us) service_pool.cc:224] Handling call
0709 14:53:27.307465 (+1981446us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.307606 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3341148032}
I20260709 14:53:27.432852  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:27.433681  3540 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:27.434764  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1446) took 5121 ms. Trace:
I20260709 14:53:27.435349  7350 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:27.393177  2610 rpcz_store.cc:276] 0709 14:53:25.489689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.489865 (+   176us) service_pool.cc:224] Handling call
0709 14:53:25.510347 (+ 20482us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:25.551697 (+ 41350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.551757 (+    60us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:25.552697 (+   940us) log.cc:844] Serialized 78 byte log entry
0709 14:53:26.044568 (+491871us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:26.503230 (+458662us) log.cc:844] Serialized 14 byte log entry
0709 14:53:26.507156 (+  3926us) log.cc:844] Serialized 14 byte log entry
0709 14:53:26.507457 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.507583 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.149883 (+642300us) raft_consensus.cc:1685] finished
0709 14:53:27.149918 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.150197 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1346856,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":79,"raft.queue_time_us":340400,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":26198,"spinlock_wait_cycles":102486656,"thread_start_us":910942,"threads_started":2,"wal-append.queue_time_us":967305}
W20260709 14:53:27.395507  7318 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.275s	user 0.015s	sys 0.000s
I20260709 14:53:27.435564  2759 rpcz_store.cc:276] 0709 14:53:22.313185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:24.035353 (+1722168us) service_pool.cc:224] Handling call
0709 14:53:24.036030 (+   677us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:24.113628 (+ 77598us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 14:53:24.451921 (+338293us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18001c8f40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:53:25.168356 (+716435us) log.cc:844] Serialized 14 byte log entry
0709 14:53:25.207058 (+ 38702us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:53:26.340482 (+1133424us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:26.355618 (+ 15136us) log.cc:844] Serialized 52 byte log entry
0709 14:53:26.439924 (+ 84306us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:27.389856 (+949932us) log.cc:844] Serialized 14 byte log entry
0709 14:53:27.434191 (+ 44335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.434324 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.434396 (+    72us) raft_consensus.cc:1685] finished
0709 14:53:27.434415 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.434654 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76918,"prepare.queue_time_us":1192790,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":4413,"raft.queue_time_us":137796,"raft.run_cpu_time_us":2797,"raft.run_wall_time_us":1080052,"spinlock_wait_cycles":27770496,"thread_start_us":2872618,"threads_started":5,"wal-append.queue_time_us":822002}
I20260709 14:53:27.438473  7318 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:27.439339  7318 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:27.443858  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.396543  7311 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:27.396847  2756 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:27.447721  2752 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:27.448928  7490 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.398183  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.452416  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.453423  2604 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:27.398909  2751 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:27.455235  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 853) took 1527 ms. Trace:
I20260709 14:53:27.455742  2751 rpcz_store.cc:276] 0709 14:53:25.928041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.189279 (+1261238us) service_pool.cc:224] Handling call
0709 14:53:27.455141 (+265862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:27.399685  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 265698us
I20260709 14:53:27.400303  7047 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:27.458537  3546 rpcz_store.cc:269] 0709 14:53:25.165549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.165820 (+   271us) service_pool.cc:224] Handling call
0709 14:53:27.308218 (+2142398us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.308380 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662699392}
I20260709 14:53:27.460278  7047 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:27.460566  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:27.398584  2681 rpcz_store.cc:276] 0709 14:53:25.485336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.485480 (+   144us) service_pool.cc:224] Handling call
0709 14:53:25.485927 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.503631 (+ 17704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.503687 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.526321 (+ 22634us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.047877 (+521556us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:26.047941 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.048085 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.143448 (+1095363us) raft_consensus.cc:1685] finished
0709 14:53:27.143486 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.143716 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":512,"raft.queue_time_us":296,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":17310,"thread_start_us":417955,"threads_started":1,"wal-append.queue_time_us":732850}
I20260709 14:53:27.402966  7305 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.461762  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:27.407207  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 227373us
I20260709 14:53:27.465322  2604 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:27.156175  7469 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:27.365248  7273 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:27.418664  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.419839  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:27.449743  7490 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:27.401419  7460 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.171451  7315 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.392747  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:27.208853  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 15) took 1354 ms. Trace:
I20260709 14:53:27.408002  7455 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:27.367240  7463 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:27.278281  7449 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 14:53:27.366783  7439 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:27.412061  7114 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:27.477320  2756 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:27.516600  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1466) took 1387 ms. Trace:
I20260709 14:53:27.207770  7263 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:27.405258  7096 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:27.462348  3546 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:27.181929  7287 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.506735  7047 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 14:53:27.526937  7476 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 14:53:27.528859  7287 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.462941  7188 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:27.404163  7403 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:27.472527  7395 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.746s	user 0.005s	sys 0.027s
I20260709 14:53:27.191759  7306 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.422693  7443 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.499599  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:27.506073  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.505177  2606 rpcz_store.cc:276] 0709 14:53:25.843580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.843763 (+   183us) service_pool.cc:224] Handling call
0709 14:53:27.198105 (+1354342us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.198246 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988158720}
W20260709 14:53:27.505014  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112410us
W20260709 14:53:27.505388  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 319600us
I20260709 14:53:27.497475  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.494875  3540 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:27.473407  7469 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:27.374681  7433 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:27.517478  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:27.506178  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 310854us
I20260709 14:53:26.646432  2758 rpcz_store.cc:276] 0709 14:53:25.520440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.567950 (+ 47510us) service_pool.cc:224] Handling call
0709 14:53:25.568477 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:25.568724 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:25.621289 (+ 52565us) log.cc:844] Serialized 14 byte log entry
0709 14:53:26.574176 (+952887us) log.cc:844] Serialized 14 byte log entry
0709 14:53:26.574481 (+   305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:26.574553 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:26.574635 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.574751 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:26.574976 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4907,"prepare.queue_time_us":54266,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":85,"raft.queue_time_us":67946,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":40493,"thread_start_us":949192,"threads_started":2,"wal-append.queue_time_us":1024064}
I20260709 14:53:27.201084  7301 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.473670  7032 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.401630  7451 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.517051  2754 rpcz_store.cc:276] 0709 14:53:26.128975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.496043 (+1367068us) service_pool.cc:224] Handling call
0709 14:53:27.515940 (+ 19897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.516183 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.516212 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:27.516262 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.516307 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.516366 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.516532 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:27.527647  7047 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:27.405422  7302 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.209431  7446 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.530773  3545 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:27.468410  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:27.535351  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 836) took 1851 ms (client timeout 2110 ms). Trace:
I20260709 14:53:27.417721  7462 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:27.406872  7113 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.509809  7273 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 14:53:27.406929  7464 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.498100  7309 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 14:53:27.709753  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:27.734341  2754 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:27.565096  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1499) took 1431 ms. Trace:
W20260709 14:53:27.764041  2753 rpcz_store.cc:269] 0709 14:53:25.683393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.700268 (+ 16875us) service_pool.cc:224] Handling call
0709 14:53:27.534936 (+1834668us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.535185 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384545664}
I20260709 14:53:27.559820  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1163) took 2285 ms. Trace:
I20260709 14:53:27.768767  2684 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:27.593528  2756 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 14:53:27.553058  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53608.9us
I20260709 14:53:27.710429  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.603044  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:27.775756  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1467) took 1646 ms. Trace:
I20260709 14:53:27.777325  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.556090  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:27.555064  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.573203  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:27.608906  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.774405  3537 rpcz_store.cc:276] 0709 14:53:25.274882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.677002 (+402120us) service_pool.cc:224] Handling call
0709 14:53:26.001435 (+324433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:26.002403 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.559588 (+1557185us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54004c4ad0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:27.559750 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730016000}
W20260709 14:53:27.779915  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 223937us
I20260709 14:53:27.783247  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:27.569295  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:27.709753  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:27.761497  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 293212us
I20260709 14:53:27.748508  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.784574  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:27.563755  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.785615  3540 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:27.565116  2612 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:27.601461  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:27.709743  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133887us
I20260709 14:53:27.560209  3543 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:27.600937  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.601550  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.787719  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.788830  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.768229  2609 rpcz_store.cc:276] 0709 14:53:26.133320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.133640 (+   320us) service_pool.cc:224] Handling call
0709 14:53:26.134291 (+   651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:26.346422 (+212131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:26.346479 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:26.385383 (+ 38904us) log.cc:844] Serialized 26 byte log entry
0709 14:53:27.196262 (+810879us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:27.196315 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.538223 (+341908us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:27.538312 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.538403 (+    91us) raft_consensus.cc:1685] finished
0709 14:53:27.538427 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.565020 (+ 26593us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70600,"raft.queue_time_us":83046,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"spinlock_wait_cycles":324207360,"thread_start_us":665476,"threads_started":1,"wal-append.queue_time_us":746097}
I20260709 14:53:27.789891  3543 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:27.569541  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1503) took 1291 ms. Trace:
I20260709 14:53:27.790310  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.554126  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:27.550374  7395 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.824s	user 0.005s	sys 0.028s
I20260709 14:53:27.837926  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1472) took 1010 ms. Trace:
I20260709 14:53:27.838815  2754 rpcz_store.cc:276] 0709 14:53:26.828109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.768715 (+940606us) service_pool.cc:224] Handling call
0709 14:53:27.837117 (+ 68402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.837400 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.837441 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:27.837509 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.837563 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.837646 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.837830 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:27.784744  2680 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:27.746623  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:27.597085  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1158) took 2343 ms. Trace:
I20260709 14:53:27.545477  7400 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.602929  2682 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:27.790803  2603 rpcz_store.cc:276] 0709 14:53:26.278591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.278929 (+   338us) service_pool.cc:224] Handling call
0709 14:53:26.648266 (+369337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:27.068901 (+420635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.068954 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:27.097682 (+ 28728us) log.cc:844] Serialized 26 byte log entry
0709 14:53:27.272170 (+174488us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.272229 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.272377 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.569284 (+296907us) raft_consensus.cc:1685] finished
0709 14:53:27.569312 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.569476 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":243207,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":30202,"thread_start_us":490885,"threads_started":2,"wal-append.queue_time_us":455836}
I20260709 14:53:27.560561   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 14:53:27.794056  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 220912us
I20260709 14:53:27.553833  7443 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:27.579690  7341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.857s	user 0.037s	sys 0.052s
W20260709 14:53:27.542721  7216 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.002s
I20260709 14:53:27.788285  3538 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:27.875036  2684 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:27.830765  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.832429  7302 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:27.832255  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 263036us
W20260709 14:53:27.830514  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47305.3us
I20260709 14:53:27.557440  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.840638  3547 rpcz_store.cc:276] 0709 14:53:25.254011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.254256 (+   245us) service_pool.cc:224] Handling call
0709 14:53:25.259598 (+  5342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:25.481406 (+221808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:25.481453 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:25.506621 (+ 25168us) log.cc:844] Serialized 26 byte log entry
0709 14:53:26.129118 (+622497us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:26.129164 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:26.376740 (+247576us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400550ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:26.376826 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:27.574146 (+1197320us) raft_consensus.cc:1685] finished
0709 14:53:27.574174 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.596968 (+ 22794us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509376896,"thread_start_us":575876,"threads_started":1,"wal-append.queue_time_us":1119490}
I20260709 14:53:27.787525  2681 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:27.605268  7089 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:27.544734  7297 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:27.571178  7301 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.702927  7476 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:27.554054  7316 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:27.845268  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1473) took 1016 ms. Trace:
I20260709 14:53:27.930461  2753 rpcz_store.cc:276] 0709 14:53:26.828429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.774433 (+946004us) service_pool.cc:224] Handling call
0709 14:53:27.844529 (+ 70096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.844804 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.844843 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:27.844909 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:27.844957 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.845037 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.845180 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:27.846362  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.555642  7378 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.770529  6748 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:27.761955  7273 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:27.847534  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.833606  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:27.545123  7315 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:27.847649  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.849119  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:27.849946  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.564045  7469 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:27.596653  7047 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:27.580858  7453 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.549710  7188 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:27.558279  7460 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:27.552595  7306 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:27.856524  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.842458  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:27.777148  2756 rpcz_store.cc:276] 0709 14:53:26.129699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.517351 (+1387652us) service_pool.cc:224] Handling call
0709 14:53:27.517678 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.775294 (+257616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.775667 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:27.860898  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:27.864168  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:28.009303  2603 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:27.572973  7439 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.269s
W20260709 14:53:28.014394  7428 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.949s	user 0.004s	sys 0.000s
I20260709 14:53:27.572608  7442 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.282s
I20260709 14:53:27.780539  2606 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:27.831724  7287 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:27.831318  7305 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:27.834928  7294 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:27.835139  7466 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:27.831548  7433 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:27.847739  7259 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 14:53:27.893836  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:27.911525  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:27.907708  3542 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:27.521063  7463 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:27.868760  7341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.146s	user 0.043s	sys 0.054s
I20260709 14:53:27.936100  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1470) took 1804 ms. Trace:
I20260709 14:53:27.862896  7308 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:27.938867  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:27.944576  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 111086us
W20260709 14:53:27.949827  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 162046us
W20260709 14:53:28.056077  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117391us
I20260709 14:53:28.056515  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:27.969242  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.057607  2753 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:27.942857  2685 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:27.889811  7389 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:27.923746  7297 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:27.936164  7089 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:27.933107  7316 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:27.933931  7358 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.011000  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.965005  7301 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:27.937608  6748 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:27.876214  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1173) took 1617 ms. Trace:
I20260709 14:53:28.017223  6978 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 14:53:28.017935  7488 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 14:53:28.019603  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.021529  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:27.892179  7216 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:27.943032  7273 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:53:28.024235  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:28.028687  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:28.029762  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 187405us
I20260709 14:53:27.867937  7263 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:27.966512  7317 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.045297  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.156479  2756 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:27.937551  7476 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:28.047785  2606 raft_consensus.cc:1217] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 8.4->[9.5-11.6]   Dedup: 11.6->[]
I20260709 14:53:28.051659  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:28.046718  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 185860us
I20260709 14:53:28.058171  2758 rpcz_store.cc:276] 0709 14:53:26.131818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.565821 (+1434003us) service_pool.cc:224] Handling call
0709 14:53:27.566192 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.567047 (+   855us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:27.567395 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 14:53:27.935520 (+368125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:27.935616 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.935690 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.935805 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:27.936004 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36401,"prepare.queue_time_us":743,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":17432,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":24183,"thread_start_us":355018,"threads_started":1}
I20260709 14:53:28.036042  7302 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:28.161708  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1505) took 1054 ms. Trace:
W20260709 14:53:28.117720  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 224013us
W20260709 14:53:28.127354  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 215761us
I20260709 14:53:28.185458  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.184835  2603 rpcz_store.cc:276] 0709 14:53:27.107730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.160543 (+1052813us) service_pool.cc:224] Handling call
0709 14:53:28.160983 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.161248 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.161307 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.161378 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:28.161430 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.161489 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.161643 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:28.059819  7411 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.664s	user 0.004s	sys 0.009s
I20260709 14:53:28.046191  7454 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:28.121942  3537 rpcz_store.cc:276] 0709 14:53:26.259194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.829034 (+1569840us) service_pool.cc:224] Handling call
0709 14:53:27.876122 (+ 47088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.043936  7294 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.052913  7462 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:28.087407  7403 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:28.083357  7287 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:28.081836  7306 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.216203  7294 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:28.227270  7306 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:28.192929  7341 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:28.229995  7341 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:28.154978  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.231236  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:28.231730  7306 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.232306  7341 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.159188  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:28.187103  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:28.192706  2606 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 14:53:28.191852  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:28.104009  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47584us
I20260709 14:53:28.125972  6748 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:28.200103  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1506) took 1092 ms. Trace:
I20260709 14:53:28.201705  2608 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 14:53:28.203528  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.250372  2603 rpcz_store.cc:276] 0709 14:53:27.108078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.199054 (+1090976us) service_pool.cc:224] Handling call
0709 14:53:28.199503 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.199718 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.199748 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.199799 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:28.199830 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.199884 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.200035 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.206722  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.125099  6978 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:28.212361  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.161062  7378 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:28.214732  7496 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:53:28.210306  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:28.129153  7316 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:28.219484  7308 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:28.220625  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 826) took 1109 ms. Trace:
W20260709 14:53:28.247222  7292 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.411s	user 0.008s	sys 0.008s
I20260709 14:53:28.221240  7476 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.232384  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:28.271627  2753 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:28.158732  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.247352  7463 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.132083  3536 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:28.187136  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.121063  7460 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 14:53:28.128819  7499 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:28.240957  3540 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:28.242046  3542 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:28.085784  7047 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:28.244041  7485 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 14:53:28.245126  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1403) took 1870 ms. Trace:
I20260709 14:53:28.192045  7301 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:28.207998  7297 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.247130  7273 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:28.192278  7188 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:28.217970  7411 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:28.249245  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62231.4us
I20260709 14:53:28.251987  2603 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:28.342865  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 831) took 1191 ms. Trace:
I20260709 14:53:28.343518  2603 rpcz_store.cc:276] 0709 14:53:27.151448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.251420 (+1099972us) service_pool.cc:224] Handling call
0709 14:53:28.251790 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.342383 (+ 90593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.342750 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:28.252061  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45501.8us
I20260709 14:53:28.253718  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:28.345532  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91933.9us
I20260709 14:53:28.277041  7389 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:28.347671  3411 connection.cc:256] RPC call timeout handler was delayed by -1.2787237230000983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 14:53:28.260319  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.160352  7113 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:28.215392  7403 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:28.202765  7317 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:28.164135  7451 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.269024  2612 rpcz_store.cc:276] 0709 14:53:27.111577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.219566 (+1107989us) service_pool.cc:224] Handling call
0709 14:53:28.220057 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.220278 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.220303 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.220348 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:28.220377 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.220432 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.220558 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.222371  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1496) took 2094 ms. Trace:
W20260709 14:53:28.273155  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114476us
I20260709 14:53:28.155683  7358 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:28.128141  7430 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 14:53:28.260334  7302 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:28.256009  7272 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:28.335379  7460 consensus_queue.cc:1243] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 14:53:28.301414  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114331us
I20260709 14:53:28.405535  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.266350  7287 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:28.268513  7446 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.257369  2756 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:28.360411  3537 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:28.268636  7464 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.345965  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:28.362619  7502 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:28.268008  7378 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:28.364274  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1475) took 1369 ms. Trace:
I20260709 14:53:28.368180  3542 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:28.368137  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:28.331559  2606 rpcz_store.cc:276] 0709 14:53:26.374287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.047131 (+1672844us) service_pool.cc:224] Handling call
0709 14:53:28.047639 (+   508us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:28.244609 (+196970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.245033 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.331938  7308 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:28.336536  7499 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:28.285132  7442 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:28.410686  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:28.468384  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57807.5us
I20260709 14:53:28.402896  2611 rpcz_store.cc:276] 0709 14:53:26.127907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.128179 (+   272us) service_pool.cc:224] Handling call
0709 14:53:26.245988 (+117809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:27.098535 (+852547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.222076 (+1123541us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:28.222302 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699785088}
I20260709 14:53:28.413501  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1050) took 1364 ms. Trace:
I20260709 14:53:28.276144  7292 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:28.415431  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69615us
I20260709 14:53:28.356426  7387 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:28.395618  7403 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:28.369163  7315 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:28.336361  7439 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 14:53:28.418988  7446 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:28.403440  7358 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:28.408768  7297 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:28.463749  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:28.484375  2686 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:28.457165  7273 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:28.467271  7432 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.002s	user 0.007s	sys 0.000s
I20260709 14:53:28.464238  2606 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:28.465078  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:28.465082  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.464632  2754 rpcz_store.cc:276] 0709 14:53:26.995424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.840343 (+844919us) service_pool.cc:224] Handling call
0709 14:53:27.872803 (+ 32460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.873136 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:27.873578 (+   442us) log.cc:844] Serialized 14 byte log entry
0709 14:53:28.363663 (+490085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.363731 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:28.363783 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.364009 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.364187 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":148,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":87,"raft.queue_time_us":112,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":489520,"threads_started":1}
I20260709 14:53:28.465072  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.492833  2684 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:28.493528  2756 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:28.466013  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.469605  3537 rpcz_store.cc:276] 0709 14:53:27.049462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.259842 (+1210380us) service_pool.cc:224] Handling call
0709 14:53:28.413446 (+153604us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.470924  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:28.495975  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.496635  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.496907  3537 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:28.408298  7287 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.472798  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:28.416838  7502 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:28.475677  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:28.505381  2679 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:28.478686  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:28.363901  7411 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:28.412523  7317 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:28.423733  2753 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:28.488868  7439 consensus_queue.cc:1243] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:53:28.467519  7442 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:28.408365  7272 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:28.493276  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 835) took 1199 ms. Trace:
I20260709 14:53:28.515829  2606 rpcz_store.cc:276] 0709 14:53:27.294168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.463428 (+1169260us) service_pool.cc:224] Handling call
0709 14:53:28.463815 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.492517 (+ 28702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.493208 (+   691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.498347  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.470206  7308 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:28.503050  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.478510  7305 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:28.499908  2760 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:28.505738  7287 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:28.417342  7294 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:28.527000  7272 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:28.528210  7503 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 14:53:28.481978  7089 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:28.483652  7446 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.510093  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.506876  3542 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:28.414319  7302 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:28.502336  7451 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:28.512120  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.533207  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.533953  2754 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:28.518864  3537 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:28.505908  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:28.507398  7476 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:28.527328  7501 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 14:53:28.529788  2753 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:28.530146  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1356) took 3183 ms. Trace:
I20260709 14:53:28.507473  7502 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:28.535868  2756 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:28.539088  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1358) took 3111 ms. Trace:
I20260709 14:53:28.538864  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:28.653780  3537 rpcz_store.cc:276] 0709 14:53:25.427043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.518142 (+3091099us) service_pool.cc:224] Handling call
0709 14:53:28.518631 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.534608 (+ 15977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.538746 (+  4138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:28.654770  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116006us
I20260709 14:53:28.656046  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.656370  3537 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 14:53:28.480345  7387 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.568420  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1359) took 3141 ms. Trace:
I20260709 14:53:28.491113  7493 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 14:53:28.471390  7301 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:28.511273  7507 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.649294  3542 rpcz_store.cc:276] 0709 14:53:25.346720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.506292 (+3159572us) service_pool.cc:224] Handling call
0709 14:53:28.506621 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.529824 (+ 23203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.530082 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.663992  3546 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:28.521656  7430 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:28.529762  7317 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:28.665042  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1363) took 3154 ms. Trace:
I20260709 14:53:28.657702  3547 rpcz_store.cc:276] 0709 14:53:25.427389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.534652 (+3107263us) service_pool.cc:224] Handling call
0709 14:53:28.567573 (+ 32921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.567893 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.567934 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.568002 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:28.568045 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.568128 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.568318 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.530665  7113 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:28.676229  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1045) took 3174 ms. Trace:
I20260709 14:53:28.689551  3537 rpcz_store.cc:276] 0709 14:53:25.482813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.655416 (+3172603us) service_pool.cc:224] Handling call
0709 14:53:28.656012 (+   596us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:28.657111 (+  1099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.657450 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.684670  3542 rpcz_store.cc:276] 0709 14:53:25.510354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.663758 (+3153404us) service_pool.cc:224] Handling call
0709 14:53:28.664248 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.664527 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.664566 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.664634 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:28.664683 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.664760 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.664952 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.685766  3547 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:28.533365  7446 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:28.692569  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1364) took 3079 ms. Trace:
I20260709 14:53:28.693123  3547 rpcz_store.cc:276] 0709 14:53:25.613560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.685132 (+3071572us) service_pool.cc:224] Handling call
0709 14:53:28.685541 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.692127 (+  6586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.692478 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.695925  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1367) took 2836 ms. Trace:
I20260709 14:53:28.696560  3547 rpcz_store.cc:276] 0709 14:53:25.859838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.694383 (+2834545us) service_pool.cc:224] Handling call
0709 14:53:28.694960 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.695416 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.695449 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.695513 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:28.695561 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.695643 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.695836 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":105}
I20260709 14:53:28.659802  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.691455  7468 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:28.699412  7468 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.701956  7508 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:28.702646  3537 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 14:53:28.702728  7508 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:28.704403  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1176) took 1828 ms. Trace:
I20260709 14:53:28.705224  3537 rpcz_store.cc:276] 0709 14:53:26.876113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.700665 (+1824552us) service_pool.cc:224] Handling call
0709 14:53:28.701870 (+  1205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.703912 (+  2042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.704329 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.644286  7451 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:28.645848  7476 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:28.645102  7305 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:28.705327  7508 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.659567  7089 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:28.729403  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 232823us
I20260709 14:53:28.730865  2753 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:28.748493  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1476) took 1752 ms. Trace:
I20260709 14:53:28.749081  2753 rpcz_store.cc:276] 0709 14:53:26.995853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.662883 (+1667030us) service_pool.cc:224] Handling call
0709 14:53:28.717926 (+ 55043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.748058 (+ 30132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.748403 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.662024  7272 raft_consensus.cc:2764] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 14:53:28.741350  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 170574us
I20260709 14:53:28.681717  7460 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:28.745653  3537 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:53:28.686528  7317 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:28.689340  7113 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:28.746014  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 273264us
I20260709 14:53:28.758862  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1046) took 1711 ms. Trace:
I20260709 14:53:28.710135  7493 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:28.730206  7451 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:28.742014  7476 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:28.765084  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:28.765885  3537 rpcz_store.cc:276] 0709 14:53:27.047632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.744932 (+1697300us) service_pool.cc:224] Handling call
0709 14:53:28.745435 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.758443 (+ 13008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.758747 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.667956  7484 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 14:53:28.772220  2753 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 14:53:28.778483  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.708631  7454 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 14:53:28.756742  7460 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:28.688980  7430 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:28.766634  7317 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:28.784549  7484 consensus_queue.cc:1243] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 14:53:28.769089  7113 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:28.807082  7358 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:28.795557  7460 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:28.815490  7387 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.816484  7239 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:28.817011  7239 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:28.817953  7506 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 14:53:28.821226  3543 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:28.821767  3543 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:28.818197  7239 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.824260  7239 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:28.824908  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 855) took 2862 ms. Trace:
I20260709 14:53:28.822674  3543 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:28.825379  2752 rpcz_store.cc:276] 0709 14:53:25.962436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.296185 (+1333749us) service_pool.cc:224] Handling call
0709 14:53:27.388556 (+ 92371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.448533 (+ 59977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.824559 (+1376026us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:28.824837 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134850816}
I20260709 14:53:28.826907  7490 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:28.827663  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1504) took 2488 ms. Trace:
I20260709 14:53:28.828130  2608 rpcz_store.cc:276] 0709 14:53:26.339298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.339721 (+   423us) service_pool.cc:224] Handling call
0709 14:53:26.340263 (+   542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:28.250530 (+1910267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.827351 (+576821us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54001eb754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:28.827589 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4279919872}
I20260709 14:53:28.829238  3543 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:28.689447  7507 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:28.817140  7504 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 14:53:28.755527  7089 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:28.668748  7499 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.842964  3543 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 14:53:28.844185  3543 pending_rounds.cc:85] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 7
I20260709 14:53:28.844489  3543 pending_rounds.cc:107] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 14:53:28.845930  3543 raft_consensus.cc:2889] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:28.784543  7464 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 14:53:28.847411  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1053) took 1784 ms. Trace:
I20260709 14:53:28.851446  3543 rpcz_store.cc:276] 0709 14:53:27.063544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.820606 (+1757062us) service_pool.cc:224] Handling call
0709 14:53:28.821069 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.847124 (+ 26055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.847329 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:28.832080  7378 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.831830  7292 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:28.858971  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:28.859604  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1464) took 2733 ms. Trace:
I20260709 14:53:28.860236  2755 rpcz_store.cc:276] 0709 14:53:26.126668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.388521 (+1261853us) service_pool.cc:224] Handling call
0709 14:53:27.388927 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:27.753682 (+364755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.753737 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:27.780796 (+ 27059us) log.cc:844] Serialized 52 byte log entry
0709 14:53:28.233216 (+452420us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:28.248685 (+ 15469us) log.cc:844] Serialized 14 byte log entry
0709 14:53:28.248948 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.249035 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:28.859330 (+610295us) raft_consensus.cc:1685] finished
0709 14:53:28.859355 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:28.859534 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15961,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":49169,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"spinlock_wait_cycles":22912,"thread_start_us":726186,"threads_started":2,"wal-append.queue_time_us":512836}
I20260709 14:53:28.773756  7302 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:28.859786  3543 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:28.781337  7308 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:28.880445  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:29.014240  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:29.014896  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.016988  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:28.869923  7272 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.194s
I20260709 14:53:28.885747  7468 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:28.869619  7511 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:29.052500  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 304878us
I20260709 14:53:29.080246  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.082455  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:29.087014  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 322038us
I20260709 14:53:29.079384  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.083137  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.082414  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:29.084877  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.085542  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.134410  2683 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:29.131461  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:29.082543  2682 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:29.137254  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.139191  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:29.131508  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49175.9us
I20260709 14:53:28.842176  7305 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.088439   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 14:53:29.069778  7096 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.466s	user 0.013s	sys 0.001s
I20260709 14:53:29.096906  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.079384  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.157356  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.158128  2684 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:29.143497  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:29.067117  7510 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 14:53:29.155304  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:29.069818  7316 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.793s	user 0.007s	sys 0.006s
W20260709 14:53:29.093840  7455 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.519s	user 0.011s	sys 0.008s
I20260709 14:53:29.056301  7318 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.067440  7464 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:29.072840  7511 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:29.189230  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.203186  2680 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:29.191306  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.142901  7484 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:29.120648  7411 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.210748  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1348) took 1362 ms. Trace:
W20260709 14:53:29.199702  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 114275us
I20260709 14:53:29.204738  2678 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:29.212884  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117655us
I20260709 14:53:29.214186  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:29.187289  7114 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.641s	user 0.018s	sys 0.000s
I20260709 14:53:29.211367  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.211370  2756 rpcz_store.cc:276] 0709 14:53:27.848808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.669727 (+820919us) service_pool.cc:224] Handling call
0709 14:53:28.714667 (+ 44940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.715040 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:28.715426 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 14:53:29.210279 (+494853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:29.210366 (+    87us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:29.210434 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.210516 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.210674 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3478,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":74,"raft.queue_time_us":78797,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":106949,"thread_start_us":494071,"threads_started":2,"wal-append.queue_time_us":296131}
I20260709 14:53:29.222381  2684 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:29.231590  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.225497  7293 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:29.226508  7271 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 14:53:29.135243  7463 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 14:53:29.228502  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1351) took 1357 ms. Trace:
I20260709 14:53:29.185745  7294 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.198525  7464 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:29.234970  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.169801  6995 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 14:53:29.233711  7293 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:29.241638  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.242658  2678 raft_consensus.cc:2376] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13. Re-sending same reply.
I20260709 14:53:29.242764  7293 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.230947  2680 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 14:53:29.223106  7484 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:29.255679  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.256568  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.184717  7316 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.233541  7271 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:29.263173  7271 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.264752  7271 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 6 FOLLOWER]: Advancing to term 7
W20260709 14:53:29.265851  7469 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.260s	user 0.014s	sys 0.000s
I20260709 14:53:29.267267  2677 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:29.234525  2752 rpcz_store.cc:276] 0709 14:53:27.871765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.826374 (+954609us) service_pool.cc:224] Handling call
0709 14:53:28.826790 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:28.833582 (+  6792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.833620 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:28.834166 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 14:53:29.053676 (+219510us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:29.053921 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.054032 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.228116 (+174084us) raft_consensus.cc:1685] finished
0709 14:53:29.228146 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.228400 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7781,"raft.queue_time_us":218823,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":2032,"thread_start_us":201183,"threads_started":1,"wal-append.queue_time_us":384525}
I20260709 14:53:29.269703  2752 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:29.170526  7096 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.207715  7511 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:29.259603  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.260233  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1041) took 3917 ms. Trace:
I20260709 14:53:29.286371  3536 rpcz_store.cc:276] 0709 14:53:25.343299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.330141 (+2986842us) service_pool.cc:224] Handling call
0709 14:53:28.339367 (+  9226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:28.364277 (+ 24910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.364334 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:28.402172 (+ 37838us) log.cc:844] Serialized 26 byte log entry
0709 14:53:28.711117 (+308945us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:28.711200 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.711314 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.259975 (+548661us) raft_consensus.cc:1685] finished
0709 14:53:29.260001 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.260161 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7688,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":13568,"thread_start_us":216612,"threads_started":1,"wal-append.queue_time_us":344023}
I20260709 14:53:29.288678  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.289609  2611 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:29.290300  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.292748  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1373) took 1736 ms. Trace:
I20260709 14:53:29.293200  3536 rpcz_store.cc:276] 0709 14:53:27.556048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.291713 (+1735665us) service_pool.cc:224] Handling call
0709 14:53:29.292103 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.292348 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.292378 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:29.292442 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:29.292482 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.292544 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.292680 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:29.256937  7446 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.008s
I20260709 14:53:29.303007  3536 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:29.243981  7464 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.283564  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.284921  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 865) took 2261 ms. Trace:
W20260709 14:53:29.285116  7297 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.003s
I20260709 14:53:29.280092  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.295912  2686 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:29.309868  2754 rpcz_store.cc:276] 0709 14:53:27.023984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.675088 (+1651104us) service_pool.cc:224] Handling call
0709 14:53:28.726368 (+ 51280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:28.754382 (+ 28014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.754439 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:28.777347 (+ 22908us) log.cc:844] Serialized 52 byte log entry
0709 14:53:28.874444 (+ 97097us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:28.874522 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.874675 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.284648 (+409973us) raft_consensus.cc:1685] finished
0709 14:53:29.284675 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.284854 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82487,"spinlock_wait_cycles":17242368,"thread_start_us":63071,"threads_started":1,"wal-append.queue_time_us":411005}
I20260709 14:53:29.290616  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.312335  2608 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.258912  7484 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.296427  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.244596  7318 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:29.241727  6995 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:29.281811  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1356) took 1343 ms. Trace:
I20260709 14:53:29.307974  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.309276  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1055) took 1453 ms. Trace:
I20260709 14:53:29.313544  2611 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:29.280896  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.316754  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.256294  7507 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.323498  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.231529  7455 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.344192  2608 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:29.266426  7469 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:29.315277  7509 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
I20260709 14:53:29.342021  2752 rpcz_store.cc:276] 0709 14:53:27.938502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.268899 (+1330397us) service_pool.cc:224] Handling call
0709 14:53:29.269411 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.281406 (+ 11995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.281724 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:29.315421  7516 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 14:53:29.321796  7523 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 14:53:29.342643  3536 rpcz_store.cc:276] 0709 14:53:27.856389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.294472 (+1438083us) service_pool.cc:224] Handling call
0709 14:53:29.294967 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.304401 (+  9434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.309197 (+  4796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.360755  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.302932  7518 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:29.322927  7502 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.002s
I20260709 14:53:29.343941  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:29.377439  2754 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 6 FOLLOWER]: Advancing to term 7
I20260709 14:53:29.342341  2606 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:29.315430  7522 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:29.368844  2605 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.369874  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.371680  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.321514  7517 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.374630  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.325294  7318 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:29.327230  6995 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:29.380049  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 232719us
W20260709 14:53:29.380697  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 225480us
I20260709 14:53:29.392706  7318 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:29.396416  2575 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.603s (SENT)
I20260709 14:53:29.397059  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.379832  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.303543  7519 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 14:53:29.398130  7413 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:29.383057  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.309092  7446 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.399237  2678 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:29.400913  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1047) took 2352 ms (client timeout 1603 ms). Trace:
I20260709 14:53:29.401793  7446 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.387112  2680 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 14:53:29.402606  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1507) took 2294 ms (client timeout 1578 ms). Trace:
I20260709 14:53:29.402875  2680 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:29.403439  2610 rpcz_store.cc:269] 0709 14:53:27.108370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.443416 (+335046us) service_pool.cc:224] Handling call
0709 14:53:29.402294 (+1958878us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:29.402501 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760351104}
I20260709 14:53:29.404300  2680 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.405931  2680 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.406208  3542 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:29.410501  7047 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:29.401806  3545 rpcz_store.cc:269] 0709 14:53:27.048158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.418089 (+369931us) service_pool.cc:224] Handling call
0709 14:53:29.400614 (+1982525us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b54002baf54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:29.400801 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79237504}
I20260709 14:53:29.374480  7518 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:29.413084  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.421267  2610 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:29.372332  7292 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:29.415820  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.417884  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:29.429183  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1042) took 4084 ms. Trace:
I20260709 14:53:29.418071  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.429618  3540 rpcz_store.cc:276] 0709 14:53:25.345046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.333417 (+2988371us) service_pool.cc:224] Handling call
0709 14:53:28.397575 (+ 64158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:28.398320 (+   745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.398368 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:28.399169 (+   801us) log.cc:844] Serialized 26 byte log entry
0709 14:53:28.687449 (+288280us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:28.687504 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:28.687710 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.428946 (+741236us) raft_consensus.cc:1685] finished
0709 14:53:29.428971 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.429122 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":8201,"thread_start_us":267296,"threads_started":1,"wal-append.queue_time_us":364478}
I20260709 14:53:29.422973  2686 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
W20260709 14:53:29.426004  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153693us
I20260709 14:53:29.431730  2678 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:29.426869  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.432600  3536 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:29.386765  7430 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.433306  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.435622  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.430433  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:29.436784  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1056) took 1580 ms. Trace:
I20260709 14:53:29.441684  2683 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:29.441596  3536 rpcz_store.cc:276] 0709 14:53:27.856721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.386591 (+1529870us) service_pool.cc:224] Handling call
0709 14:53:29.428169 (+ 41578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.433028 (+  4859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.436690 (+  3662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.439147  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:29.444696  2686 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:53:29.447538  7460 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.000s
I20260709 14:53:29.448176  7460 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:29.451269  3412 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.891s (SENT)
I20260709 14:53:29.451993  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:29.451846  7442 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.454210  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:29.455693  7405 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:29.453135  7442 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:29.456740  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.458331  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1353) took 1537 ms. Trace:
I20260709 14:53:29.458865  2755 rpcz_store.cc:276] 0709 14:53:27.920908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.867154 (+946246us) service_pool.cc:224] Handling call
0709 14:53:28.867773 (+   619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.052047 (+184274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.052082 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:29.138524 (+ 86442us) log.cc:844] Serialized 26 byte log entry
0709 14:53:29.326355 (+187831us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:29.326424 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.326716 (+   292us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.458062 (+131346us) raft_consensus.cc:1685] finished
0709 14:53:29.458086 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.458260 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2950,"raft.queue_time_us":270128,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":19199,"thread_start_us":292307,"threads_started":2,"wal-append.queue_time_us":221544}
I20260709 14:53:29.461782  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.460861  2683 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:29.463294  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45527.3us
I20260709 14:53:29.466735  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.467242  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.467690  3540 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 14:53:29.468075  2755 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:29.431183  6996 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:29.382578  7502 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:29.459702  7451 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.004s
I20260709 14:53:29.470638  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.429327  7518 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.488961  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:29.496320  2755 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.496570  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.432034  7522 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:29.467669  2754 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:53:29.392591  7463 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 14:53:29.492264  3536 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:29.428097  7262 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 14:53:29.433933  7298 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 14:53:29.497314  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:29.497392  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68612.6us
I20260709 14:53:29.472275  7533 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:29.500859  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1065) took 1443 ms. Trace:
I20260709 14:53:29.495158  2684 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:29.501852  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.502279  2758 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:29.474071  7493 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.701s	user 0.000s	sys 0.013s
W20260709 14:53:29.486617  7317 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.004s
I20260709 14:53:29.435635  7292 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:29.522792  6748 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.257s	user 0.010s	sys 0.005s
I20260709 14:53:29.531265  7536 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:29.505846  7529 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:29.513618  7316 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.537986  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1354) took 1613 ms. Trace:
I20260709 14:53:29.505235  7446 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:29.556982  7502 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.573465  7317 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:29.521935  7113 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.004s
I20260709 14:53:29.544037  7096 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:29.652449  7113 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.626907  7533 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:29.653945  7533 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:29.487794  6995 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.561584  7262 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:29.552786  7315 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:29.623350  6748 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.660369  7315 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:29.557027  7530 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 14:53:29.507910  7476 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.719s	user 0.008s	sys 0.005s
I20260709 14:53:29.504139  7451 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.582963  3536 rpcz_store.cc:276] 0709 14:53:28.057957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.443005 (+1385048us) service_pool.cc:224] Handling call
0709 14:53:29.472793 (+ 29788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.500472 (+ 27679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.500774 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.492029  6996 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:29.549295  7403 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.065s	user 0.013s	sys 0.002s
I20260709 14:53:29.480997  7293 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.668349  7403 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.591179  3540 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:29.551787  7469 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.510905  7522 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.633121  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1346) took 4112 ms. Trace:
I20260709 14:53:29.625039  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:29.672531  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:29.672416  2757 rpcz_store.cc:276] 0709 14:53:25.520843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.568926 (+ 48083us) service_pool.cc:224] Handling call
0709 14:53:25.606400 (+ 37474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:26.262695 (+656295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.632798 (+3370103us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:29.633032 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297549184}
I20260709 14:53:29.621610  7460 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:29.508029  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 74807.7us
I20260709 14:53:29.645907  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:29.560809  7287 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.002s
I20260709 14:53:29.623626  7292 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.652216  7463 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:29.650609  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1062) took 1742 ms. Trace:
I20260709 14:53:29.657795  7262 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:29.660161  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.661382  7113 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.539865  7455 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.679239  2610 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:29.662492  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 445298us
I20260709 14:53:29.663851  2609 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.623466  7493 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:29.670722  7403 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.682526  7096 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:29.686451  7096 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:29.686910  7271 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:29.687415  7271 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:29.687861  7271 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:29.687552  7096 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.689780  7517 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:29.625267  2756 rpcz_store.cc:276] 0709 14:53:27.924704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.232633 (+1307929us) service_pool.cc:224] Handling call
0709 14:53:29.246058 (+ 13425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.277456 (+ 31398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.277483 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:29.307853 (+ 30370us) log.cc:844] Serialized 26 byte log entry
0709 14:53:29.389904 (+ 82051us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:29.389972 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.390077 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.537602 (+147525us) raft_consensus.cc:1685] finished
0709 14:53:29.537639 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.537879 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22179,"raft.queue_time_us":74806,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":28230,"thread_start_us":20904,"threads_started":1,"wal-append.queue_time_us":134911}
I20260709 14:53:29.691792  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:29.692399  6996 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.693147  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:29.674595  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.642939  7396 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:29.694902  7396 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.695312  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.696759  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.648171  7499 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:29.697365  3536 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:29.641808  7294 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 14:53:29.684345  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.706128  3540 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.673504  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:29.693277  2755 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:29.642565  7446 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.674042  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:29.719504  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.721833  2758 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:29.699102  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53393us
I20260709 14:53:29.625134  7310 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 14:53:29.676810  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:29.701190  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.678916  3545 rpcz_store.cc:276] 0709 14:53:27.908262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.438800 (+1530538us) service_pool.cc:224] Handling call
0709 14:53:29.439220 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.439512 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:29.590088 (+150576us) log.cc:844] Serialized 14 byte log entry
0709 14:53:29.649914 (+ 59826us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:29.650030 (+   116us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:29.650115 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.650266 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.650504 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":132954,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":166,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":154,"thread_start_us":183836,"threads_started":2,"wal-append.queue_time_us":59302}
I20260709 14:53:29.712198  7529 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:29.683300  7047 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 14:53:29.704277  6748 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:29.699620  7463 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 14:53:29.709131  7499 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.712102  7262 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.673277  7476 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:29.716548  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1365) took 4102 ms. Trace:
I20260709 14:53:29.719880  2757 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:29.745095  3542 rpcz_store.cc:276] 0709 14:53:25.614166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.691177 (+3077011us) service_pool.cc:224] Handling call
0709 14:53:29.405686 (+714509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.413747 (+  8061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.716294 (+302547us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54002baf50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:29.716476 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641703296}
I20260709 14:53:29.746466  2604 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:29.720567  3536 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:29.728422  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1481) took 1684 ms. Trace:
I20260709 14:53:29.697341  7396 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:29.729494  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:29.740028  7316 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:29.740731  7308 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:53:29.734202  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:29.735773  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59078.1us
I20260709 14:53:29.765444  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.766757  2679 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:29.714787  7464 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:29.703018  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.736357  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.680150  7287 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.771643  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1039) took 4430 ms. Trace:
I20260709 14:53:29.772498  7522 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.773478  7287 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.743026  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.775065  3537 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 9. Current term is 10. Ops: []
I20260709 14:53:29.775466  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.776345  7476 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.776785  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.777565  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:29.754789  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.764055  7502 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 14:53:29.779148  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.780045  3543 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.780772  3545 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:29.781083  3543 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:29.781533  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1376) took 1429 ms. Trace:
I20260709 14:53:29.781678  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1377) took 1429 ms. Trace:
I20260709 14:53:29.782126  3543 rpcz_store.cc:276] 0709 14:53:28.352607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.780592 (+1427985us) service_pool.cc:224] Handling call
0709 14:53:29.780893 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.781447 (+   554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.781626 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.763823  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 823) took 4534 ms. Trace:
W20260709 14:53:29.771019  7491 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.747s	user 0.006s	sys 0.008s
I20260709 14:53:29.774012  7451 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:29.807664  7484 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.809327  2682 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:29.776975  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1051) took 2727 ms. Trace:
I20260709 14:53:29.779256  3536 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:29.811015  7318 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.813941  7318 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.819176  2752 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:29.753579  2758 rpcz_store.cc:276] 0709 14:53:28.044277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.721099 (+1676822us) service_pool.cc:224] Handling call
0709 14:53:29.721564 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.726474 (+  4910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.728356 (+  1882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.819826  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:29.771107  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:29.784008  7271 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:29.771570  7538 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:29.781922  3545 rpcz_store.cc:276] 0709 14:53:28.352140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.780235 (+1428095us) service_pool.cc:224] Handling call
0709 14:53:29.780579 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.781253 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.781475 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.767707  7262 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:29.786553  7317 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.767166  7396 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:29.828652  7316 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 14:53:29.793182  3547 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:29.832917  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1177) took 2861 ms. Trace:
I20260709 14:53:29.833451  3547 rpcz_store.cc:276] 0709 14:53:26.971623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.701227 (+1729604us) service_pool.cc:224] Handling call
0709 14:53:29.792334 (+1091107us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:53:29.792712 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.832444 (+ 39732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.832816 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383232}
I20260709 14:53:29.772154  3541 rpcz_store.cc:276] 0709 14:53:25.341605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.116712 (+775107us) service_pool.cc:224] Handling call
0709 14:53:26.162996 (+ 46284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:27.065407 (+902411us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:27.065464 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:27.100853 (+ 35389us) log.cc:844] Serialized 26 byte log entry
0709 14:53:27.175941 (+ 75088us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.176004 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.771151 (+2595147us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54003df9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:29.771222 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.771283 (+    61us) raft_consensus.cc:1685] finished
0709 14:53:29.771297 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.771575 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173940,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":151,"spinlock_wait_cycles":1287890688,"thread_start_us":377171,"threads_started":2,"wal-append.queue_time_us":188414,"wal-append.run_cpu_time_us":2267,"wal-append.run_wall_time_us":1025419}
I20260709 14:53:29.733472  3540 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:29.836274  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1378) took 1483 ms. Trace:
I20260709 14:53:29.798884  2607 rpcz_store.cc:276] 0709 14:53:25.240410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:25.282935 (+ 42525us) service_pool.cc:224] Handling call
0709 14:53:25.283403 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:27.399380 (+2115977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.763507 (+2364127us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:29.763743 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4250373504}
W20260709 14:53:29.798077  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.748663  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.803814  7298 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 14:53:29.805337  7464 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.837920  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:29.805966  7511 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.000s
I20260709 14:53:29.811813  3537 rpcz_store.cc:276] 0709 14:53:27.049769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.785509 (+1735740us) service_pool.cc:224] Handling call
0709 14:53:28.785903 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.775530 (+989627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.776698 (+  1168us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54002026d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:29.776910 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274279808}
W20260709 14:53:29.815692  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.816334  7533 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:29.817375  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:29.818325  2756 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 14:53:29.778585  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:29.821127  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50151.6us
I20260709 14:53:29.821790  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.823875  3536 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:29.825439  7293 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:29.826555  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1465) took 3698 ms. Trace:
I20260709 14:53:29.810452  7484 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:29.827294  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.828138  7518 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:29.828159  7538 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.814304  7315 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.836431  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1380) took 1314 ms. Trace:
I20260709 14:53:29.805491  7306 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:29.918952  3537 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:29.918630  3541 rpcz_store.cc:276] 0709 14:53:28.522124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.835549 (+1313425us) service_pool.cc:224] Handling call
0709 14:53:29.835896 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.836111 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.836138 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:29.836181 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:29.836221 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.836273 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.836373 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:29.801590  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.768328  7308 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:29.873358  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104257us
I20260709 14:53:29.914345  3545 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:29.923113  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.886538  3547 rpcz_store.cc:276] 0709 14:53:28.353014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.834950 (+1481936us) service_pool.cc:224] Handling call
0709 14:53:29.835459 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.835770 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.835813 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:29.835886 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:29.835939 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.836034 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.836187 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:29.915285  3412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.794s (SENT) [suppressed 36 similar messages]
I20260709 14:53:29.931936  2754 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:29.886566  2759 rpcz_store.cc:276] 0709 14:53:26.127984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.439115 (+1311131us) service_pool.cc:224] Handling call
0709 14:53:27.439760 (+   645us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:27.440342 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:27.440390 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:27.441343 (+   953us) log.cc:844] Serialized 52 byte log entry
0709 14:53:27.742807 (+301464us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:27.742876 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:27.742949 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:29.770242 (+2027293us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:29.817866 (+ 47624us) raft_consensus.cc:1685] finished
0709 14:53:29.817899 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:29.826478 (+  8579us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27574,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":23994,"spinlock_wait_cycles":9250560,"thread_start_us":300899,"threads_started":1,"wal-append.queue_time_us":483379}
I20260709 14:53:29.846880  7271 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.846987  7266 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:29.940326  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:29.932277  7533 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:29.947806  6978 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:29.949443  6978 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:29.951639  7309 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:29.952109  7309 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:29.954559  7518 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:29.955902  7489 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 14:53:29.956863  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:29.958436  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:29.960584  3541 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 14:53:29.958920  7358 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:29.962186  7089 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:29.968376  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1369) took 3020 ms (client timeout 3341 ms). Trace:
W20260709 14:53:29.969123  3546 rpcz_store.cc:269] 0709 14:53:26.947815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.856161 (+908346us) service_pool.cc:224] Handling call
0709 14:53:29.968115 (+2111954us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54002eb8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:29.968288 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508456832}
I20260709 14:53:29.971148  3541 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:29.972643  3541 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:29.917680  7293 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:29.824226  7500 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 14:53:29.977078  3546 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 14:53:29.921988  7306 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:29.928103  7308 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:29.939827  7318 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:29.942081  7350 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:30.012370  7446 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:30.012992  7293 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:30.007303  7318 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:30.015064  7315 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:29.928306  7511 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:30.020467  7538 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:30.015165  7405 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:30.028450  7405 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:30.025336  7499 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:30.030954  7511 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:30.069660  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.073681  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.796205  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:30.075490  2604 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:30.076143  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 279906us
I20260709 14:53:30.076958  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.077905  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:30.079170  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:29.850097  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:29.865370  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.082880  2610 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:29.891165  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:30.090250  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1405) took 3715 ms (client timeout 2761 ms). Trace:
I20260709 14:53:30.094019  2751 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:30.095646  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:30.097903  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 832) took 2920 ms. Trace:
W20260709 14:53:30.103960  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 226048us
W20260709 14:53:30.073496  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.112336  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1292) took 1030 ms. Trace:
I20260709 14:53:30.116852  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:30.119496  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1407) took 2955 ms (client timeout 2958 ms). Trace:
W20260709 14:53:30.124197  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 857) took 4161 ms (client timeout 2019 ms). Trace:
W20260709 14:53:30.129199  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51674.4us
W20260709 14:53:30.177467  3409 connection.cc:256] RPC call timeout handler was delayed by -2.6201607770000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 14:53:30.145054  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:30.145766  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:30.148074  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.149402  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.149904  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.151368  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.153200  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:30.153450  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.156234  2680 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 14:53:30.158720  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:30.161055  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.159765  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.163770  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.158849  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 10, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:53:30.171079  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.176872  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.180259  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:30.182667  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:30.182667  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.188545  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:30.188898  3541 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:30.190423  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1381) took 1485 ms. Trace:
I20260709 14:53:30.218279  3540 rpcz_store.cc:276] 0709 14:53:28.704369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.835701 (+1131332us) service_pool.cc:224] Handling call
0709 14:53:29.835991 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:29.836235 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.836259 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:29.836827 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 14:53:29.994233 (+157406us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:29.994306 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:29.994428 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:30.189635 (+195207us) raft_consensus.cc:1685] finished
0709 14:53:30.189688 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.190063 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":156888,"threads_started":1,"wal-append.queue_time_us":183301}
I20260709 14:53:30.192804  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1069) took 1122 ms. Trace:
I20260709 14:53:30.191044  3546 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 14:53:30.221159  2606 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:30.222990  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.197840  7292 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.199553  6996 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:30.203941   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:30.203857  7301 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:30.240978  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.203536  7271 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:30.012178  7306 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:30.278218  2757 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.018s
I20260709 14:53:30.278898  7306 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:30.212001  3409 connection.cc:256] RPC call timeout handler was delayed by -2.6057607770000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 14:53:30.219857  2607 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 9: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:30.279937  3409 connection.cc:256] RPC call timeout handler was delayed by -2.5085607769999569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 14:53:30.280588  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:30.211805  2760 rpcz_store.cc:269] 0709 14:53:25.962984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:26.701822 (+738838us) service_pool.cc:224] Handling call
0709 14:53:30.123521 (+3421699us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:30.123879 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3836488960}
I20260709 14:53:30.282505  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:30.284250  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:30.225745  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:30.196628  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:30.226244  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 335102us
I20260709 14:53:30.285899  2686 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:30.208691  2605 rpcz_store.cc:269] 0709 14:53:26.374814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.089241 (+714427us) service_pool.cc:224] Handling call
0709 14:53:30.089617 (+3000376us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b5400480654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:30.089999 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5035778432}
I20260709 14:53:30.288076  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:30.288954  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.227545  7515 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 14:53:30.211302  2609 rpcz_store.cc:269] 0709 14:53:27.164214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.828585 (+664371us) service_pool.cc:224] Handling call
0709 14:53:30.118731 (+2290146us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:30.119241 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3962141056}
W20260709 14:53:30.290735  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.291180  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.288105  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.293205  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.229219  2682 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 14:53:30.294179  2682 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 8
I20260709 14:53:30.294580  2682 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 14:53:30.227977  7113 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:30.295007  2682 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:30.295353  7113 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:30.221448  2430 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:30.192708  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:30.229727  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.297959  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:30.234704  7292 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:30.299271  3547 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [8.6-13.8]
I20260709 14:53:30.026464  7309 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:30.210597  2677 rpcz_store.cc:276] 0709 14:53:29.081971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.082105 (+   134us) service_pool.cc:224] Handling call
0709 14:53:30.111548 (+1029443us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:30.111976 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307936128}
I20260709 14:53:30.301491  7309 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:30.236271  3543 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:30.235785  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:30.303787  2752 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:30.238886  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:30.304937  2759 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:30.239889  2680 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:30.245720  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:30.273723  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.287490  2760 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:30.276054  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.276964  7271 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 14:53:30.279002  2757 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:30.142738  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:30.281029  3409 connection.cc:256] RPC call timeout handler was delayed by -2.0765607770000543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 14:53:30.224289  7405 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:30.289814  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.242748  3546 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 7.
I20260709 14:53:30.293217  2604 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:30.274739  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.307972  2756 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:30.314072  2606 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:30.320990  7311 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 14:53:30.322026  3537 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:30.281208  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.324241  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.306484  2752 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 14:53:30.325636  2757 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:30.314788  7271 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:30.333477  6995 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:30.328469  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.325546  2607 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:30.289772  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:30.220496  3542 rpcz_store.cc:276] 0709 14:53:29.070516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.839228 (+768712us) service_pool.cc:224] Handling call
0709 14:53:29.839660 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:29.840156 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:29.840187 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:29.840845 (+   658us) log.cc:844] Serialized 52 byte log entry
0709 14:53:30.191975 (+351130us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:30.192087 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.192216 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:30.192265 (+    49us) raft_consensus.cc:1685] finished
0709 14:53:30.192276 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.192575 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3091,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":41275,"thread_start_us":350284,"threads_started":1,"wal-append.queue_time_us":169352}
I20260709 14:53:30.326700  7515 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:30.328163  2752 pending_rounds.cc:85] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:30.326476  2754 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:30.363489  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:30.342768  7271 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:30.209640  2612 rpcz_store.cc:276] 0709 14:53:27.177970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.395699 (+1217729us) service_pool.cc:224] Handling call
0709 14:53:28.416149 (+ 20450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:28.481122 (+ 64973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:28.481149 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:28.481186 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:28.481207 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.097218 (+1616011us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:30.097324 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.097652 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2182,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":5366,"spinlock_wait_cycles":148378880}
I20260709 14:53:30.365415  7271 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:30.366312  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.367195  2612 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:30.344964  2757 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 6. Current term is 7. Ops: []
I20260709 14:53:30.368927  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1489) took 1012 ms. Trace:
I20260709 14:53:30.349659  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.372520  2610 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:30.372314  2757 rpcz_store.cc:276] 0709 14:53:29.357099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.344087 (+986988us) service_pool.cc:224] Handling call
0709 14:53:30.344672 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.368507 (+ 23835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.368818 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:30.345242  2611 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:53:30.345573  3409 connection.cc:256] RPC call timeout handler was delayed by -1.8344607770000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2425s for a response
I20260709 14:53:30.347010  7405 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:30.347254  7052 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.304513  7464 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 14:53:30.377943  2757 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:30.374647  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.351999  2752 pending_rounds.cc:107] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 14:53:30.452817  2752 raft_consensus.cc:2889] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:30.359120  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.351547  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.350227  2751 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:30.359566  3536 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:30.358441  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.357823  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:30.387719  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89970.9us
I20260709 14:53:30.461113  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:30.463290  3409 connection.cc:256] RPC call timeout handler was delayed by -1.3960809210000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 14:53:30.473755  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.467929  2608 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 7.
I20260709 14:53:30.375494  2755 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:30.457589  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:30.377034  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.484545  2605 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:30.353451  7292 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:30.493618  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1358) took 2294 ms. Trace:
I20260709 14:53:30.379258  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:30.375114  7318 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:30.492969  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129742us
I20260709 14:53:30.494050  2752 rpcz_store.cc:276] 0709 14:53:28.189884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.304923 (+2115039us) service_pool.cc:224] Handling call
0709 14:53:30.305715 (+   792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.481821 (+176106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.484190 (+  2369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:30.500701  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.505677  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.355182  7216 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:30.522133  2610 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:30.531378  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.532774  7096 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.533504  7096 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:30.534294  2606 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:30.535248  2607 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:30.535368  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 192059us
I20260709 14:53:30.536386  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:30.536489  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.538040  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.541152  7499 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:30.355304  7113 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:30.547480  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:30.550329  3545 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.006s
I20260709 14:53:30.477346  7052 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:30.555676  2681 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:30.557090  2606 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:53:30.545809  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.561138  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:30.564234  3545 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:30.352098  7515 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:30.571986  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:30.538707  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:30.575528  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.589214  2607 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:53:30.589603  2752 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:30.591404  3538 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 14:53:30.592653  7489 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.593514  2753 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:30.604166  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 130319us
I20260709 14:53:30.598182  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:30.554596  7262 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.010s
W20260709 14:53:30.656565  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117964us
W20260709 14:53:30.665804  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:30.667258  2611 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:30.639614  2609 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:30.669935  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:30.671139  2607 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 14:53:30.684295  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:30.688022  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 352235us
I20260709 14:53:30.682761  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1516) took 1037 ms. Trace:
I20260709 14:53:30.693181  2607 rpcz_store.cc:276] 0709 14:53:29.638332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.590218 (+951886us) service_pool.cc:224] Handling call
0709 14:53:30.590761 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.671771 (+ 81010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.676081 (+  4310us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:30.694903  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 244324us
I20260709 14:53:30.593642  7489 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:30.716195  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.716952  7114 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:30.717244  2677 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:30.717737  7114 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:30.716231  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:30.723793  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 136837us
I20260709 14:53:30.737520  7411 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:53:30.739745  7266 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.748s	user 0.011s	sys 0.015s
I20260709 14:53:30.741449  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:30.744117  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 365016us
I20260709 14:53:30.744630  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:30.746065  7032 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:30.748577  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:30.749012  6978 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.011s
I20260709 14:53:30.750383  2603 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 14:53:30.756281  2611 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:30.752892  2680 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 14:53:30.757362  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.761590  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1522) took 1121 ms. Trace:
I20260709 14:53:30.746821  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:30.763790  2611 rpcz_store.cc:276] 0709 14:53:29.640360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.677010 (+1036650us) service_pool.cc:224] Handling call
0709 14:53:30.722229 (+ 45219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.761243 (+ 39014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.761501 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:30.769169  3540 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:30.767489  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:30.760552  7032 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:30.775694  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:30.774154  7273 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.779579  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1523) took 1139 ms. Trace:
I20260709 14:53:30.781149  2685 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:30.784976  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.780968  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:30.783792  2607 rpcz_store.cc:276] 0709 14:53:29.640636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.705192 (+1064556us) service_pool.cc:224] Handling call
0709 14:53:30.752705 (+ 47513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.775466 (+ 22761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.775487 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:30.779323 (+  3836us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:30.779366 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.779415 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.779526 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:30.788616  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1371) took 1151 ms. Trace:
I20260709 14:53:30.778271  7325 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:30.793380  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52059.2us
W20260709 14:53:30.781251  7308 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.671s	user 0.013s	sys 0.007s
I20260709 14:53:30.780539  7311 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:30.798164  3536 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:30.786321  7287 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:30.790684  2752 rpcz_store.cc:276] 0709 14:53:29.637301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.590539 (+953238us) service_pool.cc:224] Handling call
0709 14:53:30.590995 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.591498 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:30.591918 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 14:53:30.782684 (+190766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:30.782781 (+    97us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:30.788226 (+  5445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.788315 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.788540 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":276,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7218,"spinlock_wait_cycles":22400,"thread_start_us":186439,"threads_started":1}
I20260709 14:53:30.788159  7096 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:30.773036  7262 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 14:53:30.798252  7325 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:30.814172  2682 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:53:30.800465  7216 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 14:53:30.824841  3543 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.005s
I20260709 14:53:30.835894  3543 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:30.820695  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.811048  6978 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:30.835811  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:30.835536  7216 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:30.850306  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96393.6us
I20260709 14:53:30.849735  7293 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:30.853618  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 105232us
I20260709 14:53:30.836097  7325 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:30.825520  7441 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 14:53:30.855736  3540 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:30.853163  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:30.791142  7273 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:30.856689  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 110038us
I20260709 14:53:30.807255  7308 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:30.853909  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 90265.1us
I20260709 14:53:30.856689  7293 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:30.866266  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.871069  2682 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:30.867048  7297 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:30.867200  2752 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:30.849844  7216 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:30.845470  6978 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:30.873381  3543 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:30.879726  7297 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:30.901269  2754 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.005s
I20260709 14:53:30.897871  3537 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 14:53:30.903235  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 162088us
W20260709 14:53:30.898658  2612 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.006s
W20260709 14:53:30.899629  7555 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 14:53:30.899930  3536 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:30.900746  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53293.8us
I20260709 14:53:30.893850  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.904503  2612 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:30.902065  2754 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:30.863579  7052 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:30.914909  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:30.906095  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57278.7us
I20260709 14:53:30.911831  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:30.887008  7533 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:30.900369  6978 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:30.923906  7473 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:30.924446  7311 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:30.925134  2677 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:30.933425  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:30.939464  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1367) took 1436 ms. Trace:
I20260709 14:53:30.940148  2757 rpcz_store.cc:276] 0709 14:53:29.499809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.476121 (+976312us) service_pool.cc:224] Handling call
0709 14:53:30.557091 (+ 80970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.645688 (+ 88597us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:30.682245 (+ 36557us) log.cc:844] Serialized 14 byte log entry
0709 14:53:30.935122 (+252877us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:30.935186 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:30.935234 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.935290 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.935448 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32917,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2367,"raft.queue_time_us":8818,"spinlock_wait_cycles":241024,"thread_start_us":195375,"threads_started":1}
I20260709 14:53:30.932333  7489 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:30.959263  7454 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:30.961393  7293 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:30.861191  7114 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:30.948989  7557 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.222s	user 0.000s	sys 0.002s
W20260709 14:53:30.962927  7558 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
W20260709 14:53:30.965694  7563 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 14:53:30.967387  7562 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 14:53:30.969702  7297 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:30.971010  7560 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 14:53:30.973073  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1488) took 1616 ms. Trace:
I20260709 14:53:30.973605  2760 rpcz_store.cc:276] 0709 14:53:29.356365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.358369 (+1002004us) service_pool.cc:224] Handling call
0709 14:53:30.359521 (+  1152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.360269 (+   748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.360305 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:30.361179 (+   874us) log.cc:844] Serialized 26 byte log entry
0709 14:53:30.639248 (+278069us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:30.639296 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.639401 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:30.972732 (+333331us) raft_consensus.cc:1685] finished
0709 14:53:30.972771 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:30.972976 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1807,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":224,"thread_start_us":277476,"threads_started":1,"wal-append.queue_time_us":431448}
W20260709 14:53:30.976153  7271 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.005s
I20260709 14:53:30.976327  2760 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:30.959822  2752 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 14:53:30.977200  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1498) took 1039 ms. Trace:
I20260709 14:53:30.977641  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1494) took 1048 ms. Trace:
I20260709 14:53:31.003165  2760 rpcz_store.cc:276] 0709 14:53:29.937490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.975368 (+1037878us) service_pool.cc:224] Handling call
0709 14:53:30.976066 (+   698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.976860 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.977133 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:30.932612  7311 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:30.993188  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96805.7us
I20260709 14:53:30.998633  2677 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:30.920783  7262 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 14:53:31.003305  2752 rpcz_store.cc:276] 0709 14:53:29.929292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.932943 (+1003651us) service_pool.cc:224] Handling call
0709 14:53:30.959458 (+ 26515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.977225 (+ 17767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.977576 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:31.006584  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80090us
I20260709 14:53:30.977273  2610 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:31.008169  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:31.008737  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:30.951650  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:31.013556  7556 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.238s	user 0.002s	sys 0.000s
I20260709 14:53:31.014578  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1484) took 2926 ms. Trace:
I20260709 14:53:31.015163  2758 rpcz_store.cc:276] 0709 14:53:28.088248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:29.819976 (+1731728us) service_pool.cc:224] Handling call
0709 14:53:29.820403 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:29.936015 (+115612us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:30.172090 (+236075us) log.cc:844] Serialized 14 byte log entry
0709 14:53:30.498761 (+326671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:30.557418 (+ 58657us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:30.586405 (+ 28987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.014060 (+427655us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:31.014120 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.014347 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10611,"prepare.queue_time_us":439637,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":31249,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":54631,"spinlock_wait_cycles":265055488,"thread_start_us":534154,"threads_started":2,"wal-append.queue_time_us":502944}
I20260709 14:53:31.016712  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:31.018429  2758 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 14:53:31.019213  2680 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:31.019937  2760 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:31.020898  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 80785.4us
I20260709 14:53:31.021006  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1499) took 1083 ms. Trace:
I20260709 14:53:31.021682  2760 rpcz_store.cc:276] 0709 14:53:29.937645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:31.019244 (+1081599us) service_pool.cc:224] Handling call
0709 14:53:31.019670 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:31.020563 (+   893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.020932 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:31.024120  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85367.4us
I20260709 14:53:31.025660  2682 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:53:31.055341  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:31.018814  2758 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 5
I20260709 14:53:31.061307  2758 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 14:53:31.061769  2758 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:31.068801  2755 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:31.006445  7311 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:31.087386  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 283801us
W20260709 14:53:31.092235  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69446.7us
I20260709 14:53:31.073966  7411 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 14:53:31.093863  7502 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.094918  7411 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:31.094859  7502 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.095532  7411 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:31.096817  7411 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:31.097898  7502 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:31.054301  7464 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:31.100039  7308 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:31.098479  7411 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:31.102865  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:31.105772  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:31.112007  2677 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:31.144631  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 128062us
W20260709 14:53:31.146303  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 264992us
W20260709 14:53:31.126250  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 211771us
I20260709 14:53:30.950281  7533 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.166631  2679 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:31.144654  2752 raft_consensus.cc:1217] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Deduplicated request from leader. Original: 11.8->[13.9-13.9]   Dedup: 13.9->[]
I20260709 14:53:31.174016  2752 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:31.174218  2677 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:31.174979  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1502) took 1236 ms. Trace:
I20260709 14:53:31.175549  2752 rpcz_store.cc:276] 0709 14:53:29.938147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:31.061392 (+1123245us) service_pool.cc:224] Handling call
0709 14:53:31.144332 (+ 82940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:31.174487 (+ 30155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.174885 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:31.077042  7441 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:31.178231  7441 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:31.131105  7306 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.847s	user 0.019s	sys 0.000s
I20260709 14:53:31.109547  7292 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 14:53:31.172881  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86238.3us
I20260709 14:53:31.173822  2679 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:31.170439  2685 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:31.142730  7263 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:31.181176  2684 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:31.216576  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:31.216512  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 275972us
I20260709 14:53:31.217900  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:31.226272  7273 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:31.227404  7273 raft_consensus.cc:2764] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 14:53:31.230579  2679 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:31.232024  2679 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:31.194574  7266 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:31.197218  7292 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:31.237651  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:31.239740  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:31.240782  7561 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.390s	user 0.001s	sys 0.001s
I20260709 14:53:31.145533  7579 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:31.246070  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1396) took 1447 ms. Trace:
W20260709 14:53:31.224277  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79375us
I20260709 14:53:31.244964  7579 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:31.257668  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56359.3us
I20260709 14:53:31.270386  2678 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:31.272010  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.251423  3547 rpcz_store.cc:276] 0709 14:53:29.798539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.300394 (+501855us) service_pool.cc:224] Handling call
0709 14:53:30.301014 (+   620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.327237 (+ 26223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.327281 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:30.365847 (+ 38566us) log.cc:844] Serialized 26 byte log entry
0709 14:53:30.664101 (+298254us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:30.664184 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.664299 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:31.245781 (+581482us) raft_consensus.cc:1685] finished
0709 14:53:31.245804 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.245996 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13250,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":23283,"thread_start_us":272835,"threads_started":1,"wal-append.queue_time_us":420902}
I20260709 14:53:31.277057  6995 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:31.277904  6995 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:31.279228  6995 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.282500  7266 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:31.268174  7533 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:31.287322  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49785.9us
I20260709 14:53:31.289101  7533 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:31.290588  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:31.290891  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 140932us
I20260709 14:53:31.292366  7499 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:31.293049  7499 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:31.296598  7499 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:31.297335  7538 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:31.297370  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:31.300554  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:31.303297  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:31.215521  7263 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:31.237703  7292 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:31.304209  2679 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 14:53:31.312994  2756 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.957s	user 0.007s	sys 0.008s
I20260709 14:53:31.317783  2756 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:31.323056  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51503.8us
I20260709 14:53:31.325378  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:31.326519  2682 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:31.351078  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:31.337010  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.355118  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:31.303598  7308 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:31.359687  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 192506us
I20260709 14:53:31.266152  7579 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:31.365888  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1420) took 1555 ms. Trace:
I20260709 14:53:31.316260  7263 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:31.254375  7502 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 14:53:31.373175  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:31.303375  7309 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 14:53:31.375617  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 851) took 1255 ms. Trace:
I20260709 14:53:31.377315  2610 rpcz_store.cc:276] 0709 14:53:30.119947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:31.007715 (+887768us) service_pool.cc:224] Handling call
0709 14:53:31.009711 (+  1996us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:31.012813 (+  3102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:31.012846 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:31.013697 (+   851us) log.cc:844] Serialized 52 byte log entry
0709 14:53:31.145953 (+132256us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:31.370033 (+224080us) log.cc:844] Serialized 14 byte log entry
0709 14:53:31.375131 (+  5098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.375229 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:31.375345 (+   116us) raft_consensus.cc:1685] finished
0709 14:53:31.375362 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.375547 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3981,"prepare.queue_time_us":130566,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":39,"raft.queue_time_us":16121,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":22902,"thread_start_us":286480,"threads_started":2,"wal-append.queue_time_us":262008}
I20260709 14:53:31.282685  7262 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 14:53:31.387844  2684 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:31.388865  7581 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:31.390230  7581 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 14:53:31.394889  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:31.395217  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 179341us
I20260709 14:53:31.396203  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:31.397904  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:31.387642  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:31.399873  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:31.399569  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:31.412281  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:31.375916  7538 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:31.413522  7538 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:31.414053  7538 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:31.415130  2755 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 14:53:31.415661  2755 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:31.416095  2755 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 14:53:31.416294  7575 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:31.416915  7575 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:31.417364  7575 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:31.415508  7266 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 14:53:31.419869  7570 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:53:31.420316  7266 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:31.421458  7266 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:31.422019  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 164512us
W20260709 14:53:31.422462  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52343.9us
I20260709 14:53:31.423516  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:31.424090  7325 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.005s
I20260709 14:53:31.423418  7271 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:31.412429  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:31.372381  7308 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:31.416433  2755 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:31.428347  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 133447us
I20260709 14:53:31.390719  7581 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:31.460793  2752 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:31.426432  7533 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.421458  7297 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.466658  2684 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:31.466765  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:31.443583  7502 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:31.476632  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:31.470817  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:31.472204  7405 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.011s	user 0.014s	sys 0.003s
W20260709 14:53:31.460826  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69872.3us
I20260709 14:53:31.447414  7114 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:31.470619  7566 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.380s	user 0.000s	sys 0.002s
I20260709 14:53:31.435556  7262 consensus_queue.cc:1243] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 14:53:31.424582  7325 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:31.493358  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 107486us
W20260709 14:53:31.472453  7567 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.388s	user 0.001s	sys 0.001s
W20260709 14:53:31.483764  7569 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 14:53:31.501901  7272 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 14:53:31.502945  7464 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:53:31.507743  7589 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 14:53:31.511256  6995 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 14:53:31.511456  7585 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:31.511732  7515 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.498687  7405 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:31.485893  7533 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.487756  7297 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:31.551955  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:31.499048  7309 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 14:53:31.497234  7311 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:31.560595  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 136267us
W20260709 14:53:31.565532  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101327us
I20260709 14:53:31.577384  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 837) took 1864 ms. Trace:
I20260709 14:53:31.591631  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1503) took 1226 ms. Trace:
I20260709 14:53:31.593230  2760 rpcz_store.cc:276] 0709 14:53:30.364970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:31.064879 (+699909us) service_pool.cc:224] Handling call
0709 14:53:31.110108 (+ 45229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:31.146369 (+ 36261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:31.514072 (+367703us) log.cc:844] Serialized 14 byte log entry
0709 14:53:31.590121 (+ 76049us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:31.590331 (+   210us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:31.590615 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.590819 (+   204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.591352 (+   533us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83682,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"thread_start_us":328534,"threads_started":3}
I20260709 14:53:31.598729  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:31.603502  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:31.472776  7581 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.578011  2611 rpcz_store.cc:276] 0709 14:53:29.700514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.767369 (+1066855us) service_pool.cc:224] Handling call
0709 14:53:30.804001 (+ 36632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.846034 (+ 42033us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:31.254558 (+408524us) log.cc:844] Serialized 14 byte log entry
0709 14:53:31.554279 (+299721us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:31.555529 (+  1250us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:31.562946 (+  7417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.563405 (+   459us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.564129 (+   724us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":423830,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"thread_start_us":601436,"threads_started":3,"wal-append.queue_time_us":261787}
I20260709 14:53:31.536634  7441 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 14:53:31.583269  7216 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.006s
I20260709 14:53:31.535514  7308 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:31.540225  7515 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:31.625587  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1071) took 1454 ms. Trace:
I20260709 14:53:31.635058  3538 rpcz_store.cc:276] 0709 14:53:30.164744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.322052 (+157308us) service_pool.cc:224] Handling call
0709 14:53:30.569481 (+247429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.591982 (+ 22501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.618342 (+1026360us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54004b37d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:31.618526 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567522176}
I20260709 14:53:31.625988  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:31.632167  7586 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 14:53:31.635771  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 166242us
I20260709 14:53:31.613567  7585 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:31.641947  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 175367us
I20260709 14:53:31.645179  7430 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 14:53:31.646725  7568 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.409s	user 0.001s	sys 0.001s
W20260709 14:53:31.647203  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 109417us
I20260709 14:53:31.648754  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.672076  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:31.678572  7096 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 14:53:31.680876  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1496) took 1743 ms (client timeout 1882 ms). Trace:
W20260709 14:53:31.681708  2756 rpcz_store.cc:269] 0709 14:53:29.937002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.145098 (+208096us) service_pool.cc:224] Handling call
0709 14:53:31.680502 (+1535404us) spinlock_profiling.cc:243] Waited 95.6 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:31.680726 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219987328}
W20260709 14:53:31.695546  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47402.8us
I20260709 14:53:31.697171  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:31.703464  7464 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:31.704344  7464 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:31.704766  7297 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:31.703691  7602 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:31.705792  7464 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:31.707414  7325 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:31.708392  7573 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:31.708767  7464 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:31.709674  7411 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
W20260709 14:53:31.709612  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83778.9us
I20260709 14:53:31.713204  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.624241  7216 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:31.716666  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 106628us
I20260709 14:53:31.674285  7581 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:31.827486  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 130770us
W20260709 14:53:31.854121  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141097us
W20260709 14:53:31.859087  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 210430us
I20260709 14:53:31.651482  7585 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:31.721860  7596 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:31.857352  7297 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.829551  7096 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:31.853216  7573 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 14:53:31.870862  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 198861us
I20260709 14:53:31.872233  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.869668  7596 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:31.874223  7590 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 14:53:31.874605  7297 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:31.917764  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:31.879590  7584 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.384s	user 0.002s	sys 0.000s
I20260709 14:53:31.880437  7598 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:31.863924  7602 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 14:53:31.964004  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 287482us
I20260709 14:53:31.930047  6996 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:31.979051  7318 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:31.979130  7601 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:31.980703  7601 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:31.981236  7297 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:31.979189  6996 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:31.982163  7311 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:31.979933  7318 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:31.982903  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106455us
I20260709 14:53:31.983057  7311 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:31.983560  7318 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:31.984099  7441 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 14:53:31.984920  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.985939  7499 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:31.980803  7599 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 14:53:31.973181  6978 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.027s	user 0.016s	sys 0.004s
W20260709 14:53:31.974393  7538 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.560s	user 0.018s	sys 0.006s
I20260709 14:53:31.987027  6996 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:31.990391  7263 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.006s
W20260709 14:53:32.024027  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 151891us
I20260709 14:53:31.987156  7411 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:31.979945  7341 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:32.033299  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:32.034183  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 178521us
I20260709 14:53:32.014545  7533 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.015676  7515 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.015612  7287 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:32.020744  7606 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.071677  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:32.073318  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51705.5us
I20260709 14:53:32.014266  7609 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.066452  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.066982  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.073189  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.079391  2682 raft_consensus.cc:2376] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13. Re-sending same reply.
I20260709 14:53:32.079022  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:31.987238  7600 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 14:53:32.039422  6996 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.093456  7473 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:32.093948  7473 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:32.071259  7533 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.095202  7533 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.029985  7297 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 14:53:32.076308  7287 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.095207  7473 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.097077  7473 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:32.026993  7538 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.098299  7533 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.078742  2683 raft_consensus.cc:2376] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14. Re-sending same reply.
I20260709 14:53:32.099325  7538 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.099988  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.101351  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:32.101565  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.102648  2683 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:32.103081  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.118624  7499 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:32.118980  7611 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:32.119613  7499 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:32.090680  7609 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.120358  7499 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:32.015805  6978 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.108615  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1495) took 2175 ms. Trace:
I20260709 14:53:32.063635  7266 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:32.129078  7266 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:32.107152  6995 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:32.129546  7604 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 14:53:31.981948  7601 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.091576  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.115521  7216 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.028343  7263 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.141338  7585 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.132638  7297 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:32.141744  2684 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:32.143005  7266 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:32.066039  7603 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:32.144189  7603 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:32.140693  7502 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.002s
I20260709 14:53:32.146111  7502 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:32.089644  7606 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.031795  7596 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.146929  7585 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:32.156986  7585 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:32.158286  7405 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.002s
I20260709 14:53:32.149852  7612 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.159350  7602 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.159942  7602 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.159926  7612 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.163306  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1350) took 4313 ms. Trace:
I20260709 14:53:32.163865  2753 rpcz_store.cc:276] 0709 14:53:27.849502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.778041 (+928539us) service_pool.cc:224] Handling call
0709 14:53:30.125050 (+1347009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.594261 (+469211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.162407 (+1568146us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:32.163024 (+   617us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002454d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:32.163197 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4125112960}
I20260709 14:53:32.149966  7606 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.151721  6978 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:32.166427  7579 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.003s
I20260709 14:53:32.166574  7585 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.135695  7616 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.167608  7585 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:32.167944  7616 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.168438  7585 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.169220  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.152285  2607 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:32.171427  7454 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:32.172050  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 840) took 2377 ms. Trace:
I20260709 14:53:32.171864  7454 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:32.167150  7579 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.173727  7454 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.174456  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1514) took 2536 ms. Trace:
I20260709 14:53:32.175027  7454 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:32.175935  7579 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:31.996383  7598 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.177915  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1406) took 5014 ms. Trace:
I20260709 14:53:32.178354  2603 rpcz_store.cc:276] 0709 14:53:27.163752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:28.364915 (+1201163us) service_pool.cc:224] Handling call
0709 14:53:28.365446 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.750958 (+2385512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.177557 (+1426599us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400345b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:32.177817 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5483022080}
I20260709 14:53:32.178997  7612 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.171787  6978 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:32.181181  7263 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:32.181461  7596 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:32.182034  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.183413  7430 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:32.183928  7430 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:32.167088  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:32.182658  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.189863  2680 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:32.190881  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1349) took 1170 ms. Trace:
I20260709 14:53:32.191607  2680 rpcz_store.cc:276] 0709 14:53:31.020114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:31.064260 (+ 44146us) service_pool.cc:224] Handling call
0709 14:53:32.166892 (+1102632us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:32.190723 (+ 23831us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30286720}
W20260709 14:53:32.192709  7588 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.680s	user 0.008s	sys 0.014s
I20260709 14:53:32.172569  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.132906  7308 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.193938  7606 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:32.194193  7308 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.142720  7297 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:32.136581  7292 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.764s	user 0.009s	sys 0.006s
I20260709 14:53:32.199422  7292 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:32.201828  7593 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.490s	user 0.005s	sys 0.005s
I20260709 14:53:32.202039  7292 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:32.209134  7502 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:32.211910  6995 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:32.189782  7609 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.212575  6995 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:32.214699  6995 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.215947  7489 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:32.216677  7489 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:32.217176  6995 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:32.220198  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 853) took 2071 ms (client timeout 1729 ms). Trace:
I20260709 14:53:32.220722  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:32.220949  2681 rpcz_store.cc:269] 0709 14:53:30.149081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.150682 (+  1601us) service_pool.cc:224] Handling call
0709 14:53:32.219800 (+2069118us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:32.220031 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354432256}
I20260709 14:53:32.069377  7411 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.172487  2607 rpcz_store.cc:276] 0709 14:53:29.794523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.790455 (+995932us) service_pool.cc:224] Handling call
0709 14:53:30.831174 (+ 40719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:32.171068 (+1339894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.171770 (+   702us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003c8054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:32.171985 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3037742976}
I20260709 14:53:32.125218  2757 rpcz_store.cc:276] 0709 14:53:29.933156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.950121 (+1016965us) service_pool.cc:224] Handling call
0709 14:53:31.065477 (+115356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:31.093633 (+ 28156us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:31.143379 (+ 49746us) log.cc:844] Serialized 14 byte log entry
0709 14:53:31.196938 (+ 53559us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:31.197552 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 14:53:31.197707 (+   155us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:31.197739 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.197808 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:31.685718 (+487910us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b1800071a40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:32.108207 (+422489us) raft_consensus.cc:1685] finished
0709 14:53:32.108364 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:32.108540 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162462,"prepare.queue_time_us":50989,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":25582,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":308,"spinlock_wait_cycles":34193792,"thread_start_us":52870,"threads_started":1,"wal-append.queue_time_us":242062}
I20260709 14:53:32.174918  2606 rpcz_store.cc:276] 0709 14:53:29.637739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.564939 (+927200us) service_pool.cc:224] Handling call
0709 14:53:30.668391 (+103452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.682561 (+ 14170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.682613 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:30.704680 (+ 22067us) log.cc:844] Serialized 26 byte log entry
0709 14:53:31.062378 (+357698us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:31.062451 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.062559 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:31.643169 (+580610us) raft_consensus.cc:1685] finished
0709 14:53:31.643194 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:32.174142 (+530948us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b54002feed0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:32.174382 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6893,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":13936,"spinlock_wait_cycles":173698304,"thread_start_us":257078,"threads_started":1,"wal-append.queue_time_us":537852}
I20260709 14:53:32.234001  7603 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:32.241771  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69308.5us
I20260709 14:53:32.170616  7616 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:32.283064  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83235.2us
I20260709 14:53:32.242015  7405 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:32.285189  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.286918  2757 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:32.289088  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.290539  2752 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:32.291869  7301 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:32.291980  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.292363  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:32.293238  7301 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:32.293951  7301 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:32.294620  2609 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 14:53:31.367653  2612 rpcz_store.cc:276] 0709 14:53:29.794734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.914453 (+1119719us) service_pool.cc:224] Handling call
0709 14:53:30.924213 (+  9760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:30.924601 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:31.109265 (+184664us) log.cc:844] Serialized 14 byte log entry
0709 14:53:31.347457 (+238192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:31.347571 (+   114us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:31.347650 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:31.347776 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:31.348015 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":351570,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":33336,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":7788,"thread_start_us":261425,"threads_started":2}
I20260709 14:53:32.295675  7301 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.296773  2753 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:32.290570  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.304175  2756 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:32.304499  7585 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:32.304781  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 858) took 2128 ms (client timeout 2264 ms). Trace:
W20260709 14:53:32.305233  2678 rpcz_store.cc:269] 0709 14:53:30.176501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.181907 (+  5406us) service_pool.cc:224] Handling call
0709 14:53:32.304579 (+2122672us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000f0354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:32.304693 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2366439296}
I20260709 14:53:32.305577  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.305727  2755 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:32.306245  2756 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:32.306923  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1196) took 2182 ms. Trace:
I20260709 14:53:32.218034  7489 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.270025  7411 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.307960  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.315701  2751 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:32.281817  7609 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:32.307323  3546 rpcz_store.cc:276] 0709 14:53:30.124620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.334415 (+209795us) service_pool.cc:224] Handling call
0709 14:53:30.348322 (+ 13907us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:30.669794 (+321472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.669851 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:30.732768 (+ 62917us) log.cc:844] Serialized 52 byte log entry
0709 14:53:31.012166 (+279398us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:31.012237 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.306469 (+1294232us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400225454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:32.306536 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:32.306588 (+    52us) raft_consensus.cc:1685] finished
0709 14:53:32.306601 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:32.306847 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9651,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":14825,"spinlock_wait_cycles":587731200,"thread_start_us":208929,"threads_started":1,"wal-append.queue_time_us":431461}
I20260709 14:53:32.326318  7611 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.327083  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.327381  7611 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.328238  7446 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:32.329993  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.331061  7411 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.331636  7602 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.331974  7411 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.333943  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 867) took 3063 ms. Trace:
I20260709 14:53:32.335089  7606 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.335842  7606 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.338764  7308 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.339038  7473 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.339677  2751 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:32.339578  7473 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:32.346465  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.326531  7489 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.342145  7611 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.347256  7292 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:32.343945  7606 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.369063  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1519) took 2729 ms. Trace:
W20260709 14:53:32.385550  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.334357  2759 rpcz_store.cc:276] 0709 14:53:29.270155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.306103 (+1035948us) service_pool.cc:224] Handling call
0709 14:53:30.485453 (+179350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:30.589203 (+103750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:30.589252 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:30.590018 (+   766us) log.cc:844] Serialized 26 byte log entry
0709 14:53:30.781693 (+191675us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:30.781750 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:30.781836 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:31.619361 (+837525us) raft_consensus.cc:1685] finished
0709 14:53:31.619398 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:32.333630 (+714232us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400570b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:32.333871 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59354,"raft.queue_time_us":88893,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"spinlock_wait_cycles":399499520,"thread_start_us":191132,"threads_started":1,"wal-append.queue_time_us":342125}
I20260709 14:53:32.348373  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.387245  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.387681  2760 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:32.353731  7411 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.385986  2609 rpcz_store.cc:276] 0709 14:53:29.639400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.673245 (+1033845us) service_pool.cc:224] Handling call
0709 14:53:30.683986 (+ 10741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:32.295101 (+1611115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.368703 (+ 73602us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:32.368986 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3703176576}
I20260709 14:53:32.392721  7611 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:32.393371  7611 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.358716  7446 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:32.382795  7575 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.952s	user 0.016s	sys 0.004s
I20260709 14:53:32.341068  7609 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:32.402535  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 72625.6us
W20260709 14:53:32.405453  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.409346  2685 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
W20260709 14:53:32.392397  7581 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.538s	user 0.015s	sys 0.000s
W20260709 14:53:32.398030  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 330400us
I20260709 14:53:32.409885  2685 pending_rounds.cc:85] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 7
I20260709 14:53:32.429654  2685 pending_rounds.cc:107] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 14:53:32.429757  2681 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:32.430166  2685 raft_consensus.cc:2889] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:32.396420  7308 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 14:53:32.437912  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.437964  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:32.441854  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:32.443064  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.399173  7292 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:32.472471  2679 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:32.402232  7411 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.474637  2609 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:32.472716  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.480037  2684 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:32.480368  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.481577  3542 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:32.484102  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:32.434378  7114 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.415508  7405 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:32.478101  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47972.2us
W20260709 14:53:32.503144  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 347203us
W20260709 14:53:32.476712  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.012s	sys 0.011s
I20260709 14:53:32.434634  7606 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:32.524608  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.488575  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:32.526984  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.012s	sys 0.012s
W20260709 14:53:32.527930  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 697455us
I20260709 14:53:32.528163  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.527937  2606 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:32.488839  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.531872  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.403865  7446 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.559796  7430 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.560822  2678 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:32.561863  7446 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.562361  7430 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.564234  7609 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:32.565233  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85971.2us
I20260709 14:53:32.565369  7430 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:32.566578  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:32.566746  7430 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:32.567226  7611 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.404429  7575 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.540032  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.570851  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:32.539865  7606 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.540350  3538 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:32.540911  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.573179  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:32.541070  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.576409  2610 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:32.541728  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.577692  2611 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:32.484061  7581 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.534687  7287 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.534480  7114 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.593791  7595 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:32.488334  7499 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 14:53:32.594617  7595 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.538329  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.547614  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.489861  7297 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:32.549623  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110443us
I20260709 14:53:32.600419  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:32.550963  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.553234  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.553568  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.603148  2605 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:32.554988  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.556321  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.477591  7502 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:32.570096  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 182987us
I20260709 14:53:32.609443  3547 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:32.609746  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.540771  7439 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 14:53:32.575852  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.589644  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:32.616386  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.616750  3541 raft_consensus.cc:2410] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 14:53:32.592857  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64097.2us
I20260709 14:53:32.569103  7341 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.620956  7341 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.621471  7613 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 14:53:32.623412  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.623740  7341 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.612143  3542 raft_consensus.cc:2410] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 14:53:32.624260  7508 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:32.571597  7533 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:32.625380  2612 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:32.615644  7585 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.626469  7499 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:32.627421  7096 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:32.627185  7585 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.627802  7473 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.488487  7411 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.629647  7585 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.630235  7411 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:32.630369  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48564.8us
I20260709 14:53:32.536725  7405 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.631289  2608 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:32.631541  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:32.632824  7405 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.633407  7306 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:32.634290  3536 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:32.634873  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.635598  3543 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:32.636199  7575 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.637312  7575 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:32.637626  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:32.638136  7616 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:32.640000  7616 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:32.578084  7263 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:32.640645  7616 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:32.645421  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.645722  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.646222  2680 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:32.647342  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:32.648007  2680 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.648231  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.645454  2685 pending_rounds.cc:85] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 9
I20260709 14:53:32.651500  2685 pending_rounds.cc:107] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 14:53:32.651834  2685 raft_consensus.cc:2889] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:32.654592  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:32.655681  2754 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.648761  7341 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.661590  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.666707  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:32.633502  7405 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:32.676385  2610 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:32.679777  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:32.658280  7271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.011s	sys 0.021s
I20260709 14:53:32.568706  7266 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.687199  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.688612  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:32.688877  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.708868  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.644483  7533 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:53:32.687918  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 343548us
I20260709 14:53:32.692130  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.692130  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.692492  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:32.685571  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 85285.5us
I20260709 14:53:32.715610  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:32.627195  7499 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:32.686352  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.719228  3546 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:32.658349  7585 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:32.627856  7096 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:32.713621  2684 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:32.734176  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.735217  2603 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:32.719503  2751 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:32.735849  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.717924  3542 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:32.664820  7439 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.641785  7616 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:32.722488  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:32.744793  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.649279  7502 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.676165  7473 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:32.725960  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.675673  7508 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:32.726752  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 117139us
I20260709 14:53:32.595781  7581 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.681650  7297 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:32.714511  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:32.761929  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:32.739996  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52946us
I20260709 14:53:32.746644  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.769099  2609 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:32.770318  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.756575  7473 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:32.771260  7297 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:32.772473  7297 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.773546  7473 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:32.711537  7318 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:32.751874  2606 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:32.775097  7318 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:32.775718  7318 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:32.776085  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.777987  2606 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 14:53:32.753273  3543 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:32.780471  3543 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:32.780864  7502 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.781418  2678 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:32.758473  7608 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:32.782123  3543 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:32.782662  7608 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:32.783272  7581 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:32.783818  3543 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:32.783910  7608 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.749323  7439 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:32.740813  7596 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.005s
I20260709 14:53:32.750531  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:32.691958  7271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.012s	sys 0.022s
I20260709 14:53:32.727200  7499 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:32.769873  7096 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.780314  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.720345  7287 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:32.686949  7341 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:32.798722  2604 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:32.788244  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:32.799281  2605 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 8.
I20260709 14:53:32.789772  3542 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:32.759580  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.800418  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.802511  7114 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.804459  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:32.798312  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:32.763363  7533 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:32.806409  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.814082  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.815481  2751 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 13 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 14:53:32.811038  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:32.811398  2605 raft_consensus.cc:2410] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:32.695272  7292 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:32.740600  7585 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:32.820222  3542 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 14:53:32.814635  2683 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:32.836125  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:32.835287  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 279305us
I20260709 14:53:32.815335  7439 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.842357  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.844355  2604 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:53:32.849261  7595 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:32.850224  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:32.851392  7585 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:32.851595  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.852535  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:32.854261  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:32.855753  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.856384  7341 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:32.857090  7341 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:32.857421  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:32.819850  7581 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:32.856907  7454 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.006s
I20260709 14:53:32.852939  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.870648  2683 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 14:53:32.871095  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111607us
I20260709 14:53:32.858798  3543 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:32.831382  7446 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 14:53:32.859006  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 227565us
W20260709 14:53:32.873539  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59585.1us
W20260709 14:53:32.879334  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 330264us
I20260709 14:53:32.881183  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:32.842675  7623 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:53:32.872247  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:32.882469  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:32.879456  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:32.838012  7301 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:32.884234  3541 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:32.888049  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.857512  7341 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:32.899952  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.896327  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:32.885229  6995 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.667s	user 0.016s	sys 0.001s
W20260709 14:53:32.910991  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:32.933754  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:32.934549  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77207.3us
I20260709 14:53:32.935060  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:32.943681  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57331us
W20260709 14:53:32.945927  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66597.3us
I20260709 14:53:32.936018  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:32.953612  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.954572  2605 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:32.957855  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.958088  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:32.963150  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48951.9us
W20260709 14:53:32.963428  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78549.9us
I20260709 14:53:32.965548  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:32.901245  7624 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:32.907572  7341 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:32.908725  7602 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:32.971237  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:32.909369  7308 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:32.974773  2684 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:32.955758  7606 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:32.977245  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43608.9us
I20260709 14:53:33.013926  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:32.908937  7585 raft_consensus.cc:2764] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:53:32.888348  7613 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 14:53:32.987731  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.019274  2605 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 9.
I20260709 14:53:32.888772  7595 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 14:53:32.888810  7114 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 14:53:32.976815  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:32.952827  7446 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 14:53:32.962170  7473 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:32.980185  7301 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:33.012387  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:33.017235  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55979.9us
I20260709 14:53:32.981498  7502 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:32.954320  7271 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 14:53:33.059161  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:33.007867  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 219670us
I20260709 14:53:32.959080  7515 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 14:53:32.983258  7602 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:33.099846  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:33.082588  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:33.081514  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 146528us
I20260709 14:53:33.005764  7308 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:32.974025  6995 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 14:53:32.962720  7625 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 14:53:33.115765  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:33.099583  3542 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:33.115705  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:33.106328  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:33.135674  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.143772  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.144902  2684 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:33.078724  7266 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:33.057216  7616 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:33.105690  7502 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.085605  7608 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:33.018098  7411 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 14:53:33.166357  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:33.152889  2680 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.504s	user 0.015s	sys 0.000s
I20260709 14:53:33.172274  2680 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:33.110270  7533 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:33.168004  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:33.136646  7606 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:33.180133  2759 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:33.179970  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.182018  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.183982  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.152526  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.125469  7308 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:33.119824  7602 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:33.154199  7585 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.136970  7473 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:33.195298  7473 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:33.198257  7585 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.192260  3545 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:33.109125  7287 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 14:53:33.192451  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:33.192556  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:33.203477  2756 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:33.193897  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.204011  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.204938  3545 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:33.205045  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.205868  2680 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:33.205907  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.206801  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.209514  2680 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:33.204011  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.211674  2605 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:33.212752  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.208365  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:33.219033  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.157469  7616 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.206666  3545 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:33.251360  2605 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:33.158262  7301 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:33.265980  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.171124  7502 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:33.267133  3541 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:33.261137  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:33.146533  7508 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 14:53:33.211309  7585 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:33.273325  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:33.273890  7635 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
W20260709 14:53:33.275297  7631 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 14:53:33.274874  7301 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:33.276211  7638 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 14:53:33.276901  7297 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:33.267622  7616 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:33.285862  7411 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:33.287891  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:33.296875  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 161425us
I20260709 14:53:33.296747  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.299332  2604 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:33.308295  2605 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:53:33.269131  7533 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:33.280283  7389 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
W20260709 14:53:33.325757  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:33.357383  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:33.268605  7613 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 14:53:33.349623  3545 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:33.225199  7602 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:33.353237  3541 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:33.321611  7114 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:33.356388  7585 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.347358  7411 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:33.378746  3541 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:33.375182  3545 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 14:53:33.364075  7630 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.364s	user 0.001s	sys 0.001s
I20260709 14:53:33.383229  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:33.385628  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:33.388087  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.390780  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:33.381464  7318 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.003s
I20260709 14:53:33.364392  7643 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:33.368614  7114 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.381220  7411 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:33.368564  7613 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:33.382458  7650 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.359232  7297 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:33.385663  7602 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:33.427529  7318 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.439313  7318 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:33.425556  7114 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:33.445096  7297 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:33.447901  7626 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 14:53:33.448850  7533 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:33.449713  7646 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 14:53:33.450721  7598 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:33.455459  7634 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.456980  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1384) took 1156 ms. Trace:
I20260709 14:53:33.456140  7634 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:33.467518  2604 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:53:33.477329  2609 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.687s	user 0.012s	sys 0.002s
W20260709 14:53:33.468750  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.005s	sys 0.017s
W20260709 14:53:33.504725  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.009s	sys 0.024s
W20260709 14:53:33.505605  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 612429us
I20260709 14:53:33.457499  2755 rpcz_store.cc:276] 0709 14:53:32.300780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.306641 (+  5861us) service_pool.cc:224] Handling call
0709 14:53:32.307023 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:32.307444 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:32.307780 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 14:53:32.812134 (+504354us) log.cc:844] Serialized 14 byte log entry
0709 14:53:32.817541 (+  5407us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:32.831553 (+ 14012us) log.cc:844] Serialized 26 byte log entry
0709 14:53:32.831735 (+   182us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:32.831767 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.831852 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:33.456727 (+624875us) raft_consensus.cc:1685] finished
0709 14:53:33.456752 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:33.456912 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1558,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":113,"raft.queue_time_us":3277,"raft.run_wall_time_us":16260,"spinlock_wait_cycles":22014848,"thread_start_us":484422,"threads_started":1,"wal-append.queue_time_us":593692}
I20260709 14:53:33.381685  7585 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:33.507714  2609 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:33.512813  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:33.516273  7341 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.002s
I20260709 14:53:33.516821  7341 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:33.519555  7648 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 14:53:33.521577  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.522627  2603 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 14:53:33.508987  7632 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 14:53:33.483299  7581 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:33.524534  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1331) took 1046 ms. Trace:
I20260709 14:53:33.547427  2679 rpcz_store.cc:276] 0709 14:53:32.477720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.480817 (+  3097us) service_pool.cc:224] Handling call
0709 14:53:32.481190 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:32.525253 (+ 44063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:32.525294 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:32.536683 (+ 11389us) log.cc:844] Serialized 52 byte log entry
0709 14:53:32.809158 (+272475us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:32.809217 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.809292 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:33.524276 (+714984us) raft_consensus.cc:1685] finished
0709 14:53:33.524303 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:33.524470 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7196,"raft.queue_time_us":7348,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9413,"thread_start_us":247395,"threads_started":1,"wal-append.queue_time_us":439637}
I20260709 14:53:33.539738  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:33.544361  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153640us
I20260709 14:53:33.560068  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.536615  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:33.524361  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:33.599387  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59777.4us
I20260709 14:53:33.602489  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:33.600346  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63838.1us
I20260709 14:53:33.599875  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.562894  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:33.602977  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:33.623596  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:33.524669  7598 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:33.527549  7341 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:33.631371  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.008s	sys 0.011s
I20260709 14:53:33.545220  7581 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:33.532246  7634 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:33.524245  7266 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:33.554780  7640 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.527871  7600 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 14:53:33.636782  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.009s	sys 0.012s
W20260709 14:53:33.675989  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 576205us
I20260709 14:53:33.462671  7650 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:33.646615  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46846us
I20260709 14:53:33.646955  7609 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:33.647567  2679 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:53:33.647909  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:33.683203  2678 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:33.648644  2685 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:33.544669  7287 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:33.522820  7606 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:33.661697  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:33.611017  7671 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:33.641140  7114 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:33.798300  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 290588us
I20260709 14:53:33.531957  7585 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:33.455533  7613 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.622408  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:33.535280  7672 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.679966  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.675208  7600 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:33.686995  7657 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:33.664211  7616 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:33.852584  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:33.694540  3537 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
I20260709 14:53:33.695984  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1438) took 1172 ms. Trace:
I20260709 14:53:33.696908  2607 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:33.678951  7341 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 14:53:33.867012  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:33.702248  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:33.867569  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:33.867898  2679 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:33.867059  2608 rpcz_store.cc:276] 0709 14:53:32.523945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.524200 (+   255us) service_pool.cc:224] Handling call
0709 14:53:33.695614 (+1171414us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540016d5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:33.695866 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239523200}
I20260709 14:53:33.663595  7533 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:33.815675  7664 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.685155  7640 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:33.660727  7649 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.173s
I20260709 14:53:33.693646  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1432) took 1258 ms. Trace:
W20260709 14:53:33.796703  7638 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 14:53:33.832830  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:33.823779  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 201460us
I20260709 14:53:33.832746  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 866) took 1187 ms. Trace:
W20260709 14:53:33.823439  7308 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.006s
I20260709 14:53:33.818208  2678 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:33.701375  7650 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:33.868584  2680 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:53:33.870393  2685 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 14:53:33.837813  7644 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.538s	user 0.000s	sys 0.001s
I20260709 14:53:33.836889  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.717935  7671 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:33.742626  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59464.1us
I20260709 14:53:33.873219  2612 rpcz_store.cc:276] 0709 14:53:32.440367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.470048 (+ 29681us) service_pool.cc:224] Handling call
0709 14:53:33.693134 (+1223086us) spinlock_profiling.cc:243] Waited 96.3 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:33.693467 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221446528}
W20260709 14:53:33.838485  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 158694us
I20260709 14:53:33.876974  2682 rpcz_store.cc:276] 0709 14:53:32.644984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.645552 (+   568us) service_pool.cc:224] Handling call
0709 14:53:32.645883 (+   331us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:32.646374 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:32.646401 (+    27us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:32.647238 (+   837us) log.cc:844] Serialized 78 byte log entry
0709 14:53:32.930384 (+283146us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:32.930451 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:32.930534 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:33.832399 (+901865us) raft_consensus.cc:1685] finished
0709 14:53:33.832437 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:33.832630 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11970,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":16257,"thread_start_us":252123,"threads_started":1,"wal-append.queue_time_us":1183014}
I20260709 14:53:33.893173  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:33.890878  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:33.843396  7032 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.816296  7670 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:33.895692  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:33.843189  7674 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:33.898295  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.817993  7681 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:33.827381  7654 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:33.900283  2608 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:33.916108  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:33.843483  7680 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:33.920495  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83676.5us
I20260709 14:53:33.720392  7609 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.834877  7318 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:33.902258  2612 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:33.888024  7499 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.079s	user 0.012s	sys 0.005s
I20260709 14:53:33.839363  7673 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:33.941877  2610 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:53:33.904258  2682 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:33.830147  7266 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:33.945724  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.946750  2683 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:53:33.912101  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:33.871186  7533 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.948381  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:33.837170  7602 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 14:53:33.906584  7439 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 14:53:33.845234  7287 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 14:53:33.924610  7575 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:33.921635  2608 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:33.895537  7308 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:33.923255  7678 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 14:53:33.923521  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1091) took 1369 ms. Trace:
I20260709 14:53:33.895219  7671 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:33.923933  7681 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:33.969449  3536 rpcz_store.cc:276] 0709 14:53:32.554530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.554660 (+   130us) service_pool.cc:224] Handling call
0709 14:53:33.923288 (+1368628us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:33.923425 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69350656}
I20260709 14:53:34.007645  2608 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:34.008363  2605 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:33.877323  7664 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:33.952931  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:34.014070  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61279.2us
I20260709 14:53:34.014500  2606 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:33.951723  7533 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:33.953325  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:34.019120  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65941.5us
I20260709 14:53:33.919350  7654 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.891639  7411 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.924979  7601 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:33.921411  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:33.963527  2757 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: []
I20260709 14:53:33.827667  7672 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.932437  7309 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 14:53:33.942366  7673 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:33.904419  7341 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 14:53:33.841650  7600 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:33.914227  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:33.930570  7680 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:33.898267  7032 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.031780  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:34.046106  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 124778us
I20260709 14:53:33.907284  7670 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.047722  7675 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:34.031204  7654 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.051964  7292 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:34.051630  7681 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.053877  7262 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:34.054358  7411 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:33.925873  7674 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.022682  7643 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:34.014283  7266 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:34.013041  7662 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:33.941911  7318 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:33.956635  7297 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:33.944020  7499 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.068934  2606 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:53:34.056283  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.141182  7318 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:34.142117  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 870) took 1495 ms (client timeout 1552 ms). Trace:
W20260709 14:53:34.142787  2681 rpcz_store.cc:269] 0709 14:53:32.646360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.646986 (+   626us) service_pool.cc:224] Handling call
0709 14:53:34.056147 (+1409161us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:34.141993 (+ 85846us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2339456}
I20260709 14:53:34.144659  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.126178  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:34.141716  2430 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 10, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:34.095175  7616 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.066817  2760 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:34.068212  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:34.066444  7650 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 14:53:34.057432  7664 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.055059  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1214) took 1200 ms. Trace:
I20260709 14:53:34.063262  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 878) took 1196 ms. Trace:
I20260709 14:53:34.056955  2608 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:34.076925  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151785us
W20260709 14:53:34.057317  7502 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.753s	user 0.014s	sys 0.002s
W20260709 14:53:34.090667  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 142387us
I20260709 14:53:34.058622  7473 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.056695  7114 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.096081  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.086942  7673 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:34.086222  7341 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:34.056075  7601 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.112947  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.099224  7675 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:34.114948  7662 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:34.191932  7662 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:34.192651  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 66599.6us
I20260709 14:53:34.122556  7308 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.126124  2754 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:34.127480  2759 raft_consensus.cc:1217] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[6.4-11.7]   Dedup: 6.4->[8.5-11.7]
I20260709 14:53:34.195806  2759 pending_rounds.cc:85] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 4
I20260709 14:53:34.196094  2759 pending_rounds.cc:107] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 14:53:34.196452  2759 raft_consensus.cc:2889] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:34.128787  2752 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 14:53:34.134642  7309 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.111395  7643 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.153146  7640 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.054787  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1505) took 1769 ms. Trace:
I20260709 14:53:34.154874  2681 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:34.138671  7499 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.157282  7670 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.157611  7297 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.203018  7309 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.209424  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 865) took 1153 ms. Trace:
I20260709 14:53:34.162513  2677 rpcz_store.cc:276] 0709 14:53:32.867264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.867495 (+   231us) service_pool.cc:224] Handling call
0709 14:53:32.889071 (+ 21576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:32.932169 (+ 43098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:32.932207 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:32.961104 (+ 28897us) log.cc:844] Serialized 26 byte log entry
0709 14:53:33.191311 (+230207us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:33.369792 (+178481us) log.cc:844] Serialized 14 byte log entry
0709 14:53:33.370259 (+   467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:33.370355 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:34.062959 (+692604us) raft_consensus.cc:1685] finished
0709 14:53:34.062991 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:34.063179 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":309100,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":64,"raft.queue_time_us":41046,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":20793,"thread_start_us":290744,"threads_started":2,"wal-append.queue_time_us":923190}
I20260709 14:53:34.109637  7287 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 14:53:34.167824  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99793.6us
I20260709 14:53:34.114290  7600 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.170300  7266 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:34.175335  7690 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:34.177062  7262 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:34.210044  7309 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.178035  7292 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:34.177892  2430 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 6 to 8, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
W20260709 14:53:34.179183  2435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 166) took 43512 ms (client timeout 15000 ms). Trace:
I20260709 14:53:34.181336  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:34.159318  7609 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:34.183991  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:34.187263  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94063us
I20260709 14:53:34.232370  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:34.233141  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:34.193619  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.135829  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.194152  7473 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:34.238215  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.162153  3541 rpcz_store.cc:276] 0709 14:53:32.861585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.393263 (+531678us) service_pool.cc:224] Handling call
0709 14:53:33.393696 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:33.615430 (+221734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:33.615482 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:33.651688 (+ 36206us) log.cc:844] Serialized 52 byte log entry
0709 14:53:33.899510 (+247822us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:33.899567 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:33.899654 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:34.054769 (+155115us) raft_consensus.cc:1685] finished
0709 14:53:34.054795 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:34.054995 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27298,"raft.queue_time_us":258301,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5875,"thread_start_us":406394,"threads_started":2,"wal-append.queue_time_us":351132}
I20260709 14:53:34.165486  7692 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.205962  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1090) took 1654 ms. Trace:
I20260709 14:53:34.239245  2685 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:34.243587  3547 rpcz_store.cc:276] 0709 14:53:32.551986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.552113 (+   127us) service_pool.cc:224] Handling call
0709 14:53:32.552459 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:32.619979 (+ 67520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.205619 (+1585640us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b5400317254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.205876 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100081536}
I20260709 14:53:34.199558  7113 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:34.207831  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1544) took 1461 ms (client timeout 1773 ms). Trace:
W20260709 14:53:34.249218  2609 rpcz_store.cc:269] 0709 14:53:32.746082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.769980 (+ 23898us) service_pool.cc:224] Handling call
0709 14:53:34.207607 (+1437627us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5400346a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:34.207763 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36191104}
I20260709 14:53:34.209949  2612 rpcz_store.cc:276] 0709 14:53:33.055921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.895087 (+839166us) service_pool.cc:224] Handling call
0709 14:53:33.899271 (+  4184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:33.928759 (+ 29488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.209134 (+280375us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b54002fe9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.209344 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6327040}
I20260709 14:53:34.200461  2758 rpcz_store.cc:276] 0709 14:53:32.285259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.289284 (+  4025us) service_pool.cc:224] Handling call
0709 14:53:32.289615 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:32.477708 (+188093us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:32.527691 (+ 49983us) log.cc:844] Serialized 14 byte log entry
0709 14:53:32.819774 (+292083us) log.cc:844] Serialized 14 byte log entry
0709 14:53:32.825839 (+  6065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:32.825910 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:32.825972 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.053551 (+1227579us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b1800351d00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419
0709 14:53:34.054493 (+   942us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.054553 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:34.054716 (+   163us) inbound_call.cc:177] Queueing success response
0709 14:53:34.130144 (+ 75428us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b1800351d00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
Metrics: {"mutex_wait_us":5651,"prepare.queue_time_us":60306,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":96,"raft.queue_time_us":6253,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":5002,"spinlock_wait_cycles":475895296,"thread_start_us":192733,"threads_started":1,"wal-append.queue_time_us":360741}
I20260709 14:53:34.177948  7096 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:34.194821  7650 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:34.267321  7581 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.606s	user 0.004s	sys 0.010s
I20260709 14:53:34.252347  7692 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.257975  2430 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:34.268076  7581 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.248423  7341 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.270319  7341 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 14:53:34.272291  7601 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.231699  7266 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.276816  7266 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.278172  2609 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:34.270179  2430 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:34.279207  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.280066  2685 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:34.167703  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:34.171564  7502 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:34.282349  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 114757us
I20260709 14:53:34.219596  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.283586  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:34.283826  7502 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.284639  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:34.223533  7690 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:34.196736  7675 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:34.218717  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.225986  2604 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:34.226291  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.199430  7643 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:34.251398  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56301.4us
I20260709 14:53:34.209481  7297 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:34.253510  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.258278  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.259851  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.296694  2754 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:34.264273  2612 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:34.281147  2430 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 6 to 7, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:34.249780  7473 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.285655  2677 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.285132  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:34.267385  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.220170  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:34.289129  2679 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 9.
I20260709 14:53:34.292008  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.294246  3545 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:34.295192  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.296073  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 10.
W20260709 14:53:34.303020  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.281188  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.306128  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:34.306983  3541 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.242887  7499 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:34.310458  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.312151  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.316524  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:34.318886  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65464.9us
I20260709 14:53:34.319814  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.287465  7308 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:34.323566  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.328846  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:34.329591  2681 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:34.328107  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.326141  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:34.330578  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.331472  2754 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:53:34.288281  7579 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:34.336141  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.339994  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:34.326745  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 90521.2us
I20260709 14:53:34.341903  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.331192  2758 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15. Re-sending same reply.
I20260709 14:53:34.344022  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.344728  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.310480  7683 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:34.348567  2612 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:34.270210  7692 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:34.355597  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.356582  2685 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 14:53:34.352272  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.345444  2758 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:34.359606  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:34.352310  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.353286  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.361152  2683 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:34.328858  7601 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:34.311443  7699 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.327960  7308 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.366215  7668 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.367026  7668 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.369791  7668 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.332736  7616 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 14:53:34.366230  7308 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:34.343190  7634 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 14:53:34.375682  7634 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:34.376125  7634 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:34.376916  7533 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.377115  7634 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.328016  7664 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:34.387173  7664 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:34.387215  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.387821  7533 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:34.225286  2435 rpcz_store.cc:269] 0709 14:52:50.687699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:50.708920 (+ 21221us) service_pool.cc:224] Handling call
0709 14:52:51.407600 (+698680us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54001130d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5b0090 00007fa16e64ef3a 00007fa1744051cc 00007fa17450337e 00007fa1729c42a4 00007fa1729c4232 00007fa1729c4161 00007fa1729c40dc 00007fa1729c3392 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.138770 (+42731170us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:53:12.795678 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:53:12.903583 (+107905us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:53:12.903639 (+    56us) write_op.cc:435] Acquired schema lock
0709 14:53:12.903658 (+    19us) tablet.cc:662] Decoding operations
0709 14:53:13.201834 (+298176us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:53:13.201976 (+   142us) write_op.cc:641] Partition lock acquired for write op
0709 14:53:13.202018 (+    42us) tablet.cc:685] Acquiring locks for 25 operations
0709 14:53:14.129131 (+927113us) tablet.cc:701] Row locks acquired
0709 14:53:14.177366 (+ 48235us) write_op.cc:260] PREPARE: finished
0709 14:53:14.231079 (+ 53713us) write_op.cc:270] Start()
0709 14:53:14.259978 (+ 28899us) write_op.cc:276] Timestamp: P: 1783608794211943 usec, L: 0
0709 14:53:14.280519 (+ 20541us) op_driver.cc:348] REPLICATION: starting
0709 14:53:14.476114 (+195595us) log.cc:844] Serialized 9853 byte log entry
0709 14:53:18.904577 (+4428463us) op_driver.cc:464] REPLICATION: finished
0709 14:53:19.533961 (+629384us) write_op.cc:301] APPLY: starting
0709 14:53:19.541528 (+  7567us) tablet.cc:1367] starting BulkCheckPresence
0709 14:53:19.695985 (+154457us) tablet.cc:1370] finished BulkCheckPresence
0709 14:53:19.696199 (+   214us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:53:20.227937 (+531738us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:53:20.228675 (+   738us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 14:53:20.228775 (+   100us) write_op.cc:312] APPLY: finished
0709 14:53:20.230077 (+  1302us) log.cc:844] Serialized 219 byte log entry
0709 14:53:20.344479 (+114402us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:20.384601 (+ 40122us) write_op.cc:454] Released schema lock
0709 14:53:20.435678 (+ 51077us) write_op.cc:341] FINISH: Updating metrics
0709 14:53:20.436365 (+   687us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:20.436409 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1712295,"reported_tablets":44,"spinlock_wait_cycles":114871552,"child_traces":[["op",{"apply.queue_time_us":586510,"apply.run_cpu_time_us":47092,"apply.run_wall_time_us":904256,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108577,"num_ops":25,"prepare.queue_time_us":245115,"prepare.run_cpu_time_us":53430,"prepare.run_wall_time_us":2994024,"raft.queue_time_us":2361579,"raft.run_cpu_time_us":178573,"raft.run_wall_time_us":806353,"replication_time_us":4472580,"thread_start_us":4570514,"threads_started":4,"wal-append.queue_time_us":1394254,"wal-append.run_cpu_time_us":64308,"wal-append.run_wall_time_us":5361762}]]}
I20260709 14:53:34.390515  7680 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:34.391904  7698 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:53:34.395776  7616 consensus_queue.cc:1243] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:34.397112  7662 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.355408  7670 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 14:53:34.358759  7262 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 14:53:34.401223  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1533) took 1721 ms (client timeout 1633 ms). Trace:
W20260709 14:53:34.401789  2610 rpcz_store.cc:269] 0709 14:53:32.679691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.692200 (+ 12509us) service_pool.cc:224] Handling call
0709 14:53:34.400930 (+1708730us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001ead54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:34.401141 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2820371200}
I20260709 14:53:34.346606  6996 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:34.318222  7266 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 14:53:34.359007  7292 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 14:53:34.358543  7318 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:34.347090  7411 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:34.349598  7657 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:34.314555  7287 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:53:34.379403  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 164756us
I20260709 14:53:34.425750  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:34.312698  7499 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:34.381279  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.378585  7634 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.387773  7664 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:34.372593  7473 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.365137  7699 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.399374  2677 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:34.402647  2751 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 14:53:34.447978  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:34.448321  2683 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:34.449011  2677 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.331890  7678 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.404178  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.404013  2430 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:34.405151  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:34.407006  2682 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:34.456001  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.457278  3541 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:34.410264  2679 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:34.407739  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
W20260709 14:53:34.418900  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91732.6us
I20260709 14:53:34.432632  2758 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:34.458634  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:34.434407  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:34.347575  7306 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 14:53:34.460776  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:34.443711  2686 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:34.419552  7262 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 14:53:34.425347  6996 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:34.421964  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 197356us
I20260709 14:53:34.473901  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:34.475240  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:34.476663  7704 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 14:53:34.456920  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52815.4us
W20260709 14:53:34.503991  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45501.1us
I20260709 14:53:34.503511  2430 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 7, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:34.503222  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:34.443903  7533 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:34.519515  2610 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 14:53:34.524158  2610 pending_rounds.cc:85] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 6
I20260709 14:53:34.524544  2610 pending_rounds.cc:107] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:34.524838  2610 raft_consensus.cc:2889] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:34.442142  7441 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:34.439775  7499 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:34.444397  7287 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:34.522622  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:34.529875  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.517444  6996 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:34.536602  7702 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 14:53:34.537060  2678 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:53:34.537843  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.537722  2430 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:34.495817  7113 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:34.479009  7454 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.545754  2677 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:34.530452  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:34.567764  7454 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.568879  7711 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:34.569450  7711 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:34.569872  7711 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.571801  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.549738  2612 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:34.550344  3541 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:34.495680  7446 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:34.478597  7609 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:34.562013  2430 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:34.523257  7640 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:34.580777  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:34.581864  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62096.7us
I20260709 14:53:34.528467  7533 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:34.532897  7650 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:34.562359  7308 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.495947  7601 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 14:53:34.597249  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.606594  3541 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:34.599452  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.608773  2430 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 3, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:34.477240  7678 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.616765  2430 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:34.618204  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:34.612524  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.620510  2758 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:34.621778  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:34.622311  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 147184us
I20260709 14:53:34.582721  7473 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.583256  7113 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:34.595398  7533 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:34.544435  7441 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.628573  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:34.632215  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.594964  7446 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:34.600600  7640 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:34.620191  7609 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:34.431669  7690 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.556506  7654 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:34.657872  7668 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.664932  7609 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.665668  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.666391  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.667088  7597 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:34.668303  7701 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 14:53:34.616444  7292 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 14:53:34.704627  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.715399  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 894) took 1357 ms. Trace:
I20260709 14:53:34.716133  2756 rpcz_store.cc:276] 0709 14:53:33.358265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.358631 (+   366us) service_pool.cc:224] Handling call
0709 14:53:33.359155 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:33.460327 (+101172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:33.460363 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:33.460416 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:33.460450 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.714993 (+1254543us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400236ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.715074 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:34.715292 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208628992}
I20260709 14:53:34.718858  2606 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:34.719444  2606 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:34.638104  7473 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.655155  7650 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:34.651445  7613 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:34.672003  7706 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 14:53:34.671880  7113 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.671934  7032 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.743041  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.683928  7668 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.737315  7613 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.738785  2606 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.687749  7662 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.695744  7441 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.728859  7690 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:34.740109  7639 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 14:53:34.735667  7650 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:34.753800  2606 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:34.791097  7446 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.796989  7473 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.798434  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.800071  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.800753  3536 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:34.801749  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.808115  7650 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:34.814772  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:34.815685  3536 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:34.818881  7654 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.692154  7287 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.823781  7699 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:34.825179  7662 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:34.826023  7662 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.829828  7662 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:34.705015  7597 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:34.832681  7597 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:34.834224  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 205703us
I20260709 14:53:34.827229  7292 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.839718  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:34.841833  3536 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:34.838416  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:34.843432  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:34.854110  7600 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.005s
I20260709 14:53:34.854610  7678 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.855582  7717 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 14:53:34.856277  7709 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 14:53:34.847277  7714 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:34.857007  7714 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:34.857535  7714 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:34.845454  7292 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:34.859897  7675 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
W20260709 14:53:34.855831  7671 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.000s
I20260709 14:53:34.867914  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:34.870074  2753 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:34.874243  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:34.856335  7717 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:34.893309  2606 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 14:53:34.760119  7640 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:34.905212  7272 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 14:53:34.915777  7292 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:34.990856  7292 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:34.920871  7654 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:34.991813  7673 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.793s	user 0.011s	sys 0.004s
I20260709 14:53:34.992029  7654 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:34.936677  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:34.892357  7717 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:34.993690  2677 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 14:53:34.943907  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.990697  7695 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:34.995455  7695 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:34.996351  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.997148  7600 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:34.998175  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.997923  7600 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.944345  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1447) took 1407 ms. Trace:
I20260709 14:53:35.000300  7598 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:35.000881  7598 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:34.996411  7695 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.937992  7664 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.002648  7695 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:34.969957  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:34.971414  7686 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.897s	user 0.001s	sys 0.004s
I20260709 14:53:34.972342  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1535) took 2267 ms. Trace:
W20260709 14:53:34.975569  7643 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.662s	user 0.013s	sys 0.000s
I20260709 14:53:34.978682  7724 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:53:34.982178  7668 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.982558  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:34.988140  7699 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.989190  6996 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:34.989329  7657 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:35.023069  7600 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:35.024750  7699 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:35.027034  2651 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.075s (SENT) [suppressed 17 similar messages]
I20260709 14:53:35.027416  7699 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:34.992496  7673 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:34.992925  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1336) took 1113 ms. Trace:
I20260709 14:53:34.998974  7430 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:34.999954  2608 rpcz_store.cc:276] 0709 14:53:33.537119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.009032 (+471913us) service_pool.cc:224] Handling call
0709 14:53:34.009403 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.162663 (+153260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.944006 (+781343us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:34.944243 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108733312}
I20260709 14:53:35.031071  2678 rpcz_store.cc:276] 0709 14:53:33.879892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.901320 (+ 21428us) service_pool.cc:224] Handling call
0709 14:53:34.992710 (+1091390us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:34.992857 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026088320}
I20260709 14:53:34.950878  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:35.033785  3545 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:34.968226  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1092) took 2412 ms (client timeout 1746 ms). Trace:
I20260709 14:53:34.969844  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:34.999754  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.036455  2759 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:34.998499  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:34.994549  7654 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.007656  2607 rpcz_store.cc:276] 0709 14:53:32.704756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.704892 (+   136us) service_pool.cc:224] Handling call
0709 14:53:33.696594 (+991702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:33.870278 (+173684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.972052 (+1101774us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54001d9050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:34.972258 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201408256}
W20260709 14:53:35.034332  3537 rpcz_store.cc:269] 0709 14:53:32.555845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.556007 (+   162us) service_pool.cc:224] Handling call
0709 14:53:34.967809 (+2411802us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002026d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:34.968030 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506707328}
I20260709 14:53:34.864849  7671 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:35.010495  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.040896  7430 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:35.048285  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.048601  7613 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.049010  2753 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:35.050053  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.050117  2607 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:35.050192  7430 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:35.051178  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:35.051776  2607 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:35.007164  7272 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:35.052127  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.016050  2681 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
W20260709 14:53:35.018889  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:34.863729  7675 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:35.023329  7664 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:35.024843  7668 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:35.045053  3545 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:53:35.013055  3536 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:35.057574  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1416) took 2258 ms. Trace:
I20260709 14:53:35.053659  2753 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:35.055051  3542 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:35.069193  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:35.069730  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91056.4us
I20260709 14:53:35.070618  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:35.071357  7272 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.056433  2604 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:35.074503  2754 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:35.074854  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.018769  7643 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:35.078348  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:35.077984  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:35.079324  3537 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:35.066530  3546 rpcz_store.cc:276] 0709 14:53:32.794775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.794948 (+   173us) service_pool.cc:224] Handling call
0709 14:53:32.812734 (+ 17786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:32.902203 (+ 89469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:32.902229 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:32.960603 (+ 58374us) log.cc:844] Serialized 26 byte log entry
0709 14:53:33.473652 (+513049us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:33.473795 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.031957 (+1558162us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54002012d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:35.032009 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:35.032055 (+    46us) raft_consensus.cc:1685] finished
0709 14:53:35.032066 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.052889 (+ 20823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5892,"raft.queue_time_us":6122,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":11492,"spinlock_wait_cycles":131649280,"thread_start_us":390145,"threads_started":1,"wal-append.queue_time_us":623767,"wal-append.run_cpu_time_us":2426,"wal-append.run_wall_time_us":1087188}
I20260709 14:53:35.035704  7711 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:35.099004  3541 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:35.077209  7446 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:35.079985  7613 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:35.101347  7634 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.003s
I20260709 14:53:35.075769  7717 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:35.123762  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:35.121057  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:35.082093  7724 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 14:53:35.077711  7668 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.085954  7657 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 14:53:35.168988  2609 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:35.124816  3537 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
W20260709 14:53:35.206694  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 128805us
I20260709 14:53:35.207348  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.208464  2604 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:35.209537  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.210188  2604 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:35.211130  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.207620  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:35.213696  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:35.204283  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:35.235342  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.235695  7675 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:35.236919  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:35.237809  2607 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 10: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:35.242441  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.243530  2604 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:35.244738  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.245828  2604 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:35.247102  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.248638  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.238678  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.250150  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.250870  2607 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:35.251999  7717 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.253062  7717 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:35.254282  7717 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 14:53:35.255439  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51268.8us
I20260709 14:53:35.257057  7690 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:35.256922  2604 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 14:53:35.257596  7690 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:35.258070  7690 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:35.258515  7717 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:35.259235  7502 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:35.259790  7717 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:35.260485  7502 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:35.260900  7502 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:35.261889  7502 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:35.264528  2759 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:35.132098  7533 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:35.265887  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:35.243705  7729 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 14:53:35.270206  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:35.268823  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:35.277158  2753 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:35.279368  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:35.279515  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.281201  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:35.283459  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 187302us
W20260709 14:53:35.322547  2577 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.882s (SENT)
I20260709 14:53:35.289780  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:35.325301  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:35.179114  7731 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
W20260709 14:53:35.276525  7727 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.080s	user 0.003s	sys 0.001s
I20260709 14:53:35.335389  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:35.321707  7730 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.202s	user 0.003s	sys 0.000s
I20260709 14:53:35.211347  7272 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:35.321364  7032 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:35.400921  7032 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:35.401542  7728 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 14:53:35.403256  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1425) took 1109 ms. Trace:
I20260709 14:53:35.331959  7113 raft_consensus.cc:2764] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 14:53:35.321208  7446 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:35.332597  7624 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:35.338984  7306 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:35.403828  3538 rpcz_store.cc:276] 0709 14:53:34.294135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.994308 (+700173us) service_pool.cc:224] Handling call
0709 14:53:34.994736 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:35.007671 (+ 12935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.007711 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:35.019409 (+ 11698us) log.cc:844] Serialized 26 byte log entry
0709 14:53:35.188385 (+168976us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:35.188459 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.188573 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:35.402906 (+214333us) raft_consensus.cc:1685] finished
0709 14:53:35.402937 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.403156 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12427,"raft.queue_time_us":14970,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":2886,"thread_start_us":122153,"threads_started":1,"wal-append.queue_time_us":332135}
W20260709 14:53:35.448714  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 167595us
W20260709 14:53:35.451869  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126641us
I20260709 14:53:35.506438  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:35.334431  7502 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:35.491168  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.520153  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:53:35.455201  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50121.4us
I20260709 14:53:35.275339  7732 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.177s
W20260709 14:53:35.531391  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.509882  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.401600  7032 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:35.534700  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:35.540392  7733 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 14:53:35.548559  3538 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:35.549712  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1099) took 1122 ms. Trace:
I20260709 14:53:35.550118  3538 rpcz_store.cc:276] 0709 14:53:34.427511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.527101 (+1099590us) service_pool.cc:224] Handling call
0709 14:53:35.548250 (+ 21149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.549324 (+  1074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.549632 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:35.556077  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1457) took 1074 ms. Trace:
I20260709 14:53:35.556804  2608 rpcz_store.cc:276] 0709 14:53:34.481670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.031975 (+550305us) service_pool.cc:224] Handling call
0709 14:53:35.032376 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:35.044313 (+ 11937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.044366 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:35.052894 (+  8528us) log.cc:844] Serialized 52 byte log entry
0709 14:53:35.276707 (+223813us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:35.276755 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.276840 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:35.554380 (+277540us) raft_consensus.cc:1685] finished
0709 14:53:35.554412 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.555980 (+  1568us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2130,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":6268,"thread_start_us":196008,"threads_started":1,"wal-append.queue_time_us":482551}
I20260709 14:53:35.559504  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:35.560880  2608 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:35.563860  7446 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:35.565430  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:35.568457  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1229) took 1160 ms. Trace:
I20260709 14:53:35.490936  7636 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:35.481465  7654 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:35.583963  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.589951  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.590274  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.497519  7624 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:35.496800  7306 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:35.494217  7671 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:35.583779  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 370113us
I20260709 14:53:35.569427  3537 rpcz_store.cc:276] 0709 14:53:34.408591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.184773 (+776182us) service_pool.cc:224] Handling call
0709 14:53:35.212020 (+ 27247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:35.212531 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.212576 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:35.236600 (+ 24024us) log.cc:844] Serialized 26 byte log entry
0709 14:53:35.492335 (+255735us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:35.492401 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.492515 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:35.568144 (+ 75629us) raft_consensus.cc:1685] finished
0709 14:53:35.568177 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.568372 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19910,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":43289,"thread_start_us":255223,"threads_started":1,"wal-append.queue_time_us":325570}
I20260709 14:53:35.601167  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:35.588394  2608 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:35.530414  7573 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:35.533761  7272 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:35.602046  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.604668  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.612170  2759 raft_consensus.cc:2368] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:35.605093  2608 raft_consensus.cc:1217] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Deduplicated request from leader. Original: 10.6->[12.7-15.8]   Dedup: 12.7->[15.8-15.8]
I20260709 14:53:35.608139  2604 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:35.595340  7714 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.008s
I20260709 14:53:35.615952  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.613494  2608 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: [12.7-15.8]
I20260709 14:53:35.597644  7624 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:35.600013  7306 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:35.625275  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:35.618266  7714 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:35.626726  3537 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 14:53:35.617604  7499 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.070s	user 0.011s	sys 0.006s
I20260709 14:53:35.600208  7643 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.630595  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1101) took 1202 ms. Trace:
I20260709 14:53:35.611894  7636 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:35.632436  3537 rpcz_store.cc:276] 0709 14:53:34.428285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.599824 (+1171539us) service_pool.cc:224] Handling call
0709 14:53:35.622936 (+ 23112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.630314 (+  7378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.630540 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:35.620851  7657 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 14:53:35.643815  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.619077  7717 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:35.638218  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.616802  7573 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:35.632782  7650 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.007s
I20260709 14:53:35.649047  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.652343  2753 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:35.633416  7636 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:35.618970  7272 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:35.657491  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.656507  7643 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:35.633715  7499 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.658490  2753 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 10. Current term is 11. Ops: []
W20260709 14:53:35.668289  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 161965us
I20260709 14:53:35.658453  2759 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:35.677665  7272 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:35.678645  7654 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:35.684578  7573 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.684823  7654 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.665279  7643 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:35.688093  7643 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:35.691041  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.692915  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.695235   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:35.699445  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.700383  2753 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:53:35.651688  7650 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:35.708097  2685 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:35.713039  7674 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:35.660310  7533 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:35.757071  2758 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:35.942034  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:36.165247  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.104462  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 899) took 2183 ms. Trace:
I20260709 14:53:36.166705  2755 rpcz_store.cc:276] 0709 14:53:33.921257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.921446 (+   189us) service_pool.cc:224] Handling call
0709 14:53:33.921960 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.142661 (+220701us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:34.143096 (+   435us) log.cc:844] Serialized 14 byte log entry
0709 14:53:34.195068 (+ 51972us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:34.195129 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:34.195196 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.952289 (+757093us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b1800032620. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419
0709 14:53:36.104017 (+1151728us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.104099 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.104354 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361,"prepare.queue_time_us":1653,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":449,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3993,"spinlock_wait_cycles":514187776,"thread_start_us":51101,"threads_started":1,"wal-append.queue_time_us":101485,"wal-append.run_cpu_time_us":2192,"wal-append.run_wall_time_us":1142748}
I20260709 14:53:35.763672  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1427) took 1381 ms. Trace:
W20260709 14:53:36.091840  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.765322  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1102) took 1306 ms. Trace:
W20260709 14:53:36.013936  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 132217us
I20260709 14:53:35.754900  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.661271  7636 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:35.710801  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:35.725194  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:35.821839  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:35.738150  2604 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:36.172560  3545 rpcz_store.cc:276] 0709 14:53:34.382216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.076598 (+694382us) service_pool.cc:224] Handling call
0709 14:53:35.094486 (+ 17888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.094769 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:35.330156 (+235387us) log.cc:844] Serialized 14 byte log entry
0709 14:53:35.761058 (+430902us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:35.761871 (+   813us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:35.763024 (+  1153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.763403 (+   379us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.763598 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":443897,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":69171,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":4973,"thread_start_us":664522,"threads_started":3,"wal-append.queue_time_us":293381}
I20260709 14:53:35.743901  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1103) took 1248 ms. Trace:
I20260709 14:53:36.266125  3547 rpcz_store.cc:276] 0709 14:53:34.495960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.702742 (+1206782us) service_pool.cc:224] Handling call
0709 14:53:35.743077 (+ 40335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.743394 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.743428 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:35.743484 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:35.743522 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.743605 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.743803 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.284106  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.258466  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.287767  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.285528  2759 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:36.289307  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:36.290023  2755 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:36.291361  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107995us
I20260709 14:53:36.177695  3537 rpcz_store.cc:276] 0709 14:53:34.459213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.634253 (+1175040us) service_pool.cc:224] Handling call
0709 14:53:35.653868 (+ 19615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.654147 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:35.663944 (+  9797us) log.cc:844] Serialized 14 byte log entry
0709 14:53:35.762784 (+ 98840us) log.cc:844] Serialized 14 byte log entry
0709 14:53:35.764782 (+  1998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:35.764866 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:35.764940 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.765046 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:35.765234 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12733,"prepare.run_cpu_time_us":406,"prepare.run_wall_time_us":410,"raft.queue_time_us":1531,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":8947,"thread_start_us":68426,"threads_started":1}
I20260709 14:53:35.713522  7499 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:35.846989  7613 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.737s	user 0.015s	sys 0.000s
W20260709 14:53:36.131413  7598 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.098s	user 0.012s	sys 0.006s
W20260709 14:53:35.849331  7744 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 14:53:36.155639  7750 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.201s
I20260709 14:53:35.849723  7272 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:36.303063  7032 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.000s
I20260709 14:53:35.716677  7657 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:35.906939  7113 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:35.704324  7717 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:35.943542  7695 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.940s	user 0.010s	sys 0.006s
W20260709 14:53:35.790023  7735 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 14:53:35.939697  7389 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.329700  2753 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:36.171512  7621 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:35.904541  7710 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.317704  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1344) took 2301 ms. Trace:
I20260709 14:53:36.337656  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:36.318692  7502 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.000s
I20260709 14:53:36.320480  7674 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 14:53:36.346578  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 257876us
I20260709 14:53:36.331106  7405 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.348462  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1233) took 1510 ms. Trace:
I20260709 14:53:36.326803  7613 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:36.356398  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1524) took 1414 ms. Trace:
I20260709 14:53:36.369690  2753 rpcz_store.cc:276] 0709 14:53:34.942095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.319185 (+1377090us) service_pool.cc:224] Handling call
0709 14:53:36.319926 (+   741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.356065 (+ 36139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.356328 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.327579  7650 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.368553  7710 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.371914  2753 pending_rounds.cc:85] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 6
I20260709 14:53:36.372262  2753 pending_rounds.cc:107] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:36.372514  2753 raft_consensus.cc:2889] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:35.762879  7624 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:36.352003  7724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.008s	sys 0.032s
W20260709 14:53:36.373862  7724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.033s
I20260709 14:53:36.365502  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:36.364810  2682 rpcz_store.cc:276] 0709 14:53:34.015962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.455706 (+439744us) service_pool.cc:224] Handling call
0709 14:53:34.456053 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.518173 (+ 62120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:34.518215 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:34.518271 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:34.518317 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.317348 (+1799031us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.317412 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.317618 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142160256}
I20260709 14:53:36.375347  7624 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:36.368510  3537 rpcz_store.cc:276] 0709 14:53:34.838502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.321396 (+1482894us) service_pool.cc:224] Handling call
0709 14:53:36.347551 (+ 26155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.347898 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.347939 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.348018 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:36.348087 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.348170 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.348363 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.362200  2608 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:36.376927  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.343940  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:36.338466  7743 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.019s
I20260709 14:53:36.377511  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1429) took 1298 ms. Trace:
I20260709 14:53:36.377503  2608 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:36.378017  2604 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 8.
W20260709 14:53:36.378027  7743 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.016s	sys 0.019s
I20260709 14:53:36.378099  3537 rpcz_store.cc:276] 0709 14:53:35.078775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.376227 (+1297452us) service_pool.cc:224] Handling call
0709 14:53:36.376723 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.377019 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.377060 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.377127 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:36.377192 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.377272 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.377422 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.379364  7710 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:36.381369  2604 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:36.328904  7643 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 14:53:36.354338  7113 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.336863  7287 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 14:53:36.362368  7748 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.365702  7389 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.327031  7598 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.369215  7654 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:36.345990  7032 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.331128  7272 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.360337  7621 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:36.386109  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1537) took 3658 ms. Trace:
I20260709 14:53:36.352806  7657 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.399956  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1415) took 3651 ms. Trace:
I20260709 14:53:36.401427  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1222) took 2035 ms. Trace:
I20260709 14:53:36.401919  3536 rpcz_store.cc:276] 0709 14:53:34.379592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.066544 (+686952us) service_pool.cc:224] Handling call
0709 14:53:35.067099 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:35.071694 (+  4595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.071732 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:35.072478 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 14:53:35.323774 (+251296us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:35.323837 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:35.323946 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.401189 (+1077243us) raft_consensus.cc:1685] finished
0709 14:53:36.401215 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.401358 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3990,"raft.queue_time_us":4736,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":250819,"threads_started":1,"wal-append.queue_time_us":439596}
I20260709 14:53:36.403015  2608 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:36.400610  3543 rpcz_store.cc:276] 0709 14:53:32.748861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.784863 (+ 36002us) service_pool.cc:224] Handling call
0709 14:53:32.785169 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:32.871978 (+ 86809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.399545 (+3527567us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.399854 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168388480}
I20260709 14:53:36.396451  2611 rpcz_store.cc:276] 0709 14:53:32.727442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.727599 (+   157us) service_pool.cc:224] Handling call
0709 14:53:32.752541 (+ 24942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:32.792466 (+ 39925us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:32.807998 (+ 15532us) log.cc:844] Serialized 14 byte log entry
0709 14:53:33.183494 (+375496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:33.185403 (+  1909us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:33.185493 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.385798 (+3200305us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.385866 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.386034 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6981,"prepare.queue_time_us":16394,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":35195,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":27389,"spinlock_wait_cycles":91068160,"thread_start_us":327220,"threads_started":2,"wal-append.queue_time_us":317073,"wal-append.run_cpu_time_us":2398,"wal-append.run_wall_time_us":1302071}
I20260709 14:53:36.389849  7753 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.395222  7643 consensus_queue.cc:1243] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:36.407385  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1431) took 1328 ms. Trace:
I20260709 14:53:36.387759  7405 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.415360  3536 rpcz_store.cc:276] 0709 14:53:35.079426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.406451 (+1327025us) service_pool.cc:224] Handling call
0709 14:53:36.406878 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.407060 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.407076 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.407109 (+    33us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:36.407133 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.407172 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.407316 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.408774  7032 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:36.399631  7611 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:36.436538  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:36.450750  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:36.452121  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:36.454622  3537 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:36.393478  7748 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.458299  7748 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:36.458972  7032 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:36.449451  7621 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.468463  7405 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:36.468753  7621 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:36.470711  7272 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:36.471555  7756 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 14:53:36.460992  7611 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.419713  7717 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.475708  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1392) took 2350 ms. Trace:
I20260709 14:53:36.476200  2751 rpcz_store.cc:276] 0709 14:53:34.125323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.130648 (+  5325us) service_pool.cc:224] Handling call
0709 14:53:34.131125 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:34.461812 (+330687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.475392 (+2013580us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.475634 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623798656}
I20260709 14:53:36.479751  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:36.481320  2651 connection.cc:256] RPC call timeout handler was delayed by -1.3634249199999431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24070000000000003s for a response
W20260709 14:53:36.482204  7690 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.224s	user 0.013s	sys 0.000s
I20260709 14:53:36.482132  7096 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:36.482867  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.483480  7096 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.420342  7753 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.484398  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.492415  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.504050  2612 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:36.499971  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1363) took 3551 ms. Trace:
I20260709 14:53:36.494379  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1340) took 2595 ms. Trace:
I20260709 14:53:36.504693  2680 rpcz_store.cc:276] 0709 14:53:32.948559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.317860 (+369301us) service_pool.cc:224] Handling call
0709 14:53:33.326634 (+  8774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:33.888866 (+562232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.499499 (+2610633us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.499870 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245587840}
I20260709 14:53:36.500871  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 886) took 3846 ms. Trace:
I20260709 14:53:36.506590  2760 rpcz_store.cc:276] 0709 14:53:32.654446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.319697 (+665251us) service_pool.cc:224] Handling call
0709 14:53:33.320211 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:34.159683 (+839472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.500536 (+2340853us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.500797 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716283776}
I20260709 14:53:36.507232  2755 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:36.508301  2751 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:36.508873  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.510568  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.505491  2683 rpcz_store.cc:276] 0709 14:53:33.899117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.395743 (+496626us) service_pool.cc:224] Handling call
0709 14:53:34.448037 (+ 52294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.463747 (+ 15710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.494086 (+2030339us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b54000c9ed0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:36.494241 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119182592}
I20260709 14:53:36.504089  2610 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:36.489871  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:36.521714  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67244.8us
W20260709 14:53:36.527078  3538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.006s	sys 0.007s
W20260709 14:53:36.528259  3538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.006s	sys 0.007s
W20260709 14:53:36.526957  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.006s	sys 0.009s
W20260709 14:53:36.529443  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.006s	sys 0.010s
I20260709 14:53:36.474601  7611 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:36.489626  7754 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.690s	user 0.007s	sys 0.004s
I20260709 14:53:36.533469  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:36.527736  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.483994  7748 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.535094  2760 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:36.501017  7760 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.484083  7657 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.518877  7670 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:36.529953  7297 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:36.488926  7113 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:36.482770  7690 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.539438  7654 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:36.490638  7389 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:36.488834  7598 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:36.541797  2760 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:36.589435  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.590298  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1405) took 1572 ms. Trace:
I20260709 14:53:36.590510  2604 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 14:53:36.541117  7748 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.554154  7753 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:36.536199  7287 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 14:53:36.556166  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.530809  7032 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:36.558557  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 881) took 2686 ms. Trace:
I20260709 14:53:36.531159  7764 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 14:53:36.463673  7619 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:36.586858  7654 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:36.624292  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.589149  7502 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.562984  7297 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:36.618139  2612 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:36.558281  7405 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.617702  2679 rpcz_store.cc:276] 0709 14:53:33.872140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.349817 (+477677us) service_pool.cc:224] Handling call
0709 14:53:34.379277 (+ 29460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.457909 (+ 78632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.558153 (+2100244us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.558464 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59224960}
W20260709 14:53:36.619953  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 137315us
I20260709 14:53:36.591540  2760 rpcz_store.cc:276] 0709 14:53:35.018460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.537507 (+1519047us) service_pool.cc:224] Handling call
0709 14:53:36.537963 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.589897 (+ 51934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.590210 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.587487  7499 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.631283  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.558506  7670 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.545624  7760 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.640616  2612 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:36.619568  7764 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:36.642765  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.638785  7405 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.694005  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.797082  3536 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:36.798508  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.800195  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.685935  7581 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:36.720384  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.699430  7689 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:36.712936  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.670593  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1407) took 1650 ms. Trace:
I20260709 14:53:36.673699  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1423) took 2380 ms. Trace:
W20260709 14:53:36.802554  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.802735  3542 rpcz_store.cc:276] 0709 14:53:34.293453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.350102 (+ 56649us) service_pool.cc:224] Handling call
0709 14:53:34.362450 (+ 12348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:35.068570 (+706120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.673354 (+1604784us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54002bb454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.673627 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592091520}
I20260709 14:53:36.687268  7700 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.685747  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1448) took 3148 ms. Trace:
I20260709 14:53:36.809181  2605 rpcz_store.cc:276] 0709 14:53:33.537561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.009305 (+471744us) service_pool.cc:224] Handling call
0709 14:53:34.009739 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:34.012782 (+  3043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:34.014112 (+  1330us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:36.427922 (+2413810us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b2c000cb828. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:53:36.449944 (+ 22022us) log.cc:844] Serialized 26 byte log entry
0709 14:53:36.611364 (+161420us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:36.611427 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.611550 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.685403 (+ 73853us) raft_consensus.cc:1685] finished
0709 14:53:36.685428 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.685676 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":879,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":17745,"spinlock_wait_cycles":60761344,"thread_start_us":132877,"threads_started":1,"wal-append.queue_time_us":220336}
I20260709 14:53:36.810705  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1560) took 1106 ms. Trace:
W20260709 14:53:36.690224  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1456) took 2217 ms (client timeout 1619 ms). Trace:
I20260709 14:53:36.627000  7654 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:36.699458  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 898) took 2778 ms. Trace:
W20260709 14:53:36.756260  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.696617  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:36.675102  7768 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.676965  7748 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:36.683641  7306 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.052s	user 0.009s	sys 0.007s
I20260709 14:53:36.720728  7575 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:36.713516  3543 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:36.776155  7585 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:36.632162  7096 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:36.705173  7760 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:36.677593  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1220) took 2312 ms (client timeout 2061 ms). Trace:
W20260709 14:53:36.831665  3541 rpcz_store.cc:269] 0709 14:53:34.365567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.672480 (+306913us) service_pool.cc:224] Handling call
0709 14:53:36.677375 (+2004895us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400317c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:36.677506 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895383936}
I20260709 14:53:36.675572  7737 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:36.622247  7619 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:36.639735  7297 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:36.728499  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1346) took 5781 ms. Trace:
I20260709 14:53:36.689211  7649 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.696094  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:36.847534  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151566us
I20260709 14:53:36.849232  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:36.711172  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:36.769273  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.772586  2759 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:36.773017  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:36.769089  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1230) took 2144 ms. Trace:
I20260709 14:53:36.719928  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:36.852864  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.784754  7717 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:36.786955  7608 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:36.787210  7758 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 14:53:36.787895  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:36.670058  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1100) took 2242 ms. Trace:
I20260709 14:53:36.802147  2760 rpcz_store.cc:276] 0709 14:53:35.020392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.647888 (+1627496us) service_pool.cc:224] Handling call
0709 14:53:36.669948 (+ 22060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.670196 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.670225 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.670281 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:36.670320 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.670377 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.670515 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.863862  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.796049  2604 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:36.835506  7737 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:36.813256  2757 rpcz_store.cc:276] 0709 14:53:33.920900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.921071 (+   171us) service_pool.cc:224] Handling call
0709 14:53:33.923343 (+  2272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.029670 (+106327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.699131 (+2669461us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.699376 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91831040}
I20260709 14:53:36.815580  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:36.868155  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.868944  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:36.869163  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53692us
W20260709 14:53:36.778542  7699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.579s	user 0.023s	sys 0.017s
I20260709 14:53:36.817046  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1526) took 1874 ms. Trace:
I20260709 14:53:36.811131  2607 rpcz_store.cc:276] 0709 14:53:35.704764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.704949 (+   185us) service_pool.cc:224] Handling call
0709 14:53:35.708268 (+  3319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:35.742113 (+ 33845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.742178 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:35.763901 (+ 21723us) log.cc:844] Serialized 52 byte log entry
0709 14:53:36.529962 (+766061us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:36.777147 (+247185us) log.cc:844] Serialized 14 byte log entry
0709 14:53:36.780212 (+  3065us) log.cc:844] Serialized 14 byte log entry
0709 14:53:36.810355 (+ 30143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.810440 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.810483 (+    43us) raft_consensus.cc:1685] finished
0709 14:53:36.810495 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.810643 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16453,"raft.queue_time_us":10069,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":13599,"thread_start_us":969818,"threads_started":2,"wal-append.queue_time_us":846086}
I20260709 14:53:36.820155  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.635386  7502 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.823226  3543 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:36.688058  7670 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.823778  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:36.819205  7742 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 14:53:36.809051  7669 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:36.813447  7643 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:36.731886  7499 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.808625  7764 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:36.768756  7636 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.842331  2686 rpcz_store.cc:276] 0709 14:53:30.947133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:30.947378 (+   245us) service_pool.cc:224] Handling call
0709 14:53:30.999327 (+ 51949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.466986 (+3467659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.728008 (+2261022us) spinlock_profiling.cc:243] Waited 3.42 s on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.728356 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7867540864}
I20260709 14:53:36.850479  3541 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:36.813076  7731 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 14:53:36.857019  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146030us
I20260709 14:53:36.851854  3545 rpcz_store.cc:276] 0709 14:53:34.624718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.296578 (+1671860us) service_pool.cc:224] Handling call
0709 14:53:36.300994 (+  4416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:36.437300 (+136306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.437333 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:36.438074 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 14:53:36.665899 (+227825us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:36.665959 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.666048 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.768834 (+102786us) raft_consensus.cc:1685] finished
0709 14:53:36.768858 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.769027 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93528,"raft.queue_time_us":56709,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":21104,"thread_start_us":296828,"threads_started":2,"wal-append.queue_time_us":262417}
I20260709 14:53:36.827312  7575 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:36.857714  7649 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:36.811388  2609 rpcz_store.cc:269] 0709 14:53:34.472843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.472973 (+   130us) service_pool.cc:224] Handling call
0709 14:53:36.690033 (+2217060us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400280354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:36.690148 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490731392}
I20260709 14:53:36.840786  7619 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:36.865912  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.861141  3538 rpcz_store.cc:276] 0709 14:53:34.427776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:35.551112 (+1123336us) service_pool.cc:224] Handling call
0709 14:53:35.551632 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:35.774549 (+222917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:35.774614 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:35.789504 (+ 14890us) log.cc:844] Serialized 26 byte log entry
0709 14:53:36.528777 (+739273us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:36.528823 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.528911 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.669798 (+140887us) raft_consensus.cc:1685] finished
0709 14:53:36.669821 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.669986 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59184,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":2670,"thread_start_us":937588,"threads_started":2,"wal-append.queue_time_us":828058}
I20260709 14:53:36.925140  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.816306  7689 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:36.870820  3409 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.552s (SENT)
I20260709 14:53:36.845547  7760 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:36.870664  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:36.864820  2760 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:36.928583  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1529) took 1984 ms. Trace:
I20260709 14:53:36.929636  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:36.878890  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.856312  7389 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:36.864125  7598 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:36.886268  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:36.859629  7650 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:36.833719  7585 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.890797  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:36.891160  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.953696  2612 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:36.953964  2752 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:36.827517  7306 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.966009  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1231) took 2339 ms. Trace:
I20260709 14:53:36.874085  2753 rpcz_store.cc:276] 0709 14:53:34.942724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.370761 (+1428037us) service_pool.cc:224] Handling call
0709 14:53:36.371681 (+   920us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:36.391724 (+ 20043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.391753 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:36.392286 (+   533us) log.cc:844] Serialized 52 byte log entry
0709 14:53:36.600077 (+207791us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:36.600156 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.600271 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:36.816735 (+216464us) raft_consensus.cc:1685] finished
0709 14:53:36.816768 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.816967 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17994,"raft.queue_time_us":19933,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":9246,"thread_start_us":136395,"threads_started":1,"wal-append.queue_time_us":324939}
I20260709 14:53:36.889191  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.815274  7700 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.871217  7581 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:36.889840  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:36.826452  7768 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:36.873883  7699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.679s	user 0.024s	sys 0.017s
I20260709 14:53:36.834178  7032 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.910620  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:36.914785  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:36.859149  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1453) took 3002 ms. Trace:
I20260709 14:53:36.984748  2606 rpcz_store.cc:276] 0709 14:53:33.856486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.139028 (+282542us) service_pool.cc:224] Handling call
0709 14:53:34.139430 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.893983 (+754553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.858865 (+1964882us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54003c7b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.859081 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332136704}
I20260709 14:53:36.900041  7690 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:36.952773  7650 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:36.927436  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:36.993696  2609 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:36.931392  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:36.934216  3538 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:36.929129  2759 rpcz_store.cc:276] 0709 14:53:34.943649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.374994 (+1431345us) service_pool.cc:224] Handling call
0709 14:53:36.375568 (+   574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:36.855983 (+480415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.928230 (+ 72247us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:36.928503 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912085376}
W20260709 14:53:36.949102  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78499.6us
I20260709 14:53:36.995473  2612 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:36.995721  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:36.949811  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:36.951565  2682 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:36.997578  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:36.856062  7439 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:37.000905  7439 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:37.001875  7700 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.964392  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.004231  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.004307  7689 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:37.005266  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.005584  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.006258  3542 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:37.007294  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.008802  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.009690  3542 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:37.009895  2608 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:37.010955  7784 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:37.002094  7439 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.954444  7598 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.012693  7439 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:37.012854  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.015453  7585 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:36.965579  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 912) took 1856 ms. Trace:
I20260709 14:53:37.016372  2752 rpcz_store.cc:276] 0709 14:53:35.108816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.953198 (+1844382us) service_pool.cc:224] Handling call
0709 14:53:36.953681 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.965240 (+ 11559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.965505 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.973170  2604 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:36.928509  7454 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.906416  7643 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.017885  2608 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:37.018170  7690 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:37.018743  7690 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:36.991163  7609 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:37.020725  7773 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 14:53:37.020946  7609 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.021312  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:36.975649  2683 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:36.966878  7748 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:36.976303  7581 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:37.022666  7603 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:37.023197  7581 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:36.946723  7389 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:36.970350  2760 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:37.024688  7311 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.025460  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1543) took 1335 ms. Trace:
I20260709 14:53:36.928676  7760 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:36.974624  3540 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:37.026120  7760 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:36.903457  7669 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.977824  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:36.986583  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:36.926733  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55946.6us
I20260709 14:53:37.040261  2609 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 14:53:36.990669  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1402) took 2002 ms. Trace:
I20260709 14:53:36.992524  2684 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 14:53:36.996166  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85653.2us
I20260709 14:53:37.000370  7405 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.054944  7640 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:37.056070  7649 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.055866  7640 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.058532  7643 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.061470  7672 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:36.999207  7650 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:37.063145  7502 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.063433  7690 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.054556  7636 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.065260  7768 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.068909  7669 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.069221  7596 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:36.965407  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.011803  7784 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.887284  3543 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:36.983197  7575 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:36.977860  2753 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:37.025904  2760 rpcz_store.cc:276] 0709 14:53:35.689804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.955359 (+1265555us) service_pool.cc:224] Handling call
0709 14:53:36.955923 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:37.025107 (+ 69184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.025389 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:36.978650  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.022864  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.077160  2683 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:36.966706  3547 rpcz_store.cc:276] 0709 14:53:34.626353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.299827 (+1673474us) service_pool.cc:224] Handling call
0709 14:53:36.342802 (+ 42975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.343214 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:36.519004 (+175790us) log.cc:844] Serialized 14 byte log entry
0709 14:53:36.965385 (+446381us) log.cc:844] Serialized 14 byte log entry
0709 14:53:36.965654 (+   269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.965714 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:36.965772 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.965845 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.965952 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":269515,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":3308,"raft.queue_time_us":5496,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"thread_start_us":591239,"threads_started":2}
I20260709 14:53:37.028509  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.079137  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.031345  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:36.979718  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:36.974725  7272 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.010584  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.041762  2755 rpcz_store.cc:276] 0709 14:53:34.988529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.514072 (+1525543us) service_pool.cc:224] Handling call
0709 14:53:36.514527 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.514869 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:36.519003 (+  4134us) log.cc:844] Serialized 14 byte log entry
0709 14:53:36.990237 (+471234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:36.990302 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:36.990355 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:36.990440 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:36.990600 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3623,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":28945,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":90,"thread_start_us":470298,"threads_started":1}
I20260709 14:53:37.049755  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.049865  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.083792  3411 leader_election.cc:400] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13.
I20260709 14:53:36.932343  7737 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.022713  7748 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.053426  7306 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:37.026528  7621 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.004s
I20260709 14:53:37.054513  7499 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.080579  3543 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:37.080955  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49711.6us
I20260709 14:53:37.094144  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:37.084363  2686 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:37.095909  2606 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:37.085042  2678 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:37.087728  3540 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:37.102154  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.102957  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.105849  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.106710  3547 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:37.087698  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.108989  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.119333  3542 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 14:53:37.128952  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:37.087980  7596 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:37.094717  7575 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:37.144083  2679 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:37.150991  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.157104  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:37.161111  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 464538us
I20260709 14:53:37.161656  3543 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:37.110266  7272 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:37.193866  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.195161  2609 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:37.160490  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:37.198225  7272 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:37.199092  7272 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:37.200199  7272 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:37.200639  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.201412  2611 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:37.202459  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.198333  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:37.203410  2609 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:37.205192  7748 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.201438  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 909) took 2248 ms. Trace:
I20260709 14:53:37.206029  2751 rpcz_store.cc:276] 0709 14:53:34.952697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.477305 (+1524608us) service_pool.cc:224] Handling call
0709 14:53:36.508097 (+ 30792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.518546 (+ 10449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.155262 (+636716us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400571050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.201357 (+ 46095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69717248}
I20260709 14:53:37.207015  7784 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.208758  7297 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.209764  7311 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.210263  2757 raft_consensus.cc:2368] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:37.211071  7609 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.100850  7643 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 14:53:37.228392  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.229514  2678 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:37.105245  7669 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 14:53:37.231053  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.231457  7669 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.131392  7596 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:37.232095  7669 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:37.232291  7596 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:37.232498  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.233212  7669 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.234352  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.235116  7658 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:37.235647  7658 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:37.236076  7658 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:37.236722  2680 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 14:53:37.237656  7656 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 14:53:37.238641  3545 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:37.238561  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.240095  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.240809  3545 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:37.240731  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.241861  2679 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.242291  3545 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:37.157708  7672 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 14:53:37.243394  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.238530  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.144078  7603 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.235450  2686 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:37.245048  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.127650  7737 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:37.191903  7763 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:37.247068  7763 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:37.247474  7763 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:37.231652  2682 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:37.253129  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:37.253998  2604 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.254091  7619 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.256837  2679 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:37.256953  2609 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:53:37.285887  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1430) took 2206 ms. Trace:
I20260709 14:53:37.286526  3537 rpcz_store.cc:276] 0709 14:53:35.079130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.380620 (+1301490us) service_pool.cc:224] Handling call
0709 14:53:36.380890 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.461821 (+ 80931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.285585 (+823764us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b5400003554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:37.285820 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162555264}
I20260709 14:53:37.263411  7753 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:37.268923  7308 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.244702  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.273586  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.290349  2611 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:37.274878  2683 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:37.244675  2680 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:37.276073  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 87346.7us
I20260709 14:53:37.284559  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.293298  2682 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:37.289605  2684 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:37.281525  2604 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:37.295605  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:37.296363  2611 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:37.298415  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 106755us
I20260709 14:53:37.234699  7669 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.302587  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.303207  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.218310  7654 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.304502  2609 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:37.304574  2680 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:37.306711  7779 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 14:53:37.315286  7784 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:37.316676  7784 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:37.316598  7690 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:37.317304  7784 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:37.315922  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.307456  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:37.319416  7768 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:37.320384  7768 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:37.320864  7768 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:37.321375  7768 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.293783  7753 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:37.301551  7581 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:37.323770  7753 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.325153  7753 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:37.327914  3537 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:37.328418  7619 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:37.329257  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.331128  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.331290  2678 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:37.333539  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 884) took 3459 ms. Trace:
I20260709 14:53:37.334004  2685 rpcz_store.cc:276] 0709 14:53:33.874035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.362460 (+488425us) service_pool.cc:224] Handling call
0709 14:53:34.409831 (+ 47371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:35.710022 (+1300191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.333275 (+1623253us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:37.333471 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2984962432}
W20260709 14:53:37.339761  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.340368  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:37.342772  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.347002  7764 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:37.348380  3542 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: []
I20260709 14:53:37.349223  7389 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.350723  2684 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:37.351331  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 863) took 4382 ms. Trace:
I20260709 14:53:37.352644  7297 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.351709  2603 rpcz_store.cc:276] 0709 14:53:32.969007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:33.533879 (+564872us) service_pool.cc:224] Handling call
0709 14:53:34.403378 (+869499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:34.403679 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:34.604112 (+200433us) log.cc:844] Serialized 14 byte log entry
0709 14:53:34.716586 (+112474us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:34.716653 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:34.716709 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:34.716784 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:34.937824 (+221040us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18001b75a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:37.350240 (+2412416us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18001b75a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4
0709 14:53:37.351122 (+   882us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54001ead50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.351265 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":285052,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2349,"raft.queue_time_us":153,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7885,"spinlock_wait_cycles":2028371840,"thread_start_us":312063,"threads_started":2,"wal-append.queue_time_us":244725,"wal-append.run_cpu_time_us":1344,"wal-append.run_wall_time_us":1053144}
I20260709 14:53:37.354840  7750 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.355432  7750 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:37.356608  7750 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:37.363538  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 902) took 2887 ms (client timeout 1912 ms). Trace:
W20260709 14:53:37.364156  2758 rpcz_store.cc:269] 0709 14:53:34.475599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.652315 (+176716us) service_pool.cc:224] Handling call
0709 14:53:37.363245 (+2710930us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000d0f54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.363441 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468997888}
I20260709 14:53:37.365274  7657 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.365851  7657 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:37.365743  7768 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:37.369001  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.372460  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:37.374270  2604 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:37.358471  7750 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.390484  3547 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:37.403309  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.404278  3547 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:37.389941  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 500224us
I20260709 14:53:37.408140  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.334120  7650 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:37.425882  3537 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:37.430033  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.455207  2604 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:53:37.459604  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.461006  2684 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:37.466353  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.028s	sys 0.032s
W20260709 14:53:37.466919  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.029s	sys 0.032s
W20260709 14:53:37.467134  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.469399  2609 raft_consensus.cc:1217] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Deduplicated request from leader. Original: 10.6->[12.7-15.8]   Dedup: 12.7->[15.8-15.8]
I20260709 14:53:37.470152  2609 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: [12.7-15.8]
I20260709 14:53:37.471920  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.472616  2683 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:37.472954  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.468967  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:37.472637  2680 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.473166  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.475991  3547 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.477145  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.478351  2682 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 14:53:37.480844  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.471385  2684 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:37.485936  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:37.487332  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:37.489328  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.494619  2682 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:37.318519  7784 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.389015  7658 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.464325  7389 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:37.506615  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.506577  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.523474  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:37.516813  7454 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:37.525682  7454 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:37.500577  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.484329  7779 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:37.468323  7611 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.529053  2756 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:37.428201  7760 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.524633  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:37.527733  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1395) took 3393 ms (client timeout 2209 ms). Trace:
I20260709 14:53:37.542775  2684 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:37.491731  7785 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:37.541404  2754 rpcz_store.cc:269] 0709 14:53:34.133983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:34.358403 (+224420us) service_pool.cc:224] Handling call
0709 14:53:37.527378 (+3168975us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400060254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.527630 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577500032}
I20260709 14:53:37.536397  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:53:37.597553  7795 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 14:53:37.578474  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.626791  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:37.581996  7724 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.050s	sys 0.077s
I20260709 14:53:37.546977  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:37.559661  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.529186  7454 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:37.475203  7603 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 14:53:37.542686  7489 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.645283  7489 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:37.646095  7764 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.646445  7489 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.647084  7318 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:37.647770  7489 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.649009  7308 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 14:53:37.650241  7308 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.473140  7672 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 14:53:37.650677  7308 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:37.651669  7672 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.652087  7672 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:37.652385  7650 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.645296  7731 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:37.653034  7672 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:37.568902  2677 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:37.528748  7690 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:37.651859  7308 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.545185  7695 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:37.625667  7297 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 14:53:37.656056  7695 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:37.656720  7297 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:37.657189  7297 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:37.657444  7695 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.611348  7768 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.607514  2680 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:37.661051  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:37.610049  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.023s	sys 0.062s
W20260709 14:53:37.662715  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.024s	sys 0.063s
W20260709 14:53:37.663168  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 621988us
I20260709 14:53:37.668853  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:37.669937  2684 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.673914  2611 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
W20260709 14:53:37.564248  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.622874  3547 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 14:53:37.569671  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 409250us
W20260709 14:53:37.634037  7724 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.051s	sys 0.078s
I20260709 14:53:37.613193  7613 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 14:53:37.636420  7654 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:37.638818  7760 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:37.639534  7789 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:37.625378  7793 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:37.708704  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.655295  7690 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:37.658170  7297 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.710390  7690 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:37.526873  7608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.014s	sys 0.043s
I20260709 14:53:37.560045  7785 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:37.715044  7297 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.716533  7789 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:37.717048  7789 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:37.716758  7763 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.717204  7785 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.686654  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.718024  7789 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.694778  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.701844  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.676597  7672 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.671938  7308 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.705560  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:37.753260  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47805.4us
I20260709 14:53:37.706275  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.755645  2755 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:37.707386  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.710584  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.757038  3547 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:37.757477  2751 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:37.686216  7596 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.683319  7731 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 14:53:37.545876  7779 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.687119  7711 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:37.715077  7608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.015s	sys 0.043s
I20260709 14:53:37.719712  7389 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.465440  7657 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:37.729601  7658 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.731158  7799 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:37.768795  2608 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:37.732209  7787 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.172s	user 0.003s	sys 0.000s
I20260709 14:53:37.730363  7603 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 14:53:37.686918  7792 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 14:53:37.736393  3546 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:37.784814  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 279637us
I20260709 14:53:37.736886  3541 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:37.738164  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1115) took 1040 ms. Trace:
I20260709 14:53:37.738907  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:37.785751  3540 rpcz_store.cc:276] 0709 14:53:36.697364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.772618 (+ 75254us) service_pool.cc:224] Handling call
0709 14:53:37.737960 (+965342us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400034654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.738095 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282499840}
W20260709 14:53:37.786366  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47600.6us
W20260709 14:53:37.786927  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.787281  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.742951  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.754489  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:37.789669  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46904.1us
I20260709 14:53:37.790664  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.791798  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:37.764936  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 292027us
I20260709 14:53:37.793613  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.795719  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.797688  3541 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:37.800710  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.775676  2682 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:37.781694  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:37.782759  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:37.803795  2755 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:53:37.767738  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.803668  3547 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:37.767047  7669 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:37.808352  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:37.809928  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:37.812721  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:37.813345  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.770051  7763 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.817656  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.810997  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.770423  7789 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:37.784577  7749 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:37.836861  3540 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:37.839792  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1433) took 1017 ms. Trace:
I20260709 14:53:37.843173  2752 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:37.846103  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:37.847345  3545 rpcz_store.cc:276] 0709 14:53:36.822851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.276381 (+453530us) service_pool.cc:224] Handling call
0709 14:53:37.304956 (+ 28575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.305235 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:37.305655 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 14:53:37.839301 (+533646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:37.839385 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:37.839454 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.839535 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:37.839716 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12432,"prepare.queue_time_us":1184,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":489911,"threads_started":1}
I20260709 14:53:37.851945  2755 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:53:37.859688  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:37.863085  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 75880.3us
I20260709 14:53:37.865936  7760 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:37.795224  7608 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:37.867640  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57805.4us
I20260709 14:53:37.807359  7711 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 14:53:37.876418  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 88373us
I20260709 14:53:37.880069  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:37.807482  7613 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:37.796856  7768 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:37.807406  7318 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:37.883189  7502 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:37.820559  7603 consensus_queue.cc:1243] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:53:37.844568  7272 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:37.900046  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1465) took 1046 ms. Trace:
I20260709 14:53:37.900779  2607 rpcz_store.cc:276] 0709 14:53:36.853144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.874451 (+ 21307us) service_pool.cc:224] Handling call
0709 14:53:36.877935 (+  3484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:36.890560 (+ 12625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.890616 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:36.896602 (+  5986us) log.cc:844] Serialized 26 byte log entry
0709 14:53:37.370630 (+474028us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:37.370682 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.370760 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:37.731524 (+360764us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:37.890404 (+158880us) raft_consensus.cc:1685] finished
0709 14:53:37.890526 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:37.899921 (+  9395us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":8802,"spinlock_wait_cycles":15948672,"thread_start_us":450163,"threads_started":1,"wal-append.queue_time_us":587019}
I20260709 14:53:37.878787  7643 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:53:37.817021  7654 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
W20260709 14:53:37.887599  7753 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.001s
I20260709 14:53:37.915478  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1244) took 1147 ms. Trace:
I20260709 14:53:37.915997  3542 rpcz_store.cc:276] 0709 14:53:36.768284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.120641 (+352357us) service_pool.cc:224] Handling call
0709 14:53:37.121086 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.348801 (+227715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.915159 (+566358us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54004b3cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:37.915396 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522159360}
I20260709 14:53:37.917650  3542 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:37.919736  7779 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.889153  7768 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:37.923178  7768 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:37.927445  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:37.881903  7489 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:37.819380  7650 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 14:53:37.911378  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:37.919780  7799 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:37.931061  7799 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:37.932140  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 915) took 1134 ms. Trace:
I20260709 14:53:37.931915  7799 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:37.884830  7669 raft_consensus.cc:2764] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 14:53:37.932745  2757 rpcz_store.cc:276] 0709 14:53:36.797323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.868320 (+ 70997us) service_pool.cc:224] Handling call
0709 14:53:37.931830 (+1063510us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:37.932033 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590310272}
I20260709 14:53:37.933406  7705 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:53:37.935108  7750 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.509s	user 0.001s	sys 0.011s
I20260709 14:53:37.933416  7799 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:37.937237  2686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:37.891441  7764 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:37.940572  7714 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:37.943043  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:37.943359  3542 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:37.939847  7596 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:37.930449  7598 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:37.968621  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57318.7us
I20260709 14:53:37.889984  7737 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:37.973860  3542 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:37.898798  7272 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:37.945427  7502 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 14:53:37.884186  7636 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 14:53:37.981336  2686 raft_consensus.cc:606] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:37.978245  2684 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:37.907907  7619 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 14:53:37.980728  7737 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:38.169461  7737 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:37.982592  7760 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 14:53:37.994716  7439 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:37.979208  7489 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:53:38.095971  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.012060  7700 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:38.003614  7779 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.160s
W20260709 14:53:38.162092  7802 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 14:53:38.163264  7668 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:37.977578  7764 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.164258  7272 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:38.232784  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 347075us
I20260709 14:53:38.207197  7598 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.211227  7700 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:38.234228  7793 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.011s
I20260709 14:53:38.230386  7669 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:38.243743  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 435436us
I20260709 14:53:38.206651  7750 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.235625  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:38.275898  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 897) took 1296 ms. Trace:
I20260709 14:53:38.209028  7489 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:38.276569  2677 rpcz_store.cc:276] 0709 14:53:36.979833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.980231 (+   398us) service_pool.cc:224] Handling call
0709 14:53:38.275570 (+1295339us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:38.275803 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276217088}
I20260709 14:53:38.279896  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.284790  7673 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:38.247490  7598 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:38.263424  7768 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:38.251610  7272 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:38.286629  7613 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:38.296236  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:38.343518  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 456208us
W20260709 14:53:38.341859  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 284633us
W20260709 14:53:38.275799  7724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.007s
W20260709 14:53:38.348511  7724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.011s	sys 0.012s
I20260709 14:53:38.338461  7768 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:38.269309  7793 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.009s	sys 0.011s
W20260709 14:53:38.344372  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:38.351342  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71622.6us
I20260709 14:53:38.346045  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.288734  7753 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:38.332096  7764 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:38.286947  7489 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:38.361874  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:38.277225  7669 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:38.346676  7785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.002s	sys 0.023s
I20260709 14:53:38.338590  7598 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:38.367134  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.36778e+06us
W20260709 14:53:38.362979  7308 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.002s
I20260709 14:53:38.360000  7113 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:53:38.369347  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.356189  7815 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:38.385661  2758 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:53:38.288340  7668 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 14:53:38.358485  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:38.382277  2575 connection.cc:256] RPC call timeout handler was delayed by -1.2506042860001116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 14:53:38.388231  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:38.284132  7750 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.389374  7768 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:38.341182  7700 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.367555  7753 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:38.395754  7668 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 14:53:38.385591  7789 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.000s
I20260709 14:53:38.397796  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1250) took 1370 ms. Trace:
W20260709 14:53:38.399806  7454 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.004s
W20260709 14:53:38.378271  7785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.002s	sys 0.023s
I20260709 14:53:38.425614  2604 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 14:53:38.392369  7806 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 14:53:38.404778  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:38.433570  3411 leader_election.cc:400] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:38.412673  2611 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 14:53:38.437740  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:38.386545  2575 connection.cc:256] RPC call timeout handler was delayed by -1.2341635390000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 14:53:38.384557  7318 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:38.386787  7263 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:38.425526  2753 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:38.426600  2610 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:38.421909  3543 rpcz_store.cc:276] 0709 14:53:37.027574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.575661 (+548087us) service_pool.cc:224] Handling call
0709 14:53:37.678239 (+102578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.678579 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:38.206033 (+527454us) log.cc:844] Serialized 14 byte log entry
0709 14:53:38.397141 (+191108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:38.397238 (+    97us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:38.397322 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:38.397439 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:38.397702 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6306,"prepare.queue_time_us":698976,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":53,"raft.queue_time_us":12339,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":645746,"threads_started":2}
I20260709 14:53:38.412897  7613 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.406221  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:38.440671  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.415206  7668 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:38.458635  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.428051  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:38.470885  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 69472.9us
I20260709 14:53:38.472139  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.416100  7724 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:38.459468  7596 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:38.474221  7297 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.002s
I20260709 14:53:38.478433  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:38.465466  7318 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:38.428422  7454 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.467581  7700 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:38.459682  7263 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:38.423121  7737 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.465258  7764 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.458894  7816 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:38.407707  7308 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 14:53:38.592813  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.474962  7658 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:38.487843  7768 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:38.597378  2751 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 14:53:38.498143  7672 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.003s
I20260709 14:53:38.608480  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1385) took 1137 ms. Trace:
I20260709 14:53:38.473567  7613 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.587225  7820 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:38.633351  7454 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.637301  7658 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:38.694722  7658 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:38.694962  7613 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.695525  7658 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:38.696382  7318 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.696810  7658 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:38.641381  7737 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.619326  7764 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.701980  7764 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:38.619922  7690 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:38.702941  7690 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.612442  7816 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:38.704810  7816 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:38.705296  2757 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:38.705559  7690 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:38.706773  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.649832  7750 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:38.708524  2607 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:38.616017  7760 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.708762  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.709332  7760 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.710064  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.710696  2759 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:38.711261  7672 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.711973  7672 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.712636  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 911) took 3654 ms. Trace:
I20260709 14:53:38.717797  2756 rpcz_store.cc:276] 0709 14:53:35.058396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.943790 (+1885394us) service_pool.cc:224] Handling call
0709 14:53:37.528767 (+584977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.540795 (+ 12028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:38.712422 (+1171627us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400060250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:38.712566 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282078080}
I20260709 14:53:38.607853  7297 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.608489  7668 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.692077  7308 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.591200  7596 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.704201  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1569) took 1547 ms. Trace:
W20260709 14:53:38.660751  7818 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
W20260709 14:53:38.644814  7799 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.002s
I20260709 14:53:38.741529  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:38.721465  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1254) took 1112 ms. Trace:
I20260709 14:53:38.722506  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:38.724565  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.724300  7737 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:38.724948  7613 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:38.623025  2685 rpcz_store.cc:276] 0709 14:53:37.470715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.470875 (+   160us) service_pool.cc:224] Handling call
0709 14:53:37.486765 (+ 15890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:37.499246 (+ 12481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:37.499304 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:37.507255 (+  7951us) log.cc:844] Serialized 26 byte log entry
0709 14:53:37.886606 (+379351us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:37.886675 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.886796 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:38.608119 (+721323us) raft_consensus.cc:1685] finished
0709 14:53:38.608150 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:38.608380 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12105,"raft.queue_time_us":25968,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":10360,"spinlock_wait_cycles":1231232,"thread_start_us":326422,"threads_started":1,"wal-append.queue_time_us":475836}
I20260709 14:53:38.729535  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:38.735543  7724 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:38.736224  7499 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.738932  2608 rpcz_store.cc:276] 0709 14:53:37.156581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.156711 (+   130us) service_pool.cc:224] Handling call
0709 14:53:38.703925 (+1547214us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:38.704120 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398627712}
I20260709 14:53:38.755682  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:38.738777  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.757085  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:38.746357  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.747766  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:38.759991  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.749974  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.762539  2684 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:38.622385  7768 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:38.753152  2607 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:38.754853  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:38.769699  2683 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:38.757318  2751 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:38.765897  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.586747  7753 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.775496  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.776605  2755 raft_consensus.cc:2376] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16. Re-sending same reply.
I20260709 14:53:38.778210  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.769568  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:38.781814  2755 raft_consensus.cc:2376] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18. Re-sending same reply.
W20260709 14:53:38.782924  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.783731  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.784890  2755 raft_consensus.cc:2376] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14. Re-sending same reply.
I20260709 14:53:38.786283  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:38.740665  2759 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:38.747599  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153987us
I20260709 14:53:38.792291  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.798048  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:38.802232  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.798485  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:38.809548  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.751485  7096 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:38.812611  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:38.813813  2608 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:38.815973  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.774681  7768 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:38.773577  7668 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:38.790802  7598 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.849753  7598 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.852483  7598 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.767462  7613 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.747370  7308 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.737701  7297 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:38.828346  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.829591  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:38.843255  7793 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:38.844985  7833 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:38.846886  7596 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:38.848263  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.751792  7263 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.868814  2683 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:53:38.819903  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 341592us
I20260709 14:53:38.870014  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:38.814503  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 281444us
I20260709 14:53:38.771148  7737 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.819384  7779 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:38.779532  7499 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.865286  2684 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:38.862651  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70505.1us
W20260709 14:53:38.862655  7825 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 14:53:38.874038  2759 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:38.857317  7598 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:38.875938  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:38.895120  2608 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:38.897526  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:38.899385  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:38.871075  7454 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:38.883756  7608 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.006s
I20260709 14:53:38.902808  2751 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:38.904867  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.908036  2751 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 14:53:38.907395  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:38.910691  2751 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:38.914094  2751 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:38.915692  2751 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:38.917726  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:38.918748  2683 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:38.900641  7263 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:38.919101  7096 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:38.920047  7096 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:38.920439  7096 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:38.920723  7826 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 14:53:38.921321  7096 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.922403  7096 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:38.871327  7833 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:38.935025  7834 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:38.946609  7737 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.948266  7760 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:38.905951  7832 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:38.954380  7832 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:38.955163  7832 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:38.919203  7829 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 14:53:38.964326  2607 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:53:38.890650  7672 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.889017  7779 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:38.912155  7272 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.573s	user 0.015s	sys 0.000s
I20260709 14:53:38.911286  7608 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:38.988178  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:38.987322  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.991302  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:38.993238  2684 raft_consensus.cc:2376] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10. Re-sending same reply.
W20260709 14:53:38.981485  7489 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.607s	user 0.015s	sys 0.001s
I20260709 14:53:38.930608  7640 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:38.971318  7598 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:38.969115  7613 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:38.882406  7724 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:38.967259  7834 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:38.994694  7839 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 14:53:38.911913  7793 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 14:53:38.957440  7690 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 14:53:39.010689  7839 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:39.001967  7831 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.011379  7839 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:39.011531  7690 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:39.012025  7690 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:39.011718  7831 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:39.012862  7839 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:39.001394  7658 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:39.014417  7839 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:38.982232  7841 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 14:53:38.997130  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 106876us
I20260709 14:53:39.005971  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.015800  7598 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.017097  2757 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:39.001761  7672 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:39.019153  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.019505  7672 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:39.020380  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.021116  2757 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:39.021991  7672 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:39.013029  7690 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:39.026329  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121601us
I20260709 14:53:39.025372  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:39.042285  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.009039  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1440) took 1678 ms. Trace:
I20260709 14:53:39.022732  2757 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:53:39.043228  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:39.042028  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.043087  3537 rpcz_store.cc:276] 0709 14:53:37.330672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.330779 (+   107us) service_pool.cc:224] Handling call
0709 14:53:39.008770 (+1677991us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b5400200dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:39.008962 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215833856}
I20260709 14:53:39.006592  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:39.044927  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:39.049645  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:38.976490  7760 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 14:53:39.104456  7749 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:39.105202  7749 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:39.106690  7596 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:39.107230  7596 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:39.107030  7779 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.108526  7596 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:39.109961  7596 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:39.060456  7834 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:39.111609  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.031s	user 0.015s	sys 0.032s
I20260709 14:53:39.112810  7816 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:39.113874  7834 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:39.115537  7831 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:39.090781  7263 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 14:53:39.068765  7836 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 14:53:39.119547  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1463) took 2267 ms. Trace:
I20260709 14:53:39.094200  7690 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:39.120107  2605 rpcz_store.cc:276] 0709 14:53:36.852365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.858800 (+  6435us) service_pool.cc:224] Handling call
0709 14:53:36.865638 (+  6838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:36.866088 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:36.866117 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:36.877228 (+ 11111us) log.cc:844] Serialized 52 byte log entry
0709 14:53:37.467577 (+590349us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:37.467647 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:37.467735 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:37.710103 (+242368us) raft_consensus.cc:1685] finished
0709 14:53:37.710128 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:39.119195 (+1409067us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b54002ff8d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:39.119465 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":423,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":10643,"spinlock_wait_cycles":14439808,"thread_start_us":558479,"threads_started":1,"wal-append.queue_time_us":748092,"wal-append.run_cpu_time_us":3812,"wal-append.run_wall_time_us":1067830}
W20260709 14:53:39.121728  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 214413us
I20260709 14:53:39.050625  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:38.997076  7272 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 14:53:39.069306  7838 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 14:53:39.060688  7640 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:39.061673  7573 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:39.061090  7672 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:38.995065  7799 raft_consensus.cc:2764] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 14:53:39.062979  7837 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.091709  7793 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:39.123617  2684 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:39.157478  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:39.054039  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:39.165493  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111707us
W20260709 14:53:39.128073  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.047s	user 0.015s	sys 0.032s
I20260709 14:53:39.167692  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1471) took 2300 ms. Trace:
I20260709 14:53:39.168278  2612 rpcz_store.cc:276] 0709 14:53:36.877498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.996133 (+118635us) service_pool.cc:224] Handling call
0709 14:53:36.996465 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:36.996702 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:37.080545 (+ 83843us) log.cc:844] Serialized 14 byte log entry
0709 14:53:39.167083 (+2086538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:39.167181 (+    98us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:39.167256 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:39.167388 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:39.167604 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85376,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"thread_start_us":2086021,"threads_started":1,"wal-append.queue_time_us":787525,"wal-append.run_cpu_time_us":1545,"wal-append.run_wall_time_us":1054440}
I20260709 14:53:39.039312  7489 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:39.172765  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.158658  2684 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:39.132480  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 260214us
I20260709 14:53:39.062100  7616 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 14:53:39.139914  7499 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:39.137315  7573 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:39.135546  7640 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:39.136122  7272 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:39.148401  7700 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.002s
I20260709 14:53:39.149461  7832 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:39.152127  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.195147  2677 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:39.179653  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:39.187803  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1476) took 2302 ms (client timeout 2193 ms). Trace:
I20260709 14:53:39.196436  2685 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:39.196648  2606 rpcz_store.cc:269] 0709 14:53:36.885196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:36.986149 (+100953us) service_pool.cc:224] Handling call
0709 14:53:39.187548 (+2201399us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:39.187721 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127267584}
I20260709 14:53:39.177003  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:39.188839  2603 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:39.189145  2686 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:39.190294  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.170894  7411 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:39.176792  7750 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:39.193325  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 306632us
I20260709 14:53:39.188730  7573 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:39.206925  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:39.190975  7640 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:39.208482  2752 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:53:39.209687  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.210695  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:39.176044  7837 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:39.194931  7764 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:39.191102  7816 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:39.231669  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:39.234369  7613 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:39.235199  7613 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:39.235756  7613 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:39.234302  7851 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:39.239409  7851 consensus_queue.cc:1243] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:39.240782  7793 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:39.241221  7793 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:39.243289  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.244247  2677 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:39.244171  2684 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:39.250284  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:39.228619  7764 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:39.246465  7793 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:39.252686  7851 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:39.242380  7753 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:39.255537  7841 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.323935  7753 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:39.324101  7842 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 14:53:39.258132  7272 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.321987  7828 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:39.324041  7841 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.265826  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.331288  2677 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:39.326750  7753 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:39.329846  7841 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:39.337581  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.340102  2677 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:39.341671  7764 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:39.343245  7764 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:39.347712  7489 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:39.342934  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.356993  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:39.358606  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.359437  2677 raft_consensus.cc:2376] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15. Re-sending same reply.
W20260709 14:53:39.360140  7768 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.004s
I20260709 14:53:39.360677  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:39.340358  7837 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:39.379719  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:39.384061  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:39.408648  2683 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:39.411796  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 157986us
I20260709 14:53:39.533389  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:39.561452  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 177561us
I20260709 14:53:39.503880  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1398) took 2008 ms. Trace:
W20260709 14:53:39.435546  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.563616  2682 rpcz_store.cc:276] 0709 14:53:37.495710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.502027 (+  6317us) service_pool.cc:224] Handling call
0709 14:53:37.502460 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.803275 (+300815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:39.503444 (+1700169us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54000c9ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:39.503780 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627175680}
W20260709 14:53:39.565032  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56724.7us
I20260709 14:53:39.565483  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:39.428123  7096 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.001s
I20260709 14:53:39.561985  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.566248  2682 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:39.575380  3538 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 14:53:39.568836  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 189251us
I20260709 14:53:39.362761  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.576854  2677 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:39.536512  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:39.571973  3543 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 14:53:39.560828  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.581503  3537 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:39.535825  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:39.582249  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45877.6us
I20260709 14:53:39.582989  3542 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:39.584028  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:39.588423  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:39.604658  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.404325  7737 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.370610  7690 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.409988  7828 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:39.368810  7832 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:39.573936  7850 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:39.578471  7602 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:39.570923  7608 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:39.360723  7768 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.626549  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:39.575112  7596 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:39.606428  7750 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:39.606235  7668 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:39.573211  7670 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 14:53:39.429016  7749 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.412001  7857 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.261s
W20260709 14:53:39.643446  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.677194  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:39.692634  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:39.692600  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66225.3us
I20260709 14:53:39.692656  2677 raft_consensus.cc:2376] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15. Re-sending same reply.
I20260709 14:53:39.695436  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:39.483711  7573 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:39.492295  7640 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:39.628050  7850 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:39.779726  2680 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:39.782908  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 213182us
I20260709 14:53:39.630993  7602 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:39.787010  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:39.773749  7853 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 14:53:39.700675  7834 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:39.813850  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.814751  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:39.815943  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:39.816848  2677 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:39.817552  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1412) took 2126 ms (client timeout 2142 ms). Trace:
I20260709 14:53:39.817742  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:39.818029  2758 rpcz_store.cc:269] 0709 14:53:37.690945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.691144 (+   199us) service_pool.cc:224] Handling call
0709 14:53:39.817346 (+2126202us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54000d0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:39.817493 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482767104}
I20260709 14:53:39.819444  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.820242  2758 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:39.821097  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:39.807798  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.822067  7758 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:39.822765  7758 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:39.823137  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:39.824357  7768 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:39.824438  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.822194  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.780767  7850 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:39.825995  2758 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:39.825083  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.782481  7670 consensus_queue.cc:1243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:39.843413  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.844467  2758 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:39.823899  7758 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.812408  2755 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:39.857417  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.858525  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.858772  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:39.818398  2677 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:39.859540  3538 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:39.860535  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.703788  7596 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:39.866879  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:39.619933  7690 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:39.846230  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 265629us
I20260709 14:53:39.887696  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:39.865280  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.845728  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:39.701409  7668 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:39.861502  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:39.906922  2607 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:39.907047  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:39.770964  7672 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 14:53:39.788543  7608 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:39.620108  7737 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.663507  7856 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.230s
W20260709 14:53:39.844623  7613 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.004s
W20260709 14:53:39.894321  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1419) took 2116 ms (client timeout 2511 ms). Trace:
W20260709 14:53:39.912882  7793 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.001s
I20260709 14:53:39.903757  7297 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:39.888509  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:39.807471  7272 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:39.909569  2677 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:39.905362  7668 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:39.803719  7602 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:39.916340  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:39.921829  2610 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:39.924760  2612 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:39.926934  2682 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 14:53:39.927801  2753 rpcz_store.cc:269] 0709 14:53:37.777961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.812362 (+ 34401us) service_pool.cc:224] Handling call
0709 14:53:39.893974 (+2081612us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b5400120054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:39.894224 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406419712}
I20260709 14:53:39.900700  7669 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:39.672467  7828 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:39.932883  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44475.6us
I20260709 14:53:39.933334  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:39.925065  7758 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:39.863205  7815 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 14:53:39.962427  7827 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:40.207304  3542 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:39.935702  7272 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:39.924398  7608 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:39.931861  7841 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 14:53:39.998405  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:39.931694  7613 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:39.931569  7793 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:40.099191  2726 connection.cc:256] RPC call timeout handler was delayed by -1.0974226220000673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
I20260709 14:53:39.923319  7672 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:39.915660  7863 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.187852  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1374) took 1061 ms. Trace:
I20260709 14:53:40.193359  3537 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:40.194988  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 329679us
I20260709 14:53:40.202157  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1432) took 1437 ms. Trace:
I20260709 14:53:40.210601  3538 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:40.214248  2682 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:40.236552  2758 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:40.237706  2677 rpcz_store.cc:276] 0709 14:53:39.126552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.886974 (+760422us) service_pool.cc:224] Handling call
0709 14:53:39.887507 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:39.935858 (+ 48351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.187753 (+251895us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:40.238595  2759 rpcz_store.cc:276] 0709 14:53:38.764911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.829117 (+ 64206us) service_pool.cc:224] Handling call
0709 14:53:40.201902 (+1372785us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5400236ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.202059 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29104896}
I20260709 14:53:40.239804  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.241209  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.232896  7672 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:40.244105  7672 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.245759  7672 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.245831  7850 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.247479  7749 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.248350  7602 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.248775  7318 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.249356  7318 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.217828  7827 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:40.222218  7272 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:40.252408  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1392) took 2773 ms. Trace:
I20260709 14:53:40.252761  2756 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:40.256958  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1444) took 2923 ms. Trace:
I20260709 14:53:40.276399  3540 rpcz_store.cc:276] 0709 14:53:37.333844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.786670 (+452826us) service_pool.cc:224] Handling call
0709 14:53:37.787109 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:37.847369 (+ 60260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.256623 (+2409254us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54002ba7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:40.256881 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113479040}
I20260709 14:53:40.257992  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 897) took 2889 ms. Trace:
I20260709 14:53:40.279057  2603 rpcz_store.cc:276] 0709 14:53:37.369014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.369165 (+   151us) service_pool.cc:224] Handling call
0709 14:53:39.188614 (+1819449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:39.200616 (+ 12002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.257773 (+1057157us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54001d8b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.257922 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4051410816}
I20260709 14:53:40.271570  2679 rpcz_store.cc:276] 0709 14:53:37.479094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.479226 (+   132us) service_pool.cc:224] Handling call
0709 14:53:37.503933 (+ 24707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.719473 (+215540us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:37.719819 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 14:53:37.974915 (+255096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:37.974982 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:37.975052 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.252130 (+2277078us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:40.252192 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:40.252343 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8160,"prepare.queue_time_us":9067,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":142,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":6172,"spinlock_wait_cycles":494997248,"thread_start_us":254647,"threads_started":1,"wal-append.queue_time_us":621836,"wal-append.run_cpu_time_us":1691,"wal-append.run_wall_time_us":1054600}
I20260709 14:53:40.267634  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:40.290483  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.265348  7439 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:40.264395  3537 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:40.300803  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1372) took 1549 ms. Trace:
I20260709 14:53:40.293406  7596 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.262090  7318 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.260329  7608 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.261385  7640 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.265075  7850 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.251343  7573 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.253716  7793 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.299288  7690 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:40.301497  2680 rpcz_store.cc:276] 0709 14:53:38.751125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.756760 (+  5635us) service_pool.cc:224] Handling call
0709 14:53:40.300482 (+1543722us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400445a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.300696 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349184384}
I20260709 14:53:40.300796  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1481) took 1897 ms. Trace:
I20260709 14:53:40.256232  7830 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.260325  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.250818  7602 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.304808  7439 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:40.310470  2679 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 14:53:40.337359  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1407) took 1097 ms. Trace:
I20260709 14:53:40.337862  2679 rpcz_store.cc:276] 0709 14:53:39.239493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.292923 (+1053430us) service_pool.cc:224] Handling call
0709 14:53:40.310183 (+ 17260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:40.336891 (+ 26708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.337268 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:40.298473  7750 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.259486  7737 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.260645  7827 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.252184  7875 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.359520  7875 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.318465  2611 rpcz_store.cc:276] 0709 14:53:38.403869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.435508 (+ 31639us) service_pool.cc:224] Handling call
0709 14:53:38.436065 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:38.436484 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:53:38.465841 (+ 29357us) log.cc:844] Serialized 14 byte log entry
0709 14:53:39.815160 (+1349319us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b180045b0e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:53:40.300120 (+484960us) log.cc:844] Serialized 14 byte log entry
0709 14:53:40.300363 (+   243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:40.300428 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:53:40.300494 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.300591 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:40.300727 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29106,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":4273,"raft.queue_time_us":2305,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":9285,"spinlock_wait_cycles":10828032,"thread_start_us":388187,"threads_started":2,"wal-append.queue_time_us":1771000}
I20260709 14:53:40.319387  2680 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:40.363729  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1411) took 1123 ms. Trace:
I20260709 14:53:40.364138  2680 rpcz_store.cc:276] 0709 14:53:39.240368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.318751 (+1078383us) service_pool.cc:224] Handling call
0709 14:53:40.319159 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:40.363415 (+ 44256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.363663 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:40.365463  7724 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 14:53:40.366040  7724 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:40.325441  7867 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.366776  7867 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:40.307426  7866 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 14:53:40.323699  7272 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.314167  7608 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.326905  3547 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:40.325169  2753 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:40.328116  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:40.376035  2759 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:40.296363  3542 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:40.282991  7668 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.335855  7830 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:40.382661  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.383425  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:40.384217  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.354007  2679 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:40.385288  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1412) took 1144 ms. Trace:
I20260709 14:53:40.385711  2679 rpcz_store.cc:276] 0709 14:53:39.240526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.339550 (+1099024us) service_pool.cc:224] Handling call
0709 14:53:40.353590 (+ 14040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:40.384849 (+ 31259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.385211 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:40.374351  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1403) took 1773 ms. Trace:
W20260709 14:53:40.386780  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.379854  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.345995  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.389362  3537 raft_consensus.cc:2376] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16. Re-sending same reply.
I20260709 14:53:40.387203  2678 rpcz_store.cc:276] 0709 14:53:38.601429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.601609 (+   180us) service_pool.cc:224] Handling call
0709 14:53:38.615912 (+ 14303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:38.802753 (+186841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:38.802792 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:38.811868 (+  9076us) log.cc:844] Serialized 26 byte log entry
0709 14:53:38.930976 (+119108us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:39.071283 (+140307us) log.cc:844] Serialized 14 byte log entry
0709 14:53:39.080439 (+  9156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:39.080544 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:40.297811 (+1217267us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b180031b160. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1
0709 14:53:40.308386 (+ 10575us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b180031b160. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:40.374049 (+ 65663us) raft_consensus.cc:1685] finished
0709 14:53:40.374098 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:40.374277 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5585,"prepare.queue_time_us":190104,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1893,"raft.queue_time_us":93562,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":3928,"spinlock_wait_cycles":16576640,"thread_start_us":364977,"threads_started":3,"wal-append.queue_time_us":225630}
W20260709 14:53:40.387677  2652 connection.cc:256] RPC call timeout handler was delayed by -1.7056869240000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 14:53:40.393002  3543 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:40.373775  7872 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:53:40.398938  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:40.413206  2652 connection.cc:256] RPC call timeout handler was delayed by -1.3759436550000146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 14:53:40.413518  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.403822  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:40.403486  3538 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:40.373411  7272 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.415261  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:40.416980  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.417172  3536 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:53:40.417444  2753 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:40.417768  3547 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:40.418802  2652 connection.cc:256] RPC call timeout handler was delayed by -1.1111280910000687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 14:53:40.419791  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.420233  2753 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 14:53:40.419806  2652 connection.cc:256] RPC call timeout handler was delayed by -1.0447364509999488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25110000000000005s for a response
I20260709 14:53:40.421492  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:40.392479  7871 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.421140  3538 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.424109  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.425100  3537 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 9: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.421240  2759 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:40.427175  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.426460  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:40.427847  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.272897  7863 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.423924  7870 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:40.374399  7750 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.431602  3542 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.433233  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.429484  2756 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:40.391912  7608 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.440662  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.443207  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.444494  3538 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:40.445366  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.445668  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.446318  3547 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:40.431984  7871 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.447276  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.448437  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.449946  3547 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:40.456815  7833 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.457297  7830 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.458694  7581 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.460615  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1440) took 1335 ms. Trace:
I20260709 14:53:40.462121  7750 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.461340  2757 rpcz_store.cc:276] 0709 14:53:39.125376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.125619 (+   243us) service_pool.cc:224] Handling call
0709 14:53:39.130124 (+  4505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:39.130629 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:39.130677 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:39.131284 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 14:53:40.254121 (+1122837us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b1800374f20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:53:40.391475 (+137354us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:40.394162 (+  2687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.394353 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:40.460099 (+ 65746us) raft_consensus.cc:1685] finished
0709 14:53:40.460141 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:40.460504 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2078,"spinlock_wait_cycles":4664192,"thread_start_us":137007,"threads_started":1,"wal-append.queue_time_us":1326491}
I20260709 14:53:40.464898  3547 pending_rounds.cc:85] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 7
I20260709 14:53:40.427758  7880 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:40.466544  3537 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:40.439653  7875 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.468204  7863 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.473280  3547 pending_rounds.cc:107] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 15.8
W20260709 14:53:40.473889  2726 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.193s (SENT)
I20260709 14:53:40.476167  3547 raft_consensus.cc:2889] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:40.476402  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.447216  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.477893  3538 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.478089  3542 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:40.443104  7608 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.449724  7871 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.481980  7880 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:40.501545  7668 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:40.512388  7880 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.544144  7827 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.639237  7758 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.640430  7758 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.422456  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:40.642978  7831 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.644105  7831 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:40.644390  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 221879us
W20260709 14:53:40.594007  2575 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.151s (SENT) [suppressed 44 similar messages]
I20260709 14:53:40.648943  7690 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:40.649371  7827 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.604719  2604 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 14:53:40.606241  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:40.607851  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:40.609385  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:40.611720  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:40.613713  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 139691us
I20260709 14:53:40.616168  2753 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 14:53:40.620517  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:40.621608  2608 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 14:53:40.629065  7611 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:40.630311  7598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.497s	user 0.007s	sys 0.010s
I20260709 14:53:40.599915  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1379) took 1466 ms. Trace:
I20260709 14:53:40.651893  7880 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:40.652918  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:40.653270  7608 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:40.650311  7690 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:40.691835  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.653299  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1383) took 1248 ms. Trace:
I20260709 14:53:40.668213  2605 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:40.665313  2608 pending_rounds.cc:85] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:53:40.663666  2604 pending_rounds.cc:85] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:53:40.664903  2753 pending_rounds.cc:85] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 10
I20260709 14:53:40.666565  2677 rpcz_store.cc:276] 0709 14:53:39.132874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.270973 (+1138099us) service_pool.cc:224] Handling call
0709 14:53:40.271404 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:40.271833 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:40.271867 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:40.282784 (+ 10917us) log.cc:844] Serialized 52 byte log entry
0709 14:53:40.598221 (+315437us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:40.598378 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.598609 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:40.598736 (+   127us) raft_consensus.cc:1685] finished
0709 14:53:40.598769 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:40.599549 (+   780us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2153,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":15190,"thread_start_us":279078,"threads_started":1,"wal-append.queue_time_us":196781}
I20260709 14:53:40.680392  7758 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:40.685081  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:40.698870  2753 pending_rounds.cc:107] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 14:53:40.656169  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:40.693186  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:40.692504  7598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.565s	user 0.007s	sys 0.010s
I20260709 14:53:40.698091  2604 pending_rounds.cc:107] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 14:53:40.697664  2608 pending_rounds.cc:107] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 14:53:40.689517  7871 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:53:40.702253  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108524us
I20260709 14:53:40.697248  2682 rpcz_store.cc:276] 0709 14:53:39.404577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.588043 (+183466us) service_pool.cc:224] Handling call
0709 14:53:40.652640 (+1064597us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400268c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.652943 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776477568}
I20260709 14:53:40.694983  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1552) took 1993 ms. Trace:
I20260709 14:53:40.700174  2753 raft_consensus.cc:2889] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:40.702059  2604 raft_consensus.cc:2889] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:40.703409  2752 rpcz_store.cc:276] 0709 14:53:38.701845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.707870 (+  6025us) service_pool.cc:224] Handling call
0709 14:53:38.708280 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:39.211849 (+503569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.694241 (+1482392us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:40.694699 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149645568}
I20260709 14:53:40.702565  2608 raft_consensus.cc:2889] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:40.707530  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.709231  2603 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:40.709426  7871 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:40.711493  7871 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.715452  7705 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:40.738394  7705 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:40.738500  7695 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:40.739123  7695 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:40.724964  7831 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.715675  7758 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 14:53:40.741034  7608 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.741170  7695 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:40.727062  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1443) took 3393 ms (client timeout 1560 ms). Trace:
W20260709 14:53:40.730120  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44540.1us
I20260709 14:53:40.727537  3537 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 14:53:40.730067  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45479.2us
I20260709 14:53:40.731238  7668 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:40.731663  7308 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:40.733933  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 910) took 1928 ms (client timeout 2138 ms). Trace:
I20260709 14:53:40.735337  7613 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:40.747197  7699 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:53:40.747567  2650 connection.cc:256] RPC call timeout handler was delayed by -1.3096476010000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
I20260709 14:53:40.765542  7699 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:40.765777  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:40.766378  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:40.766682  2650 connection.cc:256] RPC call timeout handler was delayed by -1.1955785630000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 14:53:40.767156  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76962.3us
I20260709 14:53:40.768491  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:40.769482  7640 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:40.743938  3546 rpcz_store.cc:269] 0709 14:53:37.333067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.340020 (+  6953us) service_pool.cc:224] Handling call
0709 14:53:40.726364 (+3386344us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.726659 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908991872}
I20260709 14:53:40.748944  7113 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:40.750280  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.751811  7668 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:40.752269  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.752713  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.704916  7611 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.740264  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:40.760396  3538 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:40.763509  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:40.774984  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.776335  3546 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.778534  2607 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:40.770541  7640 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.783367  7668 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.785306  7640 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:40.785852  3411 connection.cc:256] RPC call timeout handler was delayed by -2.0371223989999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 14:53:40.739143  7705 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:40.788233  7611 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.789336  3536 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:40.790920  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:40.792030  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1368) took 2053 ms. Trace:
I20260709 14:53:40.792598  2686 rpcz_store.cc:276] 0709 14:53:38.738185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.739099 (+   914us) service_pool.cc:224] Handling call
0709 14:53:38.754509 (+ 15410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:39.199691 (+445182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:40.791341 (+1591650us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540024efd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:40.791805 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999080320}
W20260709 14:53:40.755769  2685 rpcz_store.cc:269] 0709 14:53:38.805509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:38.806424 (+   915us) service_pool.cc:224] Handling call
0709 14:53:40.733380 (+1926956us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540012c854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.733638 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872293760}
I20260709 14:53:40.796942  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:40.798380  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.804185  3538 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 14:53:40.792109  3536 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:40.806017  3541 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:40.809768  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:40.807058  3411 connection.cc:256] RPC call timeout handler was delayed by -1.6717223989999184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 14:53:40.810354  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0569785630000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
W20260709 14:53:40.810351  3411 connection.cc:256] RPC call timeout handler was delayed by -1.6492223989999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
W20260709 14:53:40.810734  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0540616600001158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260709 14:53:40.814006  2606 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:40.816356  7695 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:40.817607  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77718.7us
I20260709 14:53:40.819141  7785 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:40.820546  7619 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:40.820375  7308 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:40.821892  7863 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.822757  7863 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:40.823523  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:40.823884  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.825204  7832 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:40.826136  7598 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:40.827085  3411 connection.cc:256] RPC call timeout handler was delayed by -1.5502223989999493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22730000000000003s for a response
I20260709 14:53:40.828037  7831 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.828354  7784 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.828969  7831 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.832319  7834 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.832789  7581 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.833388  7834 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.833618  7581 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:40.834440  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 918) took 1830 ms (client timeout 2151 ms). Trace:
W20260709 14:53:40.835024  2683 rpcz_store.cc:269] 0709 14:53:39.003993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.005625 (+  1632us) service_pool.cc:224] Handling call
0709 14:53:40.833977 (+1828352us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.834220 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901209984}
W20260709 14:53:40.832176  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68967.8us
W20260709 14:53:40.835534  3411 connection.cc:256] RPC call timeout handler was delayed by -1.2328225590000557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 14:53:40.837045  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:40.837271  7619 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.838235  7619 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.839217  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:40.840770  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.841305  7619 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:40.843516  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:40.844111  7454 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.844977  7454 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.842489  7831 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.848030  7863 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.848433  7834 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:40.835827  7598 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:38.744832  3545 rpcz_store.cc:276] 0709 14:53:37.609628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.979632 (+370004us) service_pool.cc:224] Handling call
0709 14:53:37.982753 (+  3121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:37.983112 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:38.430068 (+446956us) log.cc:844] Serialized 14 byte log entry
0709 14:53:38.720878 (+290810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:38.720969 (+    91us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:38.721047 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:38.721162 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:38.721375 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":415444,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":5591,"raft.queue_time_us":136,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6559,"thread_start_us":729951,"threads_started":2}
I20260709 14:53:40.849689  7690 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:40.851452  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:40.852054  7598 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.853302  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:40.859442  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:40.860292  7032 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:40.860528  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:40.860867  3541 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:40.861572  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:40.861577  2604 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:40.862624  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:40.866343  2610 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:40.855489  2607 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:40.857390  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:40.858084  7657 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:40.854194  7863 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:40.872839  2753 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:40.872817  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:40.875010  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:40.875941  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:40.876533  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63934.1us
I20260709 14:53:40.876996  2755 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:40.808765  7827 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 14:53:40.880472  2683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:40.881829  7827 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:40.881434  7699 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:40.868422  3540 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:40.882354  7827 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:40.895161  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:40.888320  2757 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 14:53:40.903273  7750 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.906299  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.906481  7750 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.908684  7799 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:40.889467  7835 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.910607  7724 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:40.903340  7454 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:40.911230  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:40.912211  7724 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:40.913188  7799 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:40.913748  7799 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:40.914462  7799 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.917889  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:40.909835  7608 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.918131  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.919140  2753 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:40.920277  7832 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:40.920815  7832 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:40.923091  7598 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 8 FOLLOWER]: Advancing to term 9
I20260709 14:53:40.910873  7835 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.927428  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1485) took 1081 ms. Trace:
I20260709 14:53:40.928153  2612 rpcz_store.cc:276] 0709 14:53:39.846103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.846282 (+   179us) service_pool.cc:224] Handling call
0709 14:53:40.926816 (+1080534us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.927129 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38895488}
I20260709 14:53:40.929098  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:40.930156  2686 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.904835  2757 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:40.931051  2757 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 14:53:40.931370  2757 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:40.934754  7619 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:40.936635  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:40.937108  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:40.938002  2682 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:40.938607  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.930300  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:40.941710  7602 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:40.942435  7602 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:40.943157  7863 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:40.943665  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.943708  7863 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:40.944921  7602 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.945870  7670 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:40.945966  7863 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:40.948570  7867 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.889802  7785 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:40.891074  7640 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:40.876868  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:40.964178  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46172.5us
I20260709 14:53:40.947039  7619 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:40.890591  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:40.888069  7581 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:40.909930  7699 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:40.968321  2685 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:40.989643  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:40.993461  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 75769.4us
I20260709 14:53:40.977310  7619 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:41.001571  7439 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.017s
I20260709 14:53:40.991266  7640 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:40.998545  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:40.997185  2682 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:41.000006  7672 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:41.002175  7581 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:53:40.997017  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1378) took 1845 ms (client timeout 2084 ms). Trace:
W20260709 14:53:41.019712  2684 rpcz_store.cc:269] 0709 14:53:39.132156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:39.209280 (+ 77124us) service_pool.cc:224] Handling call
0709 14:53:40.967718 (+1758438us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b5400269b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:40.977123 (+  9405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71938304}
W20260709 14:53:41.031457  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 154759us
I20260709 14:53:41.034781  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:41.036453  7831 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:41.039002  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.039469  7032 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:41.036490  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:41.016804  7672 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:41.057427  2612 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:53:41.039529  7272 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:53:41.062309  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 231196us
W20260709 14:53:41.063140  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:41.063557  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.066785  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:41.067469  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 177085us
I20260709 14:53:41.067970  3547 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:41.069386  7636 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:41.069748  7454 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:41.071286  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.071605  7454 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:41.072075  7454 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:41.072381  3537 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:41.073321  7454 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:41.069274  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77816us
W20260709 14:53:41.075314  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87789.7us
I20260709 14:53:41.077996  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.075206  7871 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:41.082067  7871 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:41.075043  7640 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.069763  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.048722  7834 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:41.088171  2678 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:41.088791  2678 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:41.089308  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:41.089452  7871 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:41.091153  7833 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:41.091713  7833 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:41.092124  7833 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:41.093119  7834 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:41.093508  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:41.094503  2680 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:41.094569  7834 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:41.060962  7672 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:41.096276  7834 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:41.090231  2678 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.088361  7857 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:41.086632  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.014649  7785 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:41.088234  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 148743us
I20260709 14:53:41.088563  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.078578  7454 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:41.024674  7439 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.120322  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:41.126168  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 132008us
I20260709 14:53:41.127735  2686 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:41.084328  7835 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.136974  2677 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 14:53:41.120711  2603 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 10.
I20260709 14:53:41.140529  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.141613  2678 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:41.146370  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.150025  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.123373  7608 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 14:53:41.124053  7613 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:41.160475  7613 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.164089  7717 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 14:53:41.165537  7272 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:41.165958  7717 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:41.170459  7785 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.172004  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.174898  7867 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:41.175683  7867 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.177675  7657 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:41.179047  7840 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:41.179888  7867 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.179683  7840 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:41.180749  7876 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.013s	sys 0.016s
W20260709 14:53:41.181224  7876 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.013s	sys 0.016s
I20260709 14:53:41.187637  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.188974  3542 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:41.189702  3546 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 14:53:41.191689  7863 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.192777  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:41.194679  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:41.204156  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.164714  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.146445  7750 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.194532  7863 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.146514  7640 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:41.215092  2679 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:41.174280  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:41.152202  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.220803  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.248977  2680 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
W20260709 14:53:41.227093  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 157447us
I20260709 14:53:41.199590  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.256978  2678 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:41.234176  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.235400  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.236351  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.232177  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.236835  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:41.225548  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60986.5us
I20260709 14:53:41.178382  7657 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.276968  7657 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:41.278452  2605 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:41.280221  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.281800  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.283069  2612 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:41.267392  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.284854  2680 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:41.285385  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.285926  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.273658  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.287051  2758 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:41.273145  2433 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 8 to 9, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:41.288390  7834 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 14:53:41.274753  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:41.291772  2760 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:41.153705  7827 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.293656  7827 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:41.292902  2680 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:41.295109  2607 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:41.295339  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.296977  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.297989  2755 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:41.239058  7613 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.235903  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.271580  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.223104  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.300786  2611 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:41.284480  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:41.286353  2677 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:41.307194  2754 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 14:53:41.307574  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 88287.5us
I20260709 14:53:41.302115  2609 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:41.309267  3536 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:41.312748  3537 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:41.291833  7439 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.312666  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.314337  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:41.226871  7863 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:41.319936  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.320823  2682 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:41.166685  7717 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:41.326952  7831 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:41.231748  7640 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:41.351514  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:41.348192  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.354256  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.367635  2609 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:53:41.374449  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69364us
I20260709 14:53:41.376588  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.353761  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:41.348021  7668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.019s	sys 0.029s
I20260709 14:53:41.378464  2679 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:41.378898  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57409.9us
I20260709 14:53:41.370093  2608 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:41.371575  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.385241  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:41.216430  7799 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.188916  7832 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.387586  2756 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:41.380328  2603 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 14:53:41.396471  2603 pending_rounds.cc:85] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 7
I20260709 14:53:41.396855  2603 pending_rounds.cc:107] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 14:53:41.397192  2603 raft_consensus.cc:2889] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:41.312948  7857 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:41.393440  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.400097  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.285406  7705 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.421005  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:41.406972  2678 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:41.409389  2604 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:41.393131  7657 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:41.410984  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 492500us
W20260709 14:53:41.411244  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57635.6us
I20260709 14:53:41.411672  7272 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:41.410678  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:41.465803  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 175312us
I20260709 14:53:41.348081  7863 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.472548  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.479875  2608 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:41.484596  7272 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:41.486479  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.487588  7695 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:41.488309  7695 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.488970  7863 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:41.490453  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:41.427645  7657 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.491359  3536 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:41.487016  7828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.059s	sys 0.120s
W20260709 14:53:41.492489  7828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.060s	sys 0.120s
I20260709 14:53:41.493181  7272 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:41.494964  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.495173  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.496008  2680 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:41.496706  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.497352  3537 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:41.498313  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.499846  2610 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:41.489120  7668 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:41.507396  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113500us
I20260709 14:53:41.428499  7640 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.362424  7834 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:41.408305  7840 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:41.533947  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 111529us
I20260709 14:53:41.534929  2612 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:41.510924  7705 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.511534  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.496873  7695 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.511832  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:41.543138  2678 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:41.512439  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.397778  7619 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:41.513033  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.489138  2679 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 14:53:41.516909  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.522634  2680 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:41.388615  2433 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 10 to 13, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:41.530155  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:41.551107  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.554207  2611 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:41.557366  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:41.515661  7670 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.011s
I20260709 14:53:41.558818  2760 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:41.541312  2682 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:41.560698  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.563346  2677 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:41.545207  7690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.000s	sys 0.030s
I20260709 14:53:41.478541  7833 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.561476  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.565848  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:41.572803  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:41.565302  7581 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.003s	sys 0.032s
W20260709 14:53:41.576424  7581 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.032s
I20260709 14:53:41.576961  7581 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:41.539495  7834 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:41.577965  7834 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:41.578552  7581 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.579563  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:41.581480  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:41.583007  2751 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:41.584321  7799 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.584692  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 926) took 1795 ms. Trace:
I20260709 14:53:41.585106  2751 rpcz_store.cc:276] 0709 14:53:39.789532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.187403 (+397871us) service_pool.cc:224] Handling call
0709 14:53:40.187809 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:41.583603 (+1395794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:41.584353 (+   750us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001ff9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:41.584615 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3052928384}
I20260709 14:53:41.538987  7668 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:41.587908  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.588908  7668 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:41.588991  2610 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:53:41.590487  2605 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:41.594692  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.004s	sys 0.036s
I20260709 14:53:41.595166  2757 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 14:53:41.595209  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.004s	sys 0.036s
I20260709 14:53:41.506031  7717 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:41.572546  7690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.000s	sys 0.031s
I20260709 14:53:41.605568  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:41.599637  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 565272us
I20260709 14:53:41.585155  7640 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.593822  7670 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:41.651407  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.590526  7863 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:41.612238  2677 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:41.679838  2679 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:41.715476  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:41.710644  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117786us
W20260709 14:53:41.719185  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 166670us
W20260709 14:53:41.692097  3409 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.151s (SENT)
I20260709 14:53:41.735872  2607 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:53:41.708194  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 142556us
I20260709 14:53:41.604372  7636 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:53:41.754947  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.005s	sys 0.025s
I20260709 14:53:41.787101  2603 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:41.803073  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.005s	sys 0.029s
W20260709 14:53:41.755165  3540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.007s	sys 0.023s
W20260709 14:53:41.805531  3540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.009s	sys 0.025s
I20260709 14:53:41.806353  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:41.646538  7602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.006s	sys 0.010s
W20260709 14:53:41.679215  7724 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.766s	user 0.018s	sys 0.001s
I20260709 14:53:41.678156  7717 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:41.811414  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:41.811862  7899 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 14:53:41.691380  7454 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.004s
I20260709 14:53:41.831223  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:41.813235  7602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.007s	sys 0.010s
W20260709 14:53:41.719025  7784 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.890s	user 0.018s	sys 0.003s
W20260709 14:53:41.760166  7672 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.664s	user 0.015s	sys 0.000s
W20260709 14:53:41.764125  7598 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.840s	user 0.020s	sys 0.000s
I20260709 14:53:41.814137  7636 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:41.717216  7699 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.728s	user 0.015s	sys 0.000s
I20260709 14:53:41.610915  7695 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:41.805447  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.821382  7705 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:41.723470  7904 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:41.871764  7904 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.872816  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.873803  2604 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:41.874325  7904 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:41.825073  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:41.876045  2610 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:41.860653  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 379102us
I20260709 14:53:41.826004  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:41.870679  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.753996  7832 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:41.701857  7876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.030s
I20260709 14:53:41.834771  2752 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:41.837841  7657 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:41.838347  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 146337us
W20260709 14:53:41.839031  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 802666us
I20260709 14:53:41.840031  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:41.842684  2677 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 14:53:41.836720  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:41.715724  7799 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 14:53:41.815796  7619 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:41.898137  3537 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 14:53:41.859381  7871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.033s	sys 0.018s
I20260709 14:53:41.856967  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.900736  2608 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:41.819401  7833 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:41.859189  2686 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:41.902354  2683 raft_consensus.cc:606] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:41.902989  2683 raft_consensus.cc:614] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:41.898809  3537 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:41.859776  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:41.739993  7690 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 14:53:41.825351  7905 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:41.826498  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:41.891496  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:41.891629  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:41.851599  3536 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 14:53:41.907075  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 191777us
I20260709 14:53:41.856089  7636 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:41.845429  7863 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:41.908281  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:41.915740  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:41.858704  7598 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 14:53:41.893996  7876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.019s	sys 0.035s
I20260709 14:53:41.880858  7695 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:41.907907  7871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.034s	sys 0.018s
I20260709 14:53:41.894127  7581 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 14:53:41.954201  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 145507us
I20260709 14:53:41.960062  2605 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:53:41.960235  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48921.7us
I20260709 14:53:41.966606  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:41.917501  7905 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:41.949329  3537 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:41.857839  7672 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:41.969825  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.020s	sys 0.021s
W20260709 14:53:42.003468  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.020s	sys 0.021s
I20260709 14:53:41.999890  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:41.998775  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78221.7us
I20260709 14:53:41.952479  7695 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:41.993655  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 153792us
I20260709 14:53:42.005354  2677 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:42.012307  7828 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:42.012459  7820 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:42.012862  7828 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.013274  7828 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.013207  7820 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.015398  2681 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 14:53:42.019351  2678 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:42.011023  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 129770us
I20260709 14:53:41.990428  7699 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:41.993098  7032 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:41.998006  7619 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:41.997609  7909 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:42.004479  7905 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.057246  2606 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:42.067282  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.024185  7832 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:42.077802  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:42.077899  2433 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
W20260709 14:53:42.064471  7901 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 14:53:42.090426  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.091549  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.086073  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44984 (request call id 18) took 1197 ms. Trace:
I20260709 14:53:42.077778  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:42.087010  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:42.086162  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.089102  2757 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:53:42.088488  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.093743  2683 rpcz_store.cc:276] 0709 14:53:40.880045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.880242 (+   197us) service_pool.cc:224] Handling call
0709 14:53:42.077124 (+1196882us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400189554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:42.077266 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347211520}
I20260709 14:53:42.099546  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:42.100152  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:42.101235  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 269760us
I20260709 14:53:42.099437  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.100595  2686 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:42.059170  7699 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:42.076805  7454 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 14:53:42.108407  7699 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.111268  7912 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 14:53:42.031757  7784 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 14:53:42.033407  7032 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.116171  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:42.116520  7032 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.117483  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.118259  2752 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:42.063304  7909 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:42.119841  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1594) took 1049 ms. Trace:
I20260709 14:53:42.058715  7619 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.121762  2607 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:42.119858  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.102066  2754 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
W20260709 14:53:42.095777  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 216863us
W20260709 14:53:42.126174  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 182391us
I20260709 14:53:42.128389  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:42.151260  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47177.4us
I20260709 14:53:42.129942  3537 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:42.129408  2606 rpcz_store.cc:276] 0709 14:53:41.076190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.082645 (+  6455us) service_pool.cc:224] Handling call
0709 14:53:42.119604 (+1036959us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54002feed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:42.119777 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182379264}
I20260709 14:53:42.127465  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.163864  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:42.082176  7832 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:42.166577  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 90272.4us
I20260709 14:53:42.035008  7904 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 14:53:42.177121  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:42.164143  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:42.163269  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1365) took 4621 ms (client timeout 1508 ms). Trace:
I20260709 14:53:42.147305  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:42.196135  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 136137us
I20260709 14:53:42.195228  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.194749  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:42.182487  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:42.202061  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44702.7us
I20260709 14:53:42.156770  7705 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:42.181087  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:42.211391  2681 rpcz_store.cc:269] 0709 14:53:37.526357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:37.526730 (+   373us) service_pool.cc:224] Handling call
0709 14:53:42.125493 (+4598763us) spinlock_profiling.cc:243] Waited 4.38 s on lock 0x7b54002b3754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:42.147497 (+ 22004us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10073211136}
I20260709 14:53:42.214789  2759 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16.
I20260709 14:53:42.157398  7619 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:42.218242  2677 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:42.213511  7690 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:42.227677  2612 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.232569  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:42.194896  7908 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 14:53:42.228510  2684 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:42.150491  7909 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.235970  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:42.181062  7272 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:42.229815  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:42.238791  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 152733us
W20260709 14:53:42.250890  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 295019us
I20260709 14:53:42.178738  7912 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:42.242130  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:42.255961  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:42.230297  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:42.243966  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 281642us
W20260709 14:53:42.245335  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57914.8us
I20260709 14:53:42.157130  7833 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 14:53:42.192435  7750 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:42.249991  2608 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:42.255141  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:42.257277  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:42.261281  2753 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:42.227063  7876 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:42.217669  7705 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.256275  2611 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:53:42.275782  7784 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:42.276307  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.277622  7906 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.185s
I20260709 14:53:42.237200  7690 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.279666  2754 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.281356  2752 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.285190  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:42.293855  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 206927us
I20260709 14:53:42.295828  2684 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:42.222697  7920 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 14:53:42.261082  7912 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.300549  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:42.261600  7750 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.193419  7916 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.215s
I20260709 14:53:42.279527  7705 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:42.261700  7439 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:42.324885  2608 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:42.354347  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:42.359814  7929 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:42.363094  7672 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:42.368983  7904 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:42.369668  7904 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:42.370740  7904 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:42.371950  7930 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 14:53:42.372298  7904 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.375455  7828 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:42.377653  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1280) took 1193 ms. Trace:
I20260709 14:53:42.330567  7863 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:42.335528  7912 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:42.328397  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45272.4us
I20260709 14:53:42.330200  2611 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:42.265053  7876 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.193043  7918 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 14:53:42.329170  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1273) took 1364 ms. Trace:
W20260709 14:53:42.323381  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 173042us
I20260709 14:53:42.288028  7690 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.438303  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.379654  7863 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 14:53:42.381029  7834 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.802s	user 0.013s	sys 0.002s
I20260709 14:53:42.284806  7272 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:42.381901  7932 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:42.329661  7668 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:42.378242  3545 rpcz_store.cc:276] 0709 14:53:41.184408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.189095 (+  4687us) service_pool.cc:224] Handling call
0709 14:53:41.189691 (+   596us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:41.755484 (+565793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:41.755533 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:41.812966 (+ 57433us) log.cc:844] Serialized 52 byte log entry
0709 14:53:42.063838 (+250872us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:42.063874 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:42.063939 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:42.377214 (+313275us) raft_consensus.cc:1685] finished
0709 14:53:42.377239 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:42.377438 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36405,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":728839,"threads_started":2,"wal-append.queue_time_us":433671}
W20260709 14:53:42.387785  7919 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 14:53:42.392396  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1429) took 1221 ms. Trace:
W20260709 14:53:42.396759  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 147173us
I20260709 14:53:42.401790  7828 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:42.436430  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 178415us
I20260709 14:53:42.442127  7863 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:42.421337  2603 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.614s	user 0.012s	sys 0.000s
I20260709 14:53:42.482630  2603 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:42.295162  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:42.484194  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 189130us
W20260709 14:53:42.486430  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:42.286635  7917 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:42.490125  2605 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:42.335331  7750 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:42.279474  7832 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.451568  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.452107  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.438534  7876 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.456754  2682 rpcz_store.cc:276] 0709 14:53:41.170694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.592978 (+422284us) service_pool.cc:224] Handling call
0709 14:53:41.593514 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:41.606885 (+ 13371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:41.608189 (+  1304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:41.678709 (+ 70520us) log.cc:844] Serialized 26 byte log entry
0709 14:53:42.062971 (+384262us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:42.063055 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:42.063167 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:42.392038 (+328871us) raft_consensus.cc:1685] finished
0709 14:53:42.392068 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:42.392316 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7325,"raft.queue_time_us":8024,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":68593,"thread_start_us":327309,"threads_started":1,"wal-append.queue_time_us":594284}
W20260709 14:53:42.458681  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 247147us
W20260709 14:53:42.459403  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 207346us
I20260709 14:53:42.472383  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.478519  2610 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:53:42.478287  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 236932us
I20260709 14:53:42.406072  2754 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:42.452216  7657 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:42.448834  7932 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.433192  3540 rpcz_store.cc:276] 0709 14:53:40.968887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.975203 (+  6316us) service_pool.cc:224] Handling call
0709 14:53:41.015090 (+ 39887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:41.086549 (+ 71459us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:41.806077 (+719528us) log.cc:844] Serialized 14 byte log entry
0709 14:53:42.304830 (+498753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:42.304919 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:42.305005 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:42.305110 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:42.305327 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6849,"prepare.queue_time_us":797791,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":65,"raft.queue_time_us":37687,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":31929,"thread_start_us":1120851,"threads_started":3,"wal-append.queue_time_us":425748}
I20260709 14:53:42.511966  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:42.509999  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.497566  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.475355  7439 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:42.515369  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:42.515448  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:42.519359  2677 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:42.448889  7834 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:42.497006  7920 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:42.535058  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93047.5us
W20260709 14:53:42.535998  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83982.1us
I20260709 14:53:42.474102  7636 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.518621  7439 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.554069  7439 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:42.560473  7272 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:42.561498  7863 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.522228  7833 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:42.538133  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 261887us
I20260709 14:53:42.541795  7640 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:42.551913  7920 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:42.552738  7922 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 14:53:42.542989  7949 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:42.553246  7945 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 14:53:42.547770  2604 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:42.518621  7932 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:42.565022  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.577056  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:42.519328  7831 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 14:53:42.541702  7699 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:42.586941  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.499387  7917 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:42.588861  2612 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:42.584911  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.545795  7933 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:42.586105  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.575768  7640 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:42.599116  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62223.6us
I20260709 14:53:42.640312  2754 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:42.644055  2752 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:42.661754  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 164225us
I20260709 14:53:42.599511  7699 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.599377  7863 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:42.669315  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 159452us
I20260709 14:53:42.627426  7944 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:42.632177  7833 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 14:53:42.673146  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96209.2us
I20260709 14:53:42.599079  7949 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:42.588006  7941 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:42.672310  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:42.675654  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:42.678432  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:42.684028  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:42.675424  7695 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.000s
I20260709 14:53:42.673906  7906 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:42.675357  7944 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.676367  7833 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.691565  2651 leader_election.cc:400] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16.
W20260709 14:53:42.686849  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 187237us
I20260709 14:53:42.695638  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.672786  7699 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:42.668387  7272 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:42.688405  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 102422us
I20260709 14:53:42.672470  7929 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:42.708259  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.690034  7717 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 14:53:42.715202  7938 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:42.715847  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.716336  7938 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:42.716990  7938 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.717329  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.715353  7717 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:42.682423  7657 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:42.722945  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.707295  7945 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:42.753373  7948 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:42.753729  7945 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:42.754360  7945 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.711153  7833 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.727626  7929 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:42.721711  7941 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 14:53:42.751426  7953 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 14:53:42.752070  7939 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:42.758555  7939 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.760422  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.761276  2433 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 12 to 13, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 13 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:42.752653  7934 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:42.753023  7820 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:42.699299  7695 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.708258  7906 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:42.747148  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 164943us
W20260709 14:53:42.767094  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83176.1us
W20260709 14:53:42.772233  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64066.8us
W20260709 14:53:42.779960  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84475.1us
I20260709 14:53:42.792709  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.709746  7944 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:42.780799  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 115011us
I20260709 14:53:42.731362  7916 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:42.723578  7717 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:42.738865  7828 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:42.824712  2680 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:42.835963  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 117355us
I20260709 14:53:42.763077  7933 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:42.830811  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 158603us
I20260709 14:53:42.808140  7906 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.811190  7820 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:42.731853  7272 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:42.887646  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.887991  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:42.813495  7948 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:42.906103  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:42.907233  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:42.907418  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 114863us
I20260709 14:53:42.908419  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:42.916258  2758 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:42.917749  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 185179us
I20260709 14:53:42.919113  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.920121  7957 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:42.887002  7831 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:42.925717  7937 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:42.926205  7831 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.895838  7820 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:42.926986  7958 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:42.927511  7958 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:42.927856  7958 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:42.928947  7272 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:42.930940  7695 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:42.931147  7717 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:42.910744  7938 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:42.932927  7717 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:42.933028  7938 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:42.933339  7695 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 14:53:42.788594  7640 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.827901  7946 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:42.899580  7941 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:42.908628  7939 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:42.923004  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56829.2us
I20260709 14:53:42.910545  7828 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 14:53:42.927042  7820 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.925573  7929 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:42.804930  7934 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:42.924700  2751 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:42.942746  7942 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:53:42.943094  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1415) took 1878 ms. Trace:
I20260709 14:53:42.943491  2685 rpcz_store.cc:276] 0709 14:53:41.064907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.065075 (+   168us) service_pool.cc:224] Handling call
0709 14:53:41.257016 (+191941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:41.272253 (+ 15237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:41.754811 (+482558us) log.cc:844] Serialized 14 byte log entry
0709 14:53:42.163984 (+409173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:42.164060 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:42.209529 (+ 45469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:42.209628 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:42.941432 (+731804us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b18000a0900. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4
0709 14:53:42.942880 (+  1448us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400269150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:42.943024 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67823,"prepare.queue_time_us":569353,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":68122,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":19605,"spinlock_wait_cycles":466862080,"thread_start_us":747007,"threads_started":2,"wal-append.queue_time_us":642544}
W20260709 14:53:42.948248  7931 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.449s	user 0.002s	sys 0.001s
W20260709 14:53:42.952355  7619 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.699s	user 0.014s	sys 0.000s
I20260709 14:53:42.952924  7619 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:42.954038  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:42.921162  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:42.926601  7937 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:42.923890  7032 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.807s	user 0.011s	sys 0.005s
I20260709 14:53:42.969432  7961 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:42.970683  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 673399us
I20260709 14:53:42.970466  7961 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.972730  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:42.973066  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:42.973228  7961 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:42.974159  7961 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:42.975025  7619 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:42.974674  7961 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:42.978057  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:42.979333  7961 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:42.982759  7920 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:42.985280  7916 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:42.986900  7916 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:42.916424  7918 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:42.926649  7831 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:42.936903  7640 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:42.997035  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:43.002449  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83474.9us
I20260709 14:53:42.915117  7657 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.011960  7948 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.015571  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:43.015998  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:43.016577  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.017398  2751 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:43.027120  7964 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:43.028965  7672 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:43.029629  7672 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.030048  7672 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:43.030537  7611 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:43.031091  7611 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:43.030905  7960 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.201s
I20260709 14:53:43.033417  7952 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:43.034984  7933 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:43.037361  7959 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 14:53:42.999643  7929 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:42.999847  7937 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.045933  7964 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:43.046397  7876 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.004s
I20260709 14:53:42.882529  7949 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:42.997123  7032 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.020722  7917 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.052223  7032 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.052423  7964 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:43.056502  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:43.062144  2677 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.002s
I20260709 14:53:43.073808  7934 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.075771  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:43.077665  7935 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 14:53:43.062667  2758 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:43.077382  7958 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.078630  7967 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:43.079703  7960 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.084300  2677 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:43.086194  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:43.173936  2685 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:43.069358  7905 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.970s	user 0.008s	sys 0.008s
W20260709 14:53:43.070535  7750 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.000s
I20260709 14:53:43.099983  2751 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:43.084233  7904 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.005s
W20260709 14:53:43.081197  7690 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.006s
W20260709 14:53:43.102905  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116527us
I20260709 14:53:43.209245  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.209764  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.210248  2677 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:43.080751  7917 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.105211  2681 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 14:53:43.107087  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104685us
I20260709 14:53:43.104923  7960 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:43.091133  7929 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.110546  2686 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 14:53:43.096931  7032 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:43.117082  7832 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.158515  7820 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.121280  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1436) took 1028 ms. Trace:
I20260709 14:53:43.123453  7971 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.124600  2682 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:43.126377  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.129005  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:43.244338  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 115455us
W20260709 14:53:43.130230  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:43.198727  7454 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.102299  7967 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:43.132853  7918 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:43.085638  7952 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:43.091259  7828 raft_consensus.cc:2764] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 14:53:43.135375  7668 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.135629  7636 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:43.136708  7705 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.012s
I20260709 14:53:43.067708  7876 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.136523  7831 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.137166  7834 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:43.137188  7923 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.473s	user 0.003s	sys 0.000s
I20260709 14:53:43.282658  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.105679  2678 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 14:53:43.144767  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 601772us
W20260709 14:53:43.139333  3411 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.699s (SENT)
I20260709 14:53:43.148845  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.104539  7978 raft_consensus.cc:2764] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:53:43.149693  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 926) took 1922 ms. Trace:
I20260709 14:53:43.288784  3537 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:43.150297  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.151908  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 919) took 1650 ms. Trace:
I20260709 14:53:43.152386  7958 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.156483  7833 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:43.159960  7912 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.715s	user 0.015s	sys 0.000s
I20260709 14:53:43.160194  2433 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 10 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:43.161391  2760 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:43.169394  7611 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:43.068764  7909 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.782s	user 0.013s	sys 0.002s
I20260709 14:53:43.145709  7937 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:43.194905  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:43.198732  7439 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.005s
I20260709 14:53:43.237668  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:43.413895  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 492763us
I20260709 14:53:43.231355  2686 pending_rounds.cc:85] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 7
I20260709 14:53:43.243381  2680 rpcz_store.cc:276] 0709 14:53:42.093261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:42.182190 (+ 88929us) service_pool.cc:224] Handling call
0709 14:53:43.120994 (+938804us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:43.121197 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397275008}
I20260709 14:53:43.253217  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.258426  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.231957  7960 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.213997  7917 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:43.415278  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 177750us
I20260709 14:53:43.286690  7705 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.418332  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:43.271291  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.272369  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.279036  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.193770  7699 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.284030  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.285686  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.252554  7657 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:43.209020  7750 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.258978  7949 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:43.292182  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.274413  7952 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.259306  7918 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:43.198856  7905 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.287377  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:43.290125  3545 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:43.230654  7690 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.236084  7929 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.252023  7454 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.287168  7598 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 14:53:43.327391  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 95397us
I20260709 14:53:43.361536  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.373411  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.455327  2681 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:43.288834  2679 rpcz_store.cc:276] 0709 14:53:41.227610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.802775 (+575165us) service_pool.cc:224] Handling call
0709 14:53:41.812996 (+ 10221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:41.834199 (+ 21203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:41.834257 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:41.894676 (+ 60419us) log.cc:844] Serialized 26 byte log entry
0709 14:53:42.212478 (+317802us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:42.228042 (+ 15564us) log.cc:844] Serialized 14 byte log entry
0709 14:53:42.228354 (+   312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:42.228453 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:43.149400 (+920947us) raft_consensus.cc:1685] finished
0709 14:53:43.149427 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:43.149605 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41292,"prepare.queue_time_us":16057,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":9501,"raft.queue_time_us":15071,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":22883,"thread_start_us":208261,"threads_started":1,"wal-append.queue_time_us":948704}
I20260709 14:53:43.337498  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.426702  7657 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.337611  7909 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.299896  7925 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 14:53:43.371064  7968 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 14:53:43.367355  7965 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.273731  7799 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 14:53:43.368384  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 229099us
I20260709 14:53:43.280644  7668 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.342527  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:43.300146  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.372140  7967 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.402076  3541 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 14:53:43.356757  7958 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.253192  7971 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.375370  7611 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:43.118577  7921 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.952s	user 0.008s	sys 0.003s
I20260709 14:53:43.414603  2686 pending_rounds.cc:107] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 14:53:43.509815  2686 raft_consensus.cc:2889] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:43.415695  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 157373us
I20260709 14:53:43.475018  7979 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:43.422281  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 138392us
I20260709 14:53:43.423300  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.301388  7912 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.425817  3547 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:43.289755  2604 rpcz_store.cc:276] 0709 14:53:41.501484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:41.874655 (+373171us) service_pool.cc:224] Handling call
0709 14:53:41.875384 (+   729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:42.578905 (+703521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:43.151617 (+572712us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:43.151810 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545499904}
I20260709 14:53:43.539570  3546 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:43.541049  7912 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:43.413702  7906 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.503s	user 0.015s	sys 0.002s
I20260709 14:53:43.542119  7906 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:43.425109  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.015s
W20260709 14:53:43.543255  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.015s	sys 0.015s
I20260709 14:53:43.432745  3540 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:43.440572  2677 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:43.442893  2683 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 10. Current term is 11. Ops: []
I20260709 14:53:43.422312  7917 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.446767  7980 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 14:53:43.421496  7974 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:43.449043  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 165851us
I20260709 14:53:43.422724  7032 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:43.333930  2433 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 11, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:43.450938  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:43.467182  2678 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:43.431892  7978 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.558682  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:43.564513  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113765us
I20260709 14:53:43.430934  7937 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.424100  7991 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:43.567646  7991 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:43.568190  7991 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:43.569784  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.571447  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:43.572788  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.573479  2686 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:43.574863  7272 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:53:43.575448  7272 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.575884  7272 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:43.512319  7989 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:43.579035  2430 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 161) took 61198 ms (client timeout 15000 ms). Trace:
I20260709 14:53:43.581508  7705 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:43.583503  7983 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 14:53:43.499871  7834 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 14:53:43.414372  7439 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:43.457739  7932 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.005s
I20260709 14:53:43.442574  7964 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:43.501591  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 159194us
W20260709 14:53:43.503737  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 203691us
I20260709 14:53:43.618943  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.621254  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.473359  7657 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:43.622519  3541 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:43.627511  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.628692  3537 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.631479  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.634199  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.635334  3543 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:43.511888  3409 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.420045  2685 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:43.518671  7973 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 14:53:43.524253  7905 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.526105  7454 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.527182  7942 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:43.639415  7942 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:43.640513  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.641255  7942 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.531720  7960 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.532158  7977 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.498047  7965 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.533948  7690 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.554942  7912 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:43.439894  7956 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 14:53:43.537529  7958 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.545584  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.515307  2680 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:53:43.534503  7945 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.002s
I20260709 14:53:43.566229  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:43.575454  2433 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 9 to 10, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:53:43.588444  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 707993us
I20260709 14:53:43.597653  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.657235  2679 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:43.657410  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:43.589597  7699 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.595125  7876 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.588223  7833 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.606873  7828 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:43.527769  7982 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 14:53:43.642577  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:43.612457  7990 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 14:53:43.645447  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 133642us
I20260709 14:53:43.652477  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.598915  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.658903  2686 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:43.659832  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:43.701802  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 937) took 1215 ms. Trace:
I20260709 14:53:43.702154  2606 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:43.692344  7454 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.561246  7978 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.702538  2683 rpcz_store.cc:276] 0709 14:53:42.486604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:42.486784 (+   180us) service_pool.cc:224] Handling call
0709 14:53:42.507802 (+ 21018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:43.549974 (+1042172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:43.701425 (+151451us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:43.701706 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142180096}
W20260709 14:53:43.579623  2430 rpcz_store.cc:269] 0709 14:52:42.166544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:42.175530 (+  8986us) service_pool.cc:224] Handling call
0709 14:53:43.360165 (+61184635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:53:35.276636 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:53:35.402172 (+125536us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:53:35.402241 (+    69us) write_op.cc:435] Acquired schema lock
0709 14:53:35.402267 (+    26us) tablet.cc:662] Decoding operations
0709 14:53:35.624589 (+222322us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:53:35.643435 (+ 18846us) write_op.cc:641] Partition lock acquired for write op
0709 14:53:35.646365 (+  2930us) tablet.cc:685] Acquiring locks for 13 operations
0709 14:53:36.311361 (+664996us) tablet.cc:701] Row locks acquired
0709 14:53:36.311412 (+    51us) write_op.cc:260] PREPARE: finished
0709 14:53:36.311640 (+   228us) write_op.cc:270] Start()
0709 14:53:36.311819 (+   179us) write_op.cc:276] Timestamp: P: 1783608816311554 usec, L: 0
0709 14:53:36.311854 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 14:53:36.312760 (+   906us) log.cc:844] Serialized 5197 byte log entry
0709 14:53:36.666766 (+354006us) op_driver.cc:464] REPLICATION: finished
0709 14:53:37.007081 (+340315us) write_op.cc:301] APPLY: starting
0709 14:53:37.007201 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 14:53:37.261021 (+253820us) tablet.cc:1370] finished BulkCheckPresence
0709 14:53:37.261185 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:53:37.734740 (+473555us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b180032b720. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:53:38.335643 (+600903us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:53:38.336014 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 14:53:38.336079 (+    65us) write_op.cc:312] APPLY: finished
0709 14:53:38.336579 (+   500us) log.cc:844] Serialized 120 byte log entry
0709 14:53:38.427340 (+ 90761us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:38.461666 (+ 34326us) write_op.cc:454] Released schema lock
0709 14:53:38.473933 (+ 12267us) write_op.cc:341] FINISH: Updating metrics
0709 14:53:38.598913 (+124980us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:38.604892 (+  5979us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5252453,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":330405,"apply.run_cpu_time_us":75666,"apply.run_wall_time_us":1478308,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58441,"num_ops":13,"prepare.queue_time_us":311811,"prepare.run_cpu_time_us":15135,"prepare.run_wall_time_us":1267174,"raft.queue_time_us":174171,"raft.run_cpu_time_us":61875,"raft.run_wall_time_us":306326,"replication_time_us":330895,"spinlock_wait_cycles":28598016,"thread_start_us":806917,"threads_started":5,"wal-append.queue_time_us":390278,"wal-append.run_cpu_time_us":18631,"wal-append.run_wall_time_us":2481685}]]}
I20260709 14:53:43.566748  7820 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:43.590356  7944 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.796s	user 0.010s	sys 0.015s
I20260709 14:53:43.605943  7937 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.594748  7909 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.679862  2684 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:43.680759  2760 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:43.601230  7971 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.569804  7984 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.684607  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:43.686647  2433 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 9 to 11, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:43.605412  7032 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:43.592922  7906 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:43.689332  2685 raft_consensus.cc:2368] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:53:43.690783  2677 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:43.740216  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.661196  7863 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:43.551990  7979 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.646770  7949 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:43.709649  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:43.680490  7672 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.000s
I20260709 14:53:43.656502  7916 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:43.719736  3537 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:43.600258  7925 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.085s
W20260709 14:53:43.723464  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71084.2us
I20260709 14:53:43.777334  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.602600  7970 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 14:53:43.745071  2604 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:43.654847  7932 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.744823  7984 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.744611  7944 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.761302  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.738101  7699 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:43.763130  2433 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 4 to 11, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:43.803627  3536 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:43.764945  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.665741  7945 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.765920  2685 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 14:53:43.683467  7828 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:53:43.772840  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 277538us
I20260709 14:53:43.815948  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:43.731459  7717 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 14:53:43.661890  7977 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:43.663314  7964 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.673084  7912 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.811002  7598 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:43.731703  7784 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:43.782366  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.667112  7948 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 14:53:43.722186  7905 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.738380  7937 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.698041  7956 consensus_queue.cc:1243] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:43.728070  7978 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.796567  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.798295  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:43.881449  8001 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.887310  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.887176  8001 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:43.847105  7904 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.889858  7964 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:43.889647  7904 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.891026  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.892298  7904 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.892623  3541 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 9.
I20260709 14:53:43.900758  7949 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.848634  7598 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.903209  7598 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:43.882687  3542 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:43.903891  7960 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:43.906222  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.906742  7958 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:43.908006  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.908663  7965 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:43.910494  2752 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:43.912009  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:43.767915  7863 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.800928  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.569856  7974 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.765748  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:43.776937  7909 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.811544  7957 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:43.764969  7032 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.811928  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.812650  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.927289  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.808400  7820 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:43.812498  2433 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 10 to 11, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:43.813544  7916 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:43.721071  7454 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.939036  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.795269  2606 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:43.785316  7750 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:43.840651  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100588us
I20260709 14:53:43.767697  7668 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:43.846428  2682 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:43.918892  8001 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.800475  7984 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:43.781962  7672 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.833047  7828 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:43.879662  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 317243us
I20260709 14:53:43.911792  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.972013  2752 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:43.908720  2758 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:43.913062  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:43.883342  3538 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 15
W20260709 14:53:43.879707  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 180387us
I20260709 14:53:43.973891  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:43.903229  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.849164  7784 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:43.812795  7439 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:43.931720  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:43.933868  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.910804  7912 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:43.666422  7705 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 14:53:43.944402  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.922590  7863 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:43.946637  2679 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:43.921875  7833 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:44.023391  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:43.871714  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89445.3us
I20260709 14:53:43.949510  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:43.952376  2433 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 8 to 9, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:43.923323  7971 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:43.883658  7952 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:43.929369  7820 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:43.978134  3542 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:43.980809  7970 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.994300  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.030428  2604 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:43.995266  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:43.994410  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:44.029616  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 320098us
I20260709 14:53:44.000821  3547 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:44.003175  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.004510  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.004736  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:43.882550  7905 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.007088  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.001328  7784 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:44.011128  2751 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:44.009907  2757 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:44.014729  2683 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:44.017045  2754 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:44.018983  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:44.005504  7972 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:43.953444  7957 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.027091  7636 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:43.930907  7909 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:43.994772  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.031673  2611 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:44.034198  2752 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:44.009846  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57133.4us
I20260709 14:53:44.035552  2679 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:44.060609  2755 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:43.965662  7916 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:44.001719  7958 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:43.963097  7699 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:44.045739  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.047674  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.045469  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.075707  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.075940  3542 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:44.076328  2757 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:44.024907  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.075774  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.052124  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.045423  2433 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 9, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:44.024014  8017 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.035254  7820 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:44.019951  7948 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 14:53:44.072779  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.033708  7971 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.060853  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.083922  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:44.144550  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.070418  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.026448  7640 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 14:53:44.086606  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:44.087211  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 62375.6us
I20260709 14:53:44.170895  2679 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:44.087906  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.110973  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.044344  7272 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.114653  2758 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:44.042976  7970 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.119076  7944 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.194954  2433 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 8, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 8 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:44.196092  7439 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:44.196770  7439 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.196617  2433 catalog_manager.cc:5697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:44.197889  7439 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:44.199286  7987 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.696s	user 0.005s	sys 0.000s
W20260709 14:53:44.199514  8021 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 14:53:44.199968  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.200922  2605 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:44.201938  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.088647  8017 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.203938  7699 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:44.198486  2433 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 10, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:44.204514  7699 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.204872  8017 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.120365  7957 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.122879  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.205924  2607 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.205780  2433 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 9 to 11, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:44.207472  2433 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 7 to 8, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:44.120613  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.209041  2680 raft_consensus.cc:2368] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:44.125241  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.211058  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.211681  7672 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.212117  3540 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:44.212704  2679 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:53:44.213034  8011 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 14:53:44.213203  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.213510  7668 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.194579  7272 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.214094  3540 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:44.213793  2433 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 3 to 4, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:53:44.215097  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.215893  3545 raft_consensus.cc:2368] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:53:44.215945  2433 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:44.216795  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.202814  2605 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:44.218111  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.219728  7909 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.028829  7912 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 14:53:44.144892  3536 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:44.221274  8009 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 14:53:44.077328  7958 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.222002  7958 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.170374  2604 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:44.223712  8015 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:44.169637  2683 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:44.171484  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:44.225514  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54102us
I20260709 14:53:44.225890  7944 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:44.226495  7944 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:44.189236  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110963us
I20260709 14:53:44.228569  7944 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.229310  3409 leader_election.cc:400] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:44.229758  3409 leader_election.cc:403] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:44.230508  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.231056  7668 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 14:53:44.231900  7918 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.781s	user 0.013s	sys 0.000s
I20260709 14:53:44.232393  7918 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.233494  2753 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:44.233645  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.183427  2752 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:44.192207  7454 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.191957  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:44.194628  7991 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.626s	user 0.014s	sys 0.000s
I20260709 14:53:44.246793  3547 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.147763  8008 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:44.247359  8008 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.247779  8008 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:44.248044  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.249571  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.186210  7717 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.055799  7972 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.135486  7636 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.144114  7932 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.310088  7937 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.311995  2604 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:44.312987  7937 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.313848  7937 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:44.314257  7937 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:44.316324  3536 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:44.318069  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.319216  3536 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.193181  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.193416  7960 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:44.195245  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.201973  7690 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:44.221225  7912 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:44.224241  8015 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:44.191833  7977 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.235424  2751 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:44.140040  2754 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:44.191695  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.237919  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:44.243168  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 164442us
I20260709 14:53:44.245167  2607 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:44.312991  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:44.251368  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.287986  3545 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.293493  7934 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.173841  7705 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.366657  3547 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.369544  3541 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:44.369340  7820 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.367340  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.234622  7918 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.263345  2752 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:53:44.303843  7943 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:44.251255  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.372759  2686 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:44.372396  8015 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.374063  8015 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:44.372407  7943 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:44.374981  7934 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.375468  7943 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.376935  7943 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:44.377705  7934 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.304468  7598 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.305531  7948 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.306169  7750 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:53:44.308620  7636 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.346493  2758 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 14:53:44.348896  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.351346  2757 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
W20260709 14:53:44.237769  7917 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.003s
I20260709 14:53:44.350756  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1402) took 1246 ms. Trace:
I20260709 14:53:44.178239  7945 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.406162  7917 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.406168  2681 rpcz_store.cc:276] 0709 14:53:43.104516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.471423 (+366907us) service_pool.cc:224] Handling call
0709 14:53:43.480913 (+  9490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:43.501221 (+ 20308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:43.501257 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:43.505426 (+  4169us) log.cc:844] Serialized 26 byte log entry
0709 14:53:43.612640 (+107214us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b18000b9680. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:53:44.005148 (+392508us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:44.005196 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.005283 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:44.320746 (+315463us) raft_consensus.cc:1685] finished
0709 14:53:44.320782 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:44.350663 (+ 29881us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4110,"raft.queue_time_us":6051,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":7539,"spinlock_wait_cycles":35760512,"thread_start_us":392287,"threads_started":1,"wal-append.queue_time_us":760826}
I20260709 14:53:44.408362  7831 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.408617  2760 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:44.415104  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 775641us
I20260709 14:53:44.415683  7970 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.416707  7905 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.417320  7948 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.386392  7916 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.418047  7917 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.200856  8023 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.251132  7909 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:53:44.423724  2680 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:44.424045  8023 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.425700  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.426357  8023 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.361651  2606 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 14:53:44.427565  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.428315  2681 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:44.359253  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46363.1us
I20260709 14:53:44.345710  7690 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.293725  7974 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.371070  7932 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:44.430059  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.379215  7705 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.430317  7690 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.420221  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.431635  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:44.432011  7690 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:44.432313  2757 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:44.433104  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:44.433483  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.433888  2680 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.434321  2757 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:44.434429  2575 leader_election.cc:400] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:53:44.434758  2575 leader_election.cc:403] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 14:53:44.295411  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 219198us
I20260709 14:53:44.435525  7968 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:44.436086  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:44.392649  3536 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.236764  7904 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.206s
I20260709 14:53:44.394088  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.437353  7968 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 14:53:44.395131  7598 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.396116  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.391273  7750 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:44.440524  7750 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.441852  7750 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 14:53:44.396425  7978 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.003s
I20260709 14:53:44.443449  7978 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.444800  2756 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:44.446853  7978 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.399531  7958 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.447765  7929 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.449434  7958 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.348739  7960 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.304759  7979 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.354308  7912 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:44.463750  2576 leader_election.cc:400] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:53:44.405368  3540 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 14:53:44.419728  7948 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.450358  7960 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:44.467988  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.469810  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1469) took 1240 ms. Trace:
I20260709 14:53:44.421108  7970 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.470216  7640 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:44.470518  7970 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.472107  3543 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:44.421471  2683 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 14:53:44.472365  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.422894  7699 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.430140  7932 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:44.251559  7717 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.489334  7979 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 14:53:44.489861  7979 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.490257  7979 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:44.490769  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.492192  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.493391  2755 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:44.372798  3541 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:53:44.436158  7799 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.450274  3537 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:53:44.438534  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:44.390395  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.437785  3536 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 14:53:44.403468  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 939) took 1263 ms. Trace:
I20260709 14:53:44.498131  2682 rpcz_store.cc:276] 0709 14:53:43.140359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.264637 (+124278us) service_pool.cc:224] Handling call
0709 14:53:44.403157 (+1138520us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400377154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:44.403362 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975182592}
I20260709 14:53:44.437745  2753 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:44.406517  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.314661  7937 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.464859  3540 pending_rounds.cc:85] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:53:44.465350  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.435525  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:44.451233  7876 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.024s	sys 0.031s
W20260709 14:53:44.503077  7876 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.024s	sys 0.031s
I20260709 14:53:44.469547  7831 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 14:53:44.503602  7912 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.504935  7912 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:44.505160  7831 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:44.467042  7974 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 14:53:44.422549  2679 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:44.510798  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.511222  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.511732  2611 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:44.511961  2604 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:44.510557  2607 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:53:44.515120  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.515586  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:44.474246  2754 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:53:44.516364  2610 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:44.517938  2610 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:44.517998  2604 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:44.519826  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:44.521154  7971 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.521932  7640 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:44.527168  7705 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.532897  7717 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.535734  7820 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.538277  3537 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:44.246860  7991 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.540896  7991 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.544945  7699 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.547282  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:44.550040  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.426537  2686 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:44.551685  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 938) took 2052 ms. Trace:
I20260709 14:53:44.552160  2684 rpcz_store.cc:276] 0709 14:53:42.499369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:42.499569 (+   200us) service_pool.cc:224] Handling call
0709 14:53:42.521928 (+ 22359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:43.739106 (+1217178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.551423 (+812317us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002b3c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:44.551620 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651650432}
I20260709 14:53:44.553818  7958 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:44.554006  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.554359  7958 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:44.470228  3546 rpcz_store.cc:276] 0709 14:53:43.229401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.229577 (+   176us) service_pool.cc:224] Handling call
0709 14:53:44.469536 (+1239959us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400551254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:44.469711 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285039872}
W20260709 14:53:44.403498  8019 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 14:53:44.474656  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:44.486756  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:44.500862  2751 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:44.473493  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:44.488140  7966 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.488492  7945 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 14:53:44.488904  7967 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:44.489125  7932 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:44.560824  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:44.502476  2757 raft_consensus.cc:2368] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:44.502801  8030 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:44.501852  3540 pending_rounds.cc:107] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 10.7
W20260709 14:53:44.515324  7657 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.892s	user 0.014s	sys 0.000s
I20260709 14:53:44.487602  7970 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.562202  2680 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:44.562438  2677 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:44.557778  3537 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:44.497725  3536 pending_rounds.cc:85] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 8
W20260709 14:53:44.559545  3538 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.003s
I20260709 14:53:44.625428  3536 pending_rounds.cc:107] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 14:53:44.625555  3538 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:53:44.566423  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93020.1us
I20260709 14:53:44.484302  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:44.627123  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:44.570344  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:44.566200  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:44.504323  7611 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.990s	user 0.015s	sys 0.000s
I20260709 14:53:44.625864  3536 raft_consensus.cc:2889] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:44.557888  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83332.5us
I20260709 14:53:44.621446  3540 raft_consensus.cc:2889] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:44.608536  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.492954  7909 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:44.621933  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:44.631549  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 147365us
I20260709 14:53:44.713038  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.626626  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:44.626335  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:44.554813  7784 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.004s
W20260709 14:53:44.637336  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63908.5us
W20260709 14:53:44.638304  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.036s
W20260709 14:53:44.726895  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.000s	sys 0.037s
I20260709 14:53:44.727072  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.640973  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.659245  3546 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 14:53:44.563370  7976 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.019s	sys 0.042s
I20260709 14:53:44.563258  7820 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.706805  7717 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:44.665045  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.555227  7968 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:44.625269  7705 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:44.555411  7929 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 14:53:44.573573  7965 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:44.570983  7831 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:44.753405  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.757135  3536 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.671622  8006 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 14:53:44.667486  8019 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 14:53:44.628906  7668 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:44.629457  7657 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.677060  7960 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:44.680635  7863 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.002s
I20260709 14:53:44.555444  7958 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.662907  7970 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 14:53:44.555871  7934 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 14:53:44.686930  8003 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:44.689556  8034 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 14:53:44.702138  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75099.1us
I20260709 14:53:44.658998  7945 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 14:53:44.707453  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 141347us
I20260709 14:53:44.729635  7991 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 14:53:44.664915  7972 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.721804  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:44.774585  7984 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.005s
W20260709 14:53:44.724378  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98129.8us
I20260709 14:53:44.794099  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:44.796545  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:44.724323  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 97798us
I20260709 14:53:44.798273  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.725314  2609 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:44.639772  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.572595  7966 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.624269  8008 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.568135  7699 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.734298  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.736920  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:44.737187  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:44.738854  3541 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:44.632479  8030 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.736572  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.748072  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:44.711153  7964 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:44.746548  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:44.734800  7976 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.020s	sys 0.042s
I20260709 14:53:44.733541  7949 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:44.717448  7943 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:44.756026  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.744933  7705 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:44.739703  7828 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.727s	user 0.014s	sys 0.000s
W20260709 14:53:44.752002  7975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.000s	sys 0.028s
I20260709 14:53:44.766551  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.755959  7952 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.751657  7611 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 14:53:44.708057  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.818814  8035 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:44.843850  8035 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.844352  8035 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:44.818980  7997 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:44.819149  7984 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.851179  2678 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:44.819675  7945 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.760342  7956 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.852782  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1296) took 1092 ms. Trace:
I20260709 14:53:44.752561  7717 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:44.853191  7956 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:44.663374  7032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.025s	sys 0.039s
W20260709 14:53:44.854517  7032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.026s	sys 0.039s
I20260709 14:53:44.854125  7717 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.743783  7820 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.760123  8006 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:44.856287  7956 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.856972  7820 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:44.857779  3547 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:44.800239  3537 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:44.858276  7960 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.859303  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.823827  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.860030  3537 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:44.826880  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78878.1us
I20260709 14:53:44.861088  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.862334  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.735229  7784 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.800796  7598 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.834048  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.827915  3542 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:44.826190  2677 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:44.801448  8023 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:44.802186  7970 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:44.802378  7799 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:53:44.803442  7929 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:44.804785  7657 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.763166  7965 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.812536  2756 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:44.748063  7834 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 14:53:44.778549  7944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.018s
I20260709 14:53:44.827181  7964 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.853257  7945 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.787873  7968 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:44.863018  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64909.2us
W20260709 14:53:44.839047  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 131093us
I20260709 14:53:44.820744  3540 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:44.821189  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 84381.9us
I20260709 14:53:44.834795  3536 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:44.833374  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 111702us
W20260709 14:53:44.866578  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 70139.8us
I20260709 14:53:44.853257  3538 rpcz_store.cc:276] 0709 14:53:43.760739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.764594 (+  3855us) service_pool.cc:224] Handling call
0709 14:53:44.852549 (+1087955us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b54004b0fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:44.852707 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181928320}
I20260709 14:53:44.875299  3546 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:44.875411  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86900.9us
I20260709 14:53:44.878057  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.889999  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:44.834368  7975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.000s	sys 0.029s
I20260709 14:53:44.822101  7966 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:44.833220  7943 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.812615  7991 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:44.838958  7974 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.833983  7972 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:44.825780  8030 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:44.835234  7952 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.867398  7948 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:44.881474  7949 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 14:53:44.929700  7991 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:44.827466  7905 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:44.938506  7905 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:44.939018  7905 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:44.938753  7991 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.940644  7991 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:44.930073  8027 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:44.943292  7984 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.943661  8027 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:44.776865  7863 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.944214  8027 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:44.819877  7909 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:44.819113  8033 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:44.933075  7972 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:44.953675  7972 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.955022  7964 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:44.954545  7965 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:44.957619  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1292) took 1198 ms. Trace:
I20260709 14:53:44.958289  3543 rpcz_store.cc:276] 0709 14:53:43.759476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.761968 (+  2492us) service_pool.cc:224] Handling call
0709 14:53:44.471895 (+709927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:44.481231 (+  9336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.957290 (+476059us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400551250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:44.957522 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631797120}
I20260709 14:53:44.876991  7997 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:53:44.960243  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.821568  8034 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:44.971563  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:44.883590  7932 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:44.875770  7904 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:44.834331  7828 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.921679  7968 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.922178  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:44.821977  8008 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:44.856349  8006 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:44.880873  7970 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:44.913153  3537 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:44.820434  7699 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:44.976320  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:44.977284  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:44.977705  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:44.981618  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:45.098977  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 121843us
I20260709 14:53:44.987856  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:44.993770  7657 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:44.994647  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:45.100294  2682 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:44.984879  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:44.995541  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.995692  8001 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:45.030290  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:45.041802  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1442) took 1568 ms. Trace:
I20260709 14:53:45.100414  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:44.883208  7598 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.045760  2685 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:45.102097  2678 rpcz_store.cc:276] 0709 14:53:43.473708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:43.700636 (+226928us) service_pool.cc:224] Handling call
0709 14:53:43.701303 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:44.851799 (+1150496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:45.041348 (+189549us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400423954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:45.041691 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643678976}
W20260709 14:53:44.963117  7944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.008s	sys 0.019s
I20260709 14:53:44.896010  7876 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:45.143805  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:44.910156  7994 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:44.884842  7929 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:44.935014  7784 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:45.166487  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 178693us
I20260709 14:53:45.103040  2611 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:45.044338  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:45.167698  3538 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:44.889211  7934 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 14:53:45.169584  3543 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:45.106601  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121853us
I20260709 14:53:45.171626  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.172562  2757 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:45.109257  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 133039us
I20260709 14:53:45.126888  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.105196  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:45.174536  2678 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:45.127678  2760 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:45.127856  8036 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:45.180439  2751 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:45.129771  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.183145  2685 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:45.184347  2685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:53:45.121233  7942 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:45.021377  7997 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 14:53:45.121744  7799 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:45.192024  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 214397us
I20260709 14:53:45.131361  8023 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:45.133630  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.133402  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.202316  2755 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:45.203491  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:45.204293  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:45.204873  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 174703us
I20260709 14:53:45.047319  7949 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 14:53:45.135735  8007 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:45.199573  2683 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:45.136613  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:45.139433  7943 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:45.155998  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:44.834327  7705 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:45.213223  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:45.213833  7705 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:45.213959  2603 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:45.206934  8040 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:45.214735  8040 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.159899  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:44.997723  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1147) took 1017 ms. Trace:
I20260709 14:53:45.228535  3547 rpcz_store.cc:276] 0709 14:53:43.980611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.369746 (+389135us) service_pool.cc:224] Handling call
0709 14:53:44.370160 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:44.858214 (+488054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.997425 (+139211us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:44.997655 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120903424}
I20260709 14:53:45.229184  7929 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.230711  7929 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:45.169391  2680 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:53:45.170913  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.174721  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:45.233681  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59103us
I20260709 14:53:45.129034  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.176326  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:45.234988  2756 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.215595  7916 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 14:53:45.235723  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.177649  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.185947  2685 raft_consensus.cc:606] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:45.175086  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:45.105598  8001 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:45.144311  7948 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:45.142967  7956 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:45.196031  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:45.198545  2758 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:45.211061  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.263427  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.155617  8034 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:45.060602  7904 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.212584  2759 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:45.170205  7989 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:45.275151  2754 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:45.203336  7799 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:45.167786  7876 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.213256  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.156606  7994 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:45.220810  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120921us
W20260709 14:53:45.279362  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.015s	sys 0.004s
W20260709 14:53:45.224376  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:45.281147  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.015s	sys 0.004s
I20260709 14:53:45.225719  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:45.281711  2608 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:45.231453  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:45.232704  8042 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 14:53:45.237118  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:45.237635  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:45.240598  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.285293  2612 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:45.286027  2755 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:45.245494  2685 raft_consensus.cc:614] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:45.237795  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:45.180411  8036 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:45.240656  2603 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:53:45.256714  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.032s
I20260709 14:53:45.259191  8023 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:45.260003  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.258445  2756 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:45.261873  2682 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:45.267405  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:45.298748  2679 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:45.299062  2610 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:45.268514  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:45.300056  3538 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:45.270351  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74468.1us
I20260709 14:53:45.230079  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:45.301544  3547 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:45.272518  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:44.906442  7979 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.276798  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:45.248147  7948 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:45.274534  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.280362  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:45.325273  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50856.9us
W20260709 14:53:45.325587  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45307.2us
I20260709 14:53:45.326159  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.326380  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:45.327201  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.327970  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:45.288539  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50985.9us
I20260709 14:53:45.269515  8003 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:45.238034  7997 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:45.221000  8007 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:45.241492  8040 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:45.262804  7956 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.224355  3542 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:45.313546  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.318627  2612 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:53:45.236649  7943 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 14:53:45.294857  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.003s	sys 0.032s
I20260709 14:53:45.278759  7989 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.373977  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:45.393911  3546 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.362429  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.395553  3537 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:53:45.361126  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:45.398031  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:45.398522  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.330108  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:45.401410  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71407.8us
I20260709 14:53:45.402974  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:45.331722  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:45.296551  7454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.021s
I20260709 14:53:45.278995  7784 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:45.212013  7970 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:45.318737  7831 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.013s
I20260709 14:53:45.405433  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.334798  7668 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:45.311704  8015 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:45.407023  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:45.408325  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:45.409691  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78055.7us
I20260709 14:53:45.375499  8030 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:45.382319  7944 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:45.335160  7948 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:45.497866  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.383095  8008 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:45.376271  7750 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:45.297708  8023 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.501916  2612 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:45.406086  7454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.001s	sys 0.022s
I20260709 14:53:45.590132  8030 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:45.591352  8030 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:45.502777  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 99939.1us
I20260709 14:53:45.503813  3538 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:45.391636  7956 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:45.430647  7831 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.009s	sys 0.015s
I20260709 14:53:45.283582  7967 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 14:53:45.514165  3542 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:45.530416  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:45.391422  7952 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:45.406330  7989 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:45.407336  7997 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:45.613143  7952 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:45.613595  7952 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.613943  7952 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.613979  7997 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.618299  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1143) took 1639 ms. Trace:
I20260709 14:53:45.620448  7978 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:45.621574  2685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:44984
W20260709 14:53:45.583047  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.025s	sys 0.091s
W20260709 14:53:45.622298  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.025s	sys 0.091s
I20260709 14:53:45.622669  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:45.622993  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 703720us
I20260709 14:53:45.623402  2603 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:45.624296  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.625634  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.626092  7834 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:45.627316  8015 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:45.627914  8015 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.628389  8015 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.626543  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.632690  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.627645  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:45.601447  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:45.604943  3540 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260709 14:53:45.636487  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:45.636960  3540 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:45.637233  3542 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:45.586174  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.621251  7978 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.622303  2685 raft_consensus.cc:606] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:45.638769  7932 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:45.658363  3543 pending_rounds.cc:85] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:53:45.658941  3543 pending_rounds.cc:107] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:45.659070  7968 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:45.659289  3543 raft_consensus.cc:2889] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:45.659427  7971 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:45.660027  7971 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:45.662398  7971 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:45.663327  3545 rpcz_store.cc:276] 0709 14:53:43.979373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.366956 (+387583us) service_pool.cc:224] Handling call
0709 14:53:45.618005 (+1251049us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:45.618214 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2685997440}
I20260709 14:53:45.664103  7799 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:45.640910  3411 leader_election.cc:400] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15.
I20260709 14:53:45.643754  2685 raft_consensus.cc:614] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:45.684520  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:45.586802  7994 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:45.689205  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48509.5us
I20260709 14:53:45.643514  7978 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:45.667318  7598 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:45.668303  7997 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:45.669996  7958 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:45.626582  7834 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.647367  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.671397  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:45.440014  7032 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.186s
I20260709 14:53:45.442159  7784 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.647756  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.783558  2682 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:53:45.652082  8046 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 14:53:45.672780  7831 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:45.676232  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:45.676612  7937 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.096s	user 0.013s	sys 0.002s
I20260709 14:53:45.652297  7967 consensus_queue.cc:1243] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:45.652730  7863 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:45.690997  7611 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:45.412278  7960 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.176s
I20260709 14:53:45.645280  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1304) took 1079 ms. Trace:
I20260709 14:53:45.645213  3547 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
W20260709 14:53:45.697369  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.641348  2612 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:45.510246  7944 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:45.724303  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.038s	sys 0.091s
W20260709 14:53:45.854763  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.038s	sys 0.091s
I20260709 14:53:45.774622  8008 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:45.544831  7966 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 14:53:45.652818  7828 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:45.405755  8033 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:45.589354  8023 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:45.804912  7699 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:45.883987  8040 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.546761  8034 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:45.824918  7975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.014s	sys 0.017s
W20260709 14:53:45.890769  7975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.015s	sys 0.018s
I20260709 14:53:45.891965  7975 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:45.892510  7975 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:45.893545  7975 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:45.776496  3412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.173s (SENT) [suppressed 12 similar messages]
W20260709 14:53:45.896597  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120163us
W20260709 14:53:45.695611  7933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.001s	sys 0.042s
W20260709 14:53:45.897403  7933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.002s	sys 0.042s
W20260709 14:53:45.897567  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.664117  8015 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.791100  8007 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:45.768620  7834 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:45.900826  7784 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:45.900533  7952 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:45.651511  8047 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.404s	user 0.000s	sys 0.000s
I20260709 14:53:45.796132  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1598) took 1237 ms. Trace:
W20260709 14:53:45.682602  7991 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.000s
I20260709 14:53:45.600430  7970 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.904073  2752 rpcz_store.cc:276] 0709 14:53:44.558989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.559111 (+   122us) service_pool.cc:224] Handling call
0709 14:53:44.559481 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:44.560016 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:44.560065 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:44.566087 (+  6022us) log.cc:844] Serialized 26 byte log entry
0709 14:53:45.392931 (+826844us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:45.392996 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:45.393103 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:45.795760 (+402657us) raft_consensus.cc:1685] finished
0709 14:53:45.795799 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:45.796036 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19339,"raft.queue_time_us":767,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":9128,"thread_start_us":727822,"threads_started":1,"wal-append.queue_time_us":916212}
I20260709 14:53:45.905439  8015 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:45.797662  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.907243  2679 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:53:45.730723  7912 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.225s	user 0.011s	sys 0.006s
I20260709 14:53:45.603644  7750 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:45.812636  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.828922  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:45.413383  8001 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:45.833330  2603 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:45.502303  8036 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.857497  3545 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:45.647447  8043 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 14:53:45.836593  3542 rpcz_store.cc:276] 0709 14:53:44.559067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.637917 (+1078850us) service_pool.cc:224] Handling call
0709 14:53:45.638174 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:45.638368 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:45.638422 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:45.638481 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:45.638529 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:45.638606 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:45.638755 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:45.858100  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:45.945663  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.864662  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.865682  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.773633  2610 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:45.948825  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.869306  2756 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:45.862366  7944 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:45.875815  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1517) took 1368 ms. Trace:
I20260709 14:53:45.875317  7831 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.876137  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.875854  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:45.876931  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:45.877444  7929 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.002s
I20260709 14:53:45.830225  7937 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:45.878227  7876 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:45.879211  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:45.880132  7611 consensus_queue.cc:1243] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:45.883433  7668 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:45.971000  7668 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.971554  7668 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:45.975196  2685 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:45.894225  7905 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.954s	user 0.008s	sys 0.005s
I20260709 14:53:45.504245  7984 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:45.895243  8008 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:45.983042  7984 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:45.984908  2680 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:45.984917  7750 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.653102  7909 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:45.987138  7909 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.987735  7909 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:45.886647  8040 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:45.912961  2608 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
W20260709 14:53:45.928247  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:45.931927  2677 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:45.879307  7974 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.933s	user 0.007s	sys 0.006s
I20260709 14:53:45.993563  7974 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:53:45.931912  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 103096us
I20260709 14:53:45.995142  8053 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:45.995900  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:45.997845  7598 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:45.998353  7598 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:45.998741  7598 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:45.952630  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.999125  2608 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:45.954438  2606 rpcz_store.cc:276] 0709 14:53:44.507235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.507392 (+   157us) service_pool.cc:224] Handling call
0709 14:53:44.507842 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:44.508124 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:44.508150 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:44.508759 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 14:53:44.884882 (+376123us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:44.884937 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.885030 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:45.875485 (+990455us) raft_consensus.cc:1685] finished
0709 14:53:45.875517 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:45.875738 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":375686,"threads_started":1,"wal-append.queue_time_us":728479}
W20260709 14:53:45.867774  7945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.043s	sys 0.121s
W20260709 14:53:45.875404  7964 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.920s	user 0.010s	sys 0.003s
I20260709 14:53:45.951524  3546 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:45.959076  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:45.982889  7905 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:45.999960  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:45.957940  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.009052  3537 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:46.009662  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1490) took 1028 ms. Trace:
I20260709 14:53:46.012024  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:46.023736  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64782.4us
I20260709 14:53:46.025625  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.031963  3546 rpcz_store.cc:276] 0709 14:53:44.981369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.312056 (+330687us) service_pool.cc:224] Handling call
0709 14:53:46.009562 (+697506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.034157  3540 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:46.050815  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.051970  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:45.960748  7904 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:46.007087  7945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.043s	sys 0.122s
I20260709 14:53:46.048866  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.068374  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.058990  3541 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:46.030520  8006 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.855s	user 0.013s	sys 0.000s
I20260709 14:53:46.052695  2608 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:53:46.091271  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.012s	sys 0.019s
I20260709 14:53:46.091859  3546 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:46.092181  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.012s	sys 0.020s
I20260709 14:53:46.028826  8001 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.098711  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.100121  2608 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:53:46.100612  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:46.002619  7976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.003s	sys 0.010s
I20260709 14:53:45.999214  7979 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:46.057855  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.079408  2612 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:46.107434  2603 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:53:46.108292  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.110157  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:46.052660  8043 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 14:53:46.036818  7933 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:46.118574  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1171) took 1375 ms. Trace:
I20260709 14:53:46.123361  3546 rpcz_store.cc:276] 0709 14:53:44.717244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.074206 (+1356962us) service_pool.cc:224] Handling call
0709 14:53:46.074748 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.092626 (+ 17878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.092956 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.052825  7929 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 14:53:46.130256  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61969.6us
I20260709 14:53:46.119901  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:46.082417  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.021s	sys 0.009s
I20260709 14:53:45.995851  8053 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:46.142100  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.022s	sys 0.009s
W20260709 14:53:46.142583  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 941977us
W20260709 14:53:46.118088  7976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.003s	sys 0.011s
W20260709 14:53:46.132357  7705 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.918s	user 0.010s	sys 0.003s
W20260709 14:53:46.168397  2607 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.019s	sys 0.012s
W20260709 14:53:46.168980  2607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.020s	sys 0.012s
W20260709 14:53:46.095656  7942 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.897s	user 0.009s	sys 0.005s
I20260709 14:53:46.093298  7968 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:46.027155  7964 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.187594  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:46.144910  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.014s	sys 0.013s
I20260709 14:53:46.117551  7904 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.173888  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:46.193873  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.014s	sys 0.013s
W20260709 14:53:46.194531  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 916747us
W20260709 14:53:46.171711  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.017s	sys 0.013s
I20260709 14:53:46.050604  8050 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 14:53:46.182374  8027 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.238s	user 0.014s	sys 0.002s
W20260709 14:53:46.206948  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.017s	sys 0.013s
I20260709 14:53:46.097692  8006 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.111722  7668 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.201548  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.244956  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:46.253057  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51866.8us
I20260709 14:53:46.207417  7705 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:53:46.240720  7958 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
I20260709 14:53:46.254390  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:46.265839  2678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.015s	sys 0.007s
W20260709 14:53:46.266455  2678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.016s	sys 0.007s
W20260709 14:53:46.203328  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.012s	sys 0.022s
W20260709 14:53:46.186314  8035 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.341s	user 0.012s	sys 0.005s
I20260709 14:53:46.179718  7942 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:46.205976  7904 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.279382  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.196137  7933 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:46.269639  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.012s	sys 0.022s
I20260709 14:53:46.272426  2604 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:46.242901  8050 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:46.283947  8050 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:46.243311  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.009s	sys 0.018s
I20260709 14:53:46.279588  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1415) took 1860 ms. Trace:
I20260709 14:53:46.285312  8050 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.285898  2681 rpcz_store.cc:276] 0709 14:53:44.429229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.435806 (+  6577us) service_pool.cc:224] Handling call
0709 14:53:44.436208 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:44.467424 (+ 31216us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:44.467872 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 14:53:44.759040 (+291168us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:44.776504 (+ 17464us) log.cc:844] Serialized 26 byte log entry
0709 14:53:44.776759 (+   255us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:53:44.777157 (+   398us) log.cc:844] Serialized 14 byte log entry
0709 14:53:44.777410 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:44.777529 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.279228 (+1501699us) raft_consensus.cc:1685] finished
0709 14:53:46.279264 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.279489 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33798,"prepare.queue_time_us":6500,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":98,"raft.queue_time_us":60995,"raft.run_cpu_time_us":493,"raft.run_wall_time_us":14308,"thread_start_us":270053,"threads_started":1,"wal-append.queue_time_us":689805}
W20260709 14:53:46.288383  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.010s	sys 0.018s
I20260709 14:53:46.257597  7984 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.180783  8051 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.265663  7974 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:46.267971  7932 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:46.305437  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.019s	sys 0.034s
W20260709 14:53:46.297205  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.224157  8027 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:46.282081  7956 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.617s	user 0.004s	sys 0.009s
I20260709 14:53:46.267786  7784 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:46.274945  7690 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:46.291157  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1423) took 1199 ms. Trace:
W20260709 14:53:46.309336  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.020s	sys 0.035s
W20260709 14:53:46.339198  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 667870us
I20260709 14:53:46.301023  2608 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:53:46.312608  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.012s	sys 0.029s
W20260709 14:53:46.344726  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.012s	sys 0.029s
I20260709 14:53:46.301789  7974 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:46.338608  2683 rpcz_store.cc:276] 0709 14:53:45.091572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.215305 (+123733us) service_pool.cc:224] Handling call
0709 14:53:45.241349 (+ 26044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:45.241670 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:45.297459 (+ 55789us) log.cc:844] Serialized 14 byte log entry
0709 14:53:46.290615 (+993156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:46.290702 (+    87us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:46.290781 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.290903 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.291075 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26539,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":58,"raft.queue_time_us":3615,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":6478,"thread_start_us":992473,"threads_started":1}
I20260709 14:53:46.347352  7032 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.316951  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.351522  7032 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.352946  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.311527  8050 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.355681  2753 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:46.361531  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51929.7us
I20260709 14:53:46.282181  8035 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:46.314311  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47300.6us
I20260709 14:53:46.294767  8007 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:46.299095  7949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.031s	sys 0.063s
I20260709 14:53:46.301437  7979 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.344316  7958 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:46.397399  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.400316  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.160701  2603 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 11.
I20260709 14:53:46.303738  7937 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:46.312350  7916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.017s	sys 0.022s
W20260709 14:53:46.355604  8056 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 14:53:46.355949  8066 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:46.379467  8061 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 14:53:46.381148  8060 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 14:53:46.292080  7933 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.316100  7932 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.384049  7904 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.385429  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1308) took 1760 ms. Trace:
I20260709 14:53:46.413062  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.424937  3545 rpcz_store.cc:276] 0709 14:53:44.624818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.664659 (+1039841us) service_pool.cc:224] Handling call
0709 14:53:45.857253 (+192594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:45.954111 (+ 96858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.385206 (+431095us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400550350. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.385360 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441972608}
I20260709 14:53:46.402482  8040 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:46.430296  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1614) took 1132 ms. Trace:
W20260709 14:53:46.387897  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 420476us
I20260709 14:53:46.430783  2604 rpcz_store.cc:276] 0709 14:53:45.297858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.632372 (+334514us) service_pool.cc:224] Handling call
0709 14:53:46.430100 (+797728us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.430230 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469203712}
W20260709 14:53:46.431540  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.432199  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:46.432951  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.429129  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:46.354961  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:46.312915  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.017s	sys 0.011s
W20260709 14:53:46.436425  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.017s	sys 0.011s
W20260709 14:53:46.395850  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.014s	sys 0.013s
I20260709 14:53:46.327781  7943 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 14:53:46.441753  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.409958  8062 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:46.405558  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1613) took 1109 ms. Trace:
W20260709 14:53:46.396075  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79294.8us
I20260709 14:53:46.357853  7974 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:46.446300  2603 rpcz_store.cc:276] 0709 14:53:45.297558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.014061 (+716503us) service_pool.cc:224] Handling call
0709 14:53:46.405457 (+391396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.417351  7964 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:46.406558  2684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.006s	sys 0.010s
I20260709 14:53:46.354766  7032 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:46.409502  3547 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:46.394809  7949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.032s	sys 0.064s
I20260709 14:53:46.355368  8054 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:46.411262  3536 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:46.327502  7956 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.359289  7784 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:46.388330  7905 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:46.470358  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.435274  7929 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:46.407145  7916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.018s	sys 0.022s
I20260709 14:53:46.349078  7912 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.390895  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:46.454496  7717 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:46.308314  8051 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.433980  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.451890  8040 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.434676  3541 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:46.482049  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1309) took 1852 ms. Trace:
I20260709 14:53:46.482424  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1303) took 1923 ms. Trace:
I20260709 14:53:46.483006  3540 rpcz_store.cc:276] 0709 14:53:44.558706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.637733 (+1079027us) service_pool.cc:224] Handling call
0709 14:53:45.638075 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.050247 (+412172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.482103 (+431856us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54002dc3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:46.482341 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910156160}
I20260709 14:53:46.429039  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1533) took 1191 ms. Trace:
W20260709 14:53:46.305362  8030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.005s	sys 0.048s
I20260709 14:53:46.484589  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:46.485878  3540 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 14:53:46.387682  8059 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 14:53:46.443758  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 966) took 1191 ms. Trace:
I20260709 14:53:46.413398  8053 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:46.449440  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.450490  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:46.497310  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47991.6us
W20260709 14:53:46.455729  2684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.007s	sys 0.010s
I20260709 14:53:46.499850  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.500751  2752 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:46.501744  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.402482  7979 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.407265  7958 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:46.436887  7997 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.744s	user 0.015s	sys 0.000s
I20260709 14:53:46.462330  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:46.437340  7876 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:46.467486  2681 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 14:53:46.467486  2760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.000s	sys 0.012s
I20260709 14:53:46.475611  8006 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:46.511734  2760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.001s	sys 0.012s
I20260709 14:53:46.436234  7668 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.471017  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:46.414209  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.017s	sys 0.017s
I20260709 14:53:46.451728  7991 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:46.521492  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1481) took 1837 ms. Trace:
I20260709 14:53:46.423280  7657 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 14:53:46.472872  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.473493  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:46.474406  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83677.6us
I20260709 14:53:46.534022  2756 raft_consensus.cc:2368] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:46.489305  8008 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:46.475526  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:46.437315  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.015s	sys 0.013s
W20260709 14:53:46.478600  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44685.5us
I20260709 14:53:46.543495  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.437563  2608 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:46.545096  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.420238  8066 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.484400  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1306) took 1916 ms. Trace:
I20260709 14:53:46.486328  2685 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:46.408036  7978 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:46.546675  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.484656  2607 rpcz_store.cc:276] 0709 14:53:45.237827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.261834 (+ 24007us) service_pool.cc:224] Handling call
0709 14:53:45.283901 (+ 22067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:45.284193 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:46.169963 (+885770us) log.cc:844] Serialized 14 byte log entry
0709 14:53:46.428335 (+258372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:46.428417 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:46.428522 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.428638 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.428953 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3012,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7984,"thread_start_us":1104407,"threads_started":2}
I20260709 14:53:46.495185  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.496778  2753 rpcz_store.cc:276] 0709 14:53:45.239482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.272165 (+ 32683us) service_pool.cc:224] Handling call
0709 14:53:46.430657 (+1158492us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.430802 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403917568}
W20260709 14:53:46.485122  8023 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.003s
I20260709 14:53:46.504170  7905 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:53:46.515911  7989 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.899s	user 0.013s	sys 0.001s
W20260709 14:53:46.488796  8030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.006s	sys 0.049s
I20260709 14:53:46.510973  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.512387  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.513885  2612 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 14:53:46.476511  7717 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:46.515491  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.017s	sys 0.017s
I20260709 14:53:46.515498  2679 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:46.573511  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.06585e+06us
I20260709 14:53:46.517252  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:46.518163  8040 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.458140  8062 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.488930  7439 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:46.535254  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.424775  8034 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 14:53:46.531836  8070 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 14:53:46.482586  3547 rpcz_store.cc:276] 0709 14:53:44.630116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.866698 (+1236582us) service_pool.cc:224] Handling call
0709 14:53:45.867386 (+   688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.455357 (+587971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.481758 (+ 26401us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54003200d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:46.481978 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246122752}
I20260709 14:53:46.567239  7945 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:46.546249  3537 rpcz_store.cc:276] 0709 14:53:44.567720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.651286 (+1083566us) service_pool.cc:224] Handling call
0709 14:53:46.008726 (+357440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.016817 (+  8091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.484176 (+467359us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400224cd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.484338 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":820805888}
I20260709 14:53:46.558123  2758 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:46.478314  7912 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.558518  7984 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 14:53:46.560909  8076 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 14:53:46.566659  2682 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:53:46.568773  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.570567  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.574819  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.576377  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.568281  7989 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.572594  7784 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:46.647588  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.557677  7991 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:46.594869  7909 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.007s
W20260709 14:53:46.521598  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.512159  8065 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 14:53:46.694948  7991 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.695941  8065 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:46.696404  8065 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.697551  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.697427  8065 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.560245  7905 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:46.699381  7905 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:46.699751  8077 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.228s	user 0.003s	sys 0.000s
I20260709 14:53:46.701339  8030 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:46.701772  8051 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:46.682334  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1312) took 1876 ms (client timeout 1593 ms). Trace:
W20260709 14:53:46.702888  8093 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 14:53:46.703617  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 133213us
I20260709 14:53:46.705158  7958 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:46.705158  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.707840  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1427) took 1512 ms. Trace:
I20260709 14:53:46.708338  2684 rpcz_store.cc:276] 0709 14:53:45.195254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.235470 (+ 40216us) service_pool.cc:224] Handling call
0709 14:53:45.235902 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:45.236909 (+  1007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:45.236942 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:45.259918 (+ 22976us) log.cc:844] Serialized 26 byte log entry
0709 14:53:46.498239 (+1238321us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:46.498298 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.498388 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.707618 (+209230us) raft_consensus.cc:1685] finished
0709 14:53:46.707644 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.707775 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4293,"raft.queue_time_us":27836,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4928,"thread_start_us":1108176,"threads_started":1,"wal-append.queue_time_us":1316419}
I20260709 14:53:46.710459  7912 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.711436  7956 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.703186  7916 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:46.712688  7916 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:46.713335  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.558601  7705 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 14:53:46.565838  7828 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 14:53:46.564473  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:46.713758  7916 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.712688  8034 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:53:46.715296  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.715667  8034 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.716013  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:53:46.716673  8034 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.717695  8007 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:46.718178  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:46.568429  8023 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.718915  2684 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:46.719282  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.719846  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1425) took 1560 ms. Trace:
I20260709 14:53:46.720094  2612 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:46.720830  8023 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:46.721436  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1172) took 2003 ms (client timeout 1969 ms). Trace:
W20260709 14:53:46.721858  3536 rpcz_store.cc:269] 0709 14:53:44.717522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.971110 (+253588us) service_pool.cc:224] Handling call
0709 14:53:46.721219 (+1750109us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.721360 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3307865344}
I20260709 14:53:46.718227  8007 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:46.723160  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:46.723405  7960 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:46.723812  7960 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.723896  3536 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:53:46.725063  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:46.725248  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.727780  7690 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:46.728451  7690 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:46.729802  7994 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:46.561578  7978 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.692366  7933 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.731963  7952 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.734762  7876 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:46.735509  7876 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.720257  2678 rpcz_store.cc:276] 0709 14:53:45.161271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.215625 (+ 54354us) service_pool.cc:224] Handling call
0709 14:53:45.290447 (+ 74822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:45.298474 (+  8027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:45.298537 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:45.373896 (+ 75359us) log.cc:844] Serialized 26 byte log entry
0709 14:53:46.267050 (+893154us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:46.267117 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.267251 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.629804 (+362553us) raft_consensus.cc:1685] finished
0709 14:53:46.629834 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.719574 (+ 89740us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b5400445550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.719785 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7479,"raft.queue_time_us":21317,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10311,"spinlock_wait_cycles":57851776,"thread_start_us":878764,"threads_started":1,"wal-append.queue_time_us":948174}
I20260709 14:53:46.736794  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.737483  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:46.737854  7943 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:46.738246  8033 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:46.738296  7943 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:46.738076  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:46.726491  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:46.720587  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1616) took 1347 ms. Trace:
I20260709 14:53:46.738819  8033 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:46.627679  8083 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 14:53:46.522207  3542 rpcz_store.cc:276] 0709 14:53:44.683611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.945118 (+1261507us) service_pool.cc:224] Handling call
0709 14:53:46.001808 (+ 56690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:46.002322 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:46.002369 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:46.037948 (+ 35579us) log.cc:844] Serialized 26 byte log entry
0709 14:53:46.458429 (+420481us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:46.458499 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.458589 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.521228 (+ 62639us) raft_consensus.cc:1685] finished
0709 14:53:46.521253 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.521422 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1913,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":12136,"thread_start_us":339012,"threads_started":1,"wal-append.queue_time_us":454138}
I20260709 14:53:46.745632  8033 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.739440  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:46.626915  8075 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 14:53:46.748534  2755 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
W20260709 14:53:46.628412  8071 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 14:53:46.750164  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1484) took 1218 ms. Trace:
I20260709 14:53:46.751397  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1534) took 1492 ms. Trace:
I20260709 14:53:46.751829  2605 rpcz_store.cc:276] 0709 14:53:45.259503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.273822 (+ 14319us) service_pool.cc:224] Handling call
0709 14:53:45.296939 (+ 23117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:45.328330 (+ 31391us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:45.328365 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:45.378409 (+ 50044us) log.cc:844] Serialized 26 byte log entry
0709 14:53:46.289010 (+910601us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:46.289079 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.289191 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.751183 (+461992us) raft_consensus.cc:1685] finished
0709 14:53:46.751207 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.751338 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":297538,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":787866,"threads_started":1,"wal-append.queue_time_us":1070988}
I20260709 14:53:46.753744  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.754556  2605 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:46.626912  8082 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 14:53:46.750643  2755 rpcz_store.cc:276] 0709 14:53:45.531290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.619275 (+ 87985us) service_pool.cc:224] Handling call
0709 14:53:45.619763 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.748990 (+1129227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.749890 (+   900us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:46.750094 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595361024}
I20260709 14:53:46.755851  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.739928  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:46.606208  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:46.742105  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:46.742524  2608 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:46.759017  2607 raft_consensus.cc:2368] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
W20260709 14:53:46.749382  7948 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.160s	user 0.010s	sys 0.004s
I20260709 14:53:46.749956  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1307) took 2165 ms. Trace:
I20260709 14:53:46.744355  2612 rpcz_store.cc:276] 0709 14:53:45.373665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.718995 (+1345330us) service_pool.cc:224] Handling call
0709 14:53:46.720514 (+  1519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.578558  8027 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.727828  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1173) took 1985 ms. Trace:
I20260709 14:53:46.739252  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:46.764706  3543 rpcz_store.cc:276] 0709 14:53:44.584944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.657597 (+1072653us) service_pool.cc:224] Handling call
0709 14:53:45.658096 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:45.660157 (+  2061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:45.660208 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:45.660820 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 14:53:46.436918 (+776098us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:46.436969 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.437055 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.749706 (+312651us) raft_consensus.cc:1685] finished
0709 14:53:46.749731 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.749892 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":775649,"threads_started":1,"wal-append.queue_time_us":781851}
I20260709 14:53:46.596863  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:46.659678  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113113us
I20260709 14:53:46.777170  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:46.768101  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.762502  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:46.762722  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65297us
I20260709 14:53:46.766804  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:46.765033  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:46.703231  3541 rpcz_store.cc:269] 0709 14:53:44.806478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:44.823480 (+ 17002us) service_pool.cc:224] Handling call
0709 14:53:46.682095 (+1858615us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004b55d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.682240 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756209408}
I20260709 14:53:46.506286  3540 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:46.579260  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:53:46.775982  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:46.631389  8089 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 14:53:46.631372  7944 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.003s
W20260709 14:53:46.674994  7970 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.770s	user 0.015s	sys 0.000s
I20260709 14:53:46.634205  8066 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:46.776227  3545 rpcz_store.cc:276] 0709 14:53:44.742871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.437896 (+1695025us) service_pool.cc:224] Handling call
0709 14:53:46.456787 (+ 18891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.539554 (+ 82767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:46.539589 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:46.539642 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:46.539680 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.726689 (+187009us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b1800180dc0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419
0709 14:53:46.727563 (+   874us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54004b1ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:46.727622 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.727763 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":12476,"spinlock_wait_cycles":225603712}
I20260709 14:53:46.633800  8036 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.630306  8035 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.675606  8078 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.661988  7977 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 14:53:46.633983  8063 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 14:53:46.779443  3542 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:46.680824  7932 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.780321  2755 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:46.780475  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:46.806174  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.693148  7972 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:46.876422  8023 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:46.877200  8023 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:46.877717  8023 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.865021  8078 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.739060  7943 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:46.742869  7975 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.780992  2603 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:46.663061  7598 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.649s	user 0.016s	sys 0.000s
I20260709 14:53:46.807042  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.888141  2757 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:46.743714  8065 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:46.784271  2753 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:46.635497  8062 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.588743  8081 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:46.797338  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:46.798753  2606 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:46.905210  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1605) took 1623 ms. Trace:
I20260709 14:53:46.905732  2606 rpcz_store.cc:276] 0709 14:53:45.281778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.798019 (+1516241us) service_pool.cc:224] Handling call
0709 14:53:46.798466 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:46.904751 (+106285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.905131 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:46.801517  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63873.5us
W20260709 14:53:46.814253  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1609) took 1523 ms (client timeout 1979 ms). Trace:
I20260709 14:53:46.800380  2612 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 14:53:46.850302  8051 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:46.586417  7032 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.206s
I20260709 14:53:46.840317  7977 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:46.826516  2605 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:46.828212  2752 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 14:53:46.833995  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:46.921327  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1535) took 1647 ms. Trace:
W20260709 14:53:46.921926  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 88091.3us
I20260709 14:53:46.922276  8076 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:46.922860  8076 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:46.923291  8076 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:46.813665  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:46.924410  3537 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:46.900525  8048 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 14:53:46.794003  2609 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:46.787710  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:46.930305  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142754us
I20260709 14:53:46.779050  2758 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:46.838950  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:46.802552  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:46.694653  7909 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:46.795706  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83296.7us
I20260709 14:53:46.934427  3547 raft_consensus.cc:2376] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13. Re-sending same reply.
I20260709 14:53:46.850147  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:46.936623  3545 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:46.818343  2608 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:46.938261  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1610) took 1647 ms. Trace:
I20260709 14:53:46.938880  2608 rpcz_store.cc:276] 0709 14:53:45.290723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.817693 (+1526970us) service_pool.cc:224] Handling call
0709 14:53:46.818097 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.937856 (+119759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.938179 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.935676  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.817080  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:46.817721  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 242954us
W20260709 14:53:46.817574  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 282429us
I20260709 14:53:46.940482  3541 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:53:46.854086  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 93216.6us
I20260709 14:53:46.854223  8103 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.607000  7717 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.587899  7929 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.810138  7944 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.801676  7956 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:53:46.810676  7705 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:46.790714  7912 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 14:53:46.798514  7834 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.898s	user 0.012s	sys 0.001s
I20260709 14:53:46.806852  7974 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:46.866714  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86331us
I20260709 14:53:46.868585  2607 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 14:53:46.841161  7960 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.873627  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:46.816501  8098 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:53:46.798902  8033 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:46.816946  8001 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.003s
I20260709 14:53:46.775574  7948 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.975361  2754 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 14:53:46.909267  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1606) took 1627 ms. Trace:
I20260709 14:53:46.975909  2612 rpcz_store.cc:276] 0709 14:53:45.282091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.799720 (+1517629us) service_pool.cc:224] Handling call
0709 14:53:46.800102 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:46.908725 (+108623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.909158 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.976971  7948 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:46.977721  3538 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:46.978247  7657 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.978892  7657 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.981181  7978 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.983882  7784 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.984747  7784 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:46.960150  8051 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:46.990520  8098 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:46.960811  7916 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:53:46.991447  7979 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.839731  7439 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:46.993556  7439 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:46.907168  2611 rpcz_store.cc:269] 0709 14:53:45.290410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.627294 (+336884us) service_pool.cc:224] Handling call
0709 14:53:46.814004 (+1186710us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54001ead54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:46.814169 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518459136}
I20260709 14:53:46.951033  7932 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:46.995445  7932 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:46.996996  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1628) took 1112 ms. Trace:
I20260709 14:53:46.779032  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:46.998195  7932 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:46.864841  7984 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:46.810683  2610 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:47.000849  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.793447  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:46.971810  7717 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:46.921790  2605 rpcz_store.cc:276] 0709 14:53:45.274135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.801578 (+1527443us) service_pool.cc:224] Handling call
0709 14:53:46.826310 (+ 24732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.920880 (+ 94570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.921247 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.935918  3411 leader_election.cc:400] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:47.004798  3411 leader_election.cc:403] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 14:53:46.816596  7952 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
W20260709 14:53:47.005417  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 69644.5us
I20260709 14:53:47.005617  2605 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:47.006476  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 938) took 1121 ms. Trace:
I20260709 14:53:46.903129  8008 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:46.947962  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.997697  2611 rpcz_store.cc:276] 0709 14:53:45.884500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.995390 (+1110890us) service_pool.cc:224] Handling call
0709 14:53:46.995912 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.996411 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:46.996458 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:46.996530 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:46.996597 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.996689 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.996899 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2732}
I20260709 14:53:46.951812  3537 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:46.911890  2684 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:46.959606  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.927444  8035 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.960808  3540 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:46.962895  8078 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:46.964098  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1422) took 1903 ms. Trace:
I20260709 14:53:46.899255  7598 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.007076  7948 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:46.991822  7916 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:47.028703  7916 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:47.029024  7784 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.029435  7598 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.006883  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.030453  2604 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17.
W20260709 14:53:47.031994  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:46.900166  7690 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:46.965322  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:46.962662  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55305us
I20260709 14:53:46.961946  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:46.967262  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1608) took 1684 ms. Trace:
I20260709 14:53:47.033856  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:46.967301  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:47.034104  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:47.034613  2680 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:46.957224  7972 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:46.944461  7958 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:46.943490  7945 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:46.930992  8081 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:46.973016  8086 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 14:53:46.803486  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:46.930856  8007 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.038465  7690 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.001041  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1619) took 1419 ms. Trace:
I20260709 14:53:46.955452  8064 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:47.007082  3547 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:47.094874  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1174) took 2221 ms. Trace:
I20260709 14:53:47.095445  3547 rpcz_store.cc:276] 0709 14:53:44.873130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.951359 (+2078229us) service_pool.cc:224] Handling call
0709 14:53:46.966926 (+ 15567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.094383 (+127457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.094730 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:46.956861  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:46.949344  2603 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 14:53:47.006804  2605 rpcz_store.cc:276] 0709 14:53:45.884821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.005008 (+1120187us) service_pool.cc:224] Handling call
0709 14:53:47.005407 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.006193 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.006413 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:47.097422  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1618) took 1595 ms. Trace:
I20260709 14:53:47.097787  2603 rpcz_store.cc:276] 0709 14:53:45.501630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.923934 (+1422304us) service_pool.cc:224] Handling call
0709 14:53:46.949086 (+ 25152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:47.097073 (+147987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.097367 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:47.008858  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61023.6us
I20260709 14:53:46.963798  7705 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:47.011281  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:47.099001  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:47.034075  2607 rpcz_store.cc:276] 0709 14:53:45.282615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.815717 (+1533102us) service_pool.cc:224] Handling call
0709 14:53:46.867221 (+ 51504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.966994 (+ 99773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.967201 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:47.099468  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 88347.9us
W20260709 14:53:47.035542  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68335.6us
I20260709 14:53:47.101171  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.101464  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:47.103404  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146651us
I20260709 14:53:46.963197  8062 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 14:53:46.940093  3542 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:47.040514  8030 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:53:47.039171  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.036510  2686 rpcz_store.cc:276] 0709 14:53:45.060356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.208518 (+148162us) service_pool.cc:224] Handling call
0709 14:53:45.229587 (+ 21069us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:45.230085 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:45.230117 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:45.230681 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 14:53:45.644093 (+413412us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b4800541158. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:53:46.092977 (+448884us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:46.093043 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.093152 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:46.957585 (+864433us) raft_consensus.cc:1685] finished
0709 14:53:46.957619 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.963994 (+  6375us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17410,"raft.run_cpu_time_us":3657,"raft.run_wall_time_us":381912,"spinlock_wait_cycles":11725696,"thread_start_us":861758,"threads_started":1,"wal-append.queue_time_us":1128564}
W20260709 14:53:47.060560  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100316us
I20260709 14:53:47.062415  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:47.072013  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 268627us
W20260709 14:53:47.134629  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 72283.6us
I20260709 14:53:46.991860  8098 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.131435  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.008570  7974 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.008925  8054 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 14:53:47.082935  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.075848  7784 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:46.966105  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1625) took 1082 ms. Trace:
I20260709 14:53:47.203289  2608 rpcz_store.cc:276] 0709 14:53:45.883626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.947745 (+1064119us) service_pool.cc:224] Handling call
0709 14:53:46.948149 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:46.965549 (+ 17400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:46.965607 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:46.965679 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:46.965728 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:46.965824 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:46.966042 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14021,"raft.queue_time_us":17861,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":37938}
I20260709 14:53:47.116199  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:46.952797  8103 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.095203  7994 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:47.041453  8035 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:47.125612  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:47.126778  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.213960  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.060686  7958 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:47.136408  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.232758  3542 raft_consensus.cc:2376] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14. Re-sending same reply.
I20260709 14:53:47.136866  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 967) took 1897 ms. Trace:
W20260709 14:53:47.178083  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:47.184422  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50577.7us
I20260709 14:53:47.071748  8100 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:47.185652  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54292us
I20260709 14:53:47.196637  3547 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:47.239414  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1493) took 1966 ms. Trace:
I20260709 14:53:47.239806  3547 rpcz_store.cc:276] 0709 14:53:45.273344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.119651 (+1846307us) service_pool.cc:224] Handling call
0709 14:53:47.196212 (+ 76561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.239073 (+ 42861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.239332 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:47.075599  2610 rpcz_store.cc:276] 0709 14:53:45.581755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.948347 (+366592us) service_pool.cc:224] Handling call
0709 14:53:47.000768 (+1052421us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.000955 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979590016}
I20260709 14:53:47.075564  7598 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:53:46.963539  7834 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.060921  8078 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:47.137028  8090 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:47.115787  8064 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.126868  7912 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:47.007941  8008 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:46.991441  8051 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:47.242302  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126250us
I20260709 14:53:47.237682  7668 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.280257  3412 leader_election.cc:400] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14.
I20260709 14:53:47.280995  3540 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.066941  7977 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.187178  7948 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:47.185003  7997 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:47.233587  2754 rpcz_store.cc:276] 0709 14:53:45.239791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.276337 (+ 36546us) service_pool.cc:224] Handling call
0709 14:53:47.136637 (+1860300us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.136793 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3646295424}
I20260709 14:53:47.240690  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.038872  7876 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.252105  2755 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.185320  7968 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:47.255692  8078 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:47.202644  8066 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 14:53:47.278347  8030 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:47.208412  7705 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.266245  2605 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:47.221356  7979 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:47.209785  7032 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:47.276958  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1178) took 1877 ms (client timeout 1525 ms). Trace:
W20260709 14:53:47.342446  3538 rpcz_store.cc:269] 0709 14:53:45.399891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:45.600380 (+200489us) service_pool.cc:224] Handling call
0709 14:53:47.276688 (+1676308us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.276875 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079661952}
I20260709 14:53:47.277079  3546 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:47.235697  7958 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:47.301008  8100 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:47.278491  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:47.346793  2610 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 9 FOLLOWER]: Advancing to term 10
I20260709 14:53:47.071697  7972 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:47.253078  8093 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:53:47.287047  2612 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:47.292311  3542 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 14:53:47.292434  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51882.2us
I20260709 14:53:47.296686  8121 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:47.294312  7912 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:47.221889  7690 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:47.231042  8099 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 14:53:47.276553  8090 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:47.233808  7994 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.138005  7799 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 14:53:47.283622  7598 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.138684  7966 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.305939  7968 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.279321  8081 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.307283  8078 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.303154  8102 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 14:53:47.355499  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1179) took 1855 ms. Trace:
I20260709 14:53:47.355553  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.299876  7668 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:47.359652  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53321.4us
I20260709 14:53:47.390221  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:47.406409  3542 rpcz_store.cc:276] 0709 14:53:45.500238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.264851 (+1764613us) service_pool.cc:224] Handling call
0709 14:53:47.292052 (+ 27201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:47.355174 (+ 63122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.355429 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:47.387187  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.408458  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1175) took 2535 ms. Trace:
I20260709 14:53:47.408950  3537 rpcz_store.cc:276] 0709 14:53:44.873527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.027827 (+2154300us) service_pool.cc:224] Handling call
0709 14:53:47.028261 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:47.028675 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:47.028708 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:47.029281 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 14:53:47.203011 (+173730us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:47.203067 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.205725 (+  2658us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:47.408201 (+202476us) raft_consensus.cc:1685] finished
0709 14:53:47.408229 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:47.408395 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2068,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":27930,"thread_start_us":173244,"threads_started":1,"wal-append.queue_time_us":263599}
I20260709 14:53:47.411217  2679 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:47.411871  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.412750  3537 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 14:53:47.413568  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.414690  2679 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:47.385555  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.424142  2610 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 10.
I20260709 14:53:47.339951  7979 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:47.432407  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:47.351965  7905 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.003s
I20260709 14:53:47.461352  7905 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.463500  3538 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 14:53:47.435268  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46222.6us
I20260709 14:53:47.464389  7994 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.465299  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.464013  7905 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.465852  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.376164  7944 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.466447  7905 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:47.272082  7945 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.434275  7598 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.460525  7968 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.437273  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.367063  8030 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.468502  8030 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.469156  7598 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:47.429386  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73902.3us
W20260709 14:53:47.365972  8113 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 14:53:47.382488  8023 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 14:53:47.448164  2679 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:47.455260  7943 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.004s
I20260709 14:53:47.456475  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.456773  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.477246  2755 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:47.477557  2686 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.477550  2751 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:47.375756  8007 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.460433  7668 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.364210  7690 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:47.440382  7984 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:47.466166  3542 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:47.426352  8090 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.468755  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.471205  3540 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 14:53:47.501852  3540 pending_rounds.cc:85] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 7
I20260709 14:53:47.502151  3540 pending_rounds.cc:107] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 14:53:47.502609  3540 raft_consensus.cc:2889] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:47.465324  3536 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:47.360824  8104 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 14:53:47.353506  8100 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:47.445523  3537 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:53:47.471215  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.441942  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:47.356927  7909 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.474557  7905 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.474128  8033 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.433897  7966 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.364322  8035 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.382962  7834 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.384266  7989 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.381426  8103 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.519529  8030 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.519810  7905 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.488986  7948 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:47.526486  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.363831  7912 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.527376  2684 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:47.505520  7799 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.528741  7799 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.476079  7944 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 14:53:47.511660  7984 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:47.531859  7984 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.535281  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.535972  7972 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.536154  2678 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:53:47.352303  7705 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:47.500430  7994 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 14:53:47.538340  7994 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:47.538959  7994 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:47.542289  7905 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.484549  8023 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.545384  8023 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.498477  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.549548  2677 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.507296  2610 raft_consensus.cc:2376] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17. Re-sending same reply.
I20260709 14:53:47.510460  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.512934  7979 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.383260  8121 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.496459  7943 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.492298  2679 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:47.505205  8100 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.575851  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.575770  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:47.576399  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.578735  3541 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:47.580171  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.581585  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:47.582026  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.582587  2608 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:47.583200  2679 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:47.614359  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:47.614893  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.615381  2610 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:47.615499  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.614946  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.615990  2686 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:47.616267  3542 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
I20260709 14:53:47.616364  2684 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:47.527035  8120 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:47.523974  7966 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.540206  7994 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.627214  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.628312  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.630132  3541 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:47.631176  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.632344  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.633021  3541 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:47.633852  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.526870  7690 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.524519  8033 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:47.617728  7717 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.000s
I20260709 14:53:47.719458  2677 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:47.691090  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:47.634620  3541 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:47.617923  8110 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.691570  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.692091  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:47.707883  7916 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.001s
I20260709 14:53:47.751740  2608 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:47.621327  8120 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:47.713364  7994 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:47.711833  7975 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.810s	user 0.013s	sys 0.002s
I20260709 14:53:47.625578  7997 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.627072  7979 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:47.626803  8076 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.003s
I20260709 14:53:47.760450  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.635169  7799 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.767643  2608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:53:47.641366  7943 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.770390  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.772953  3542 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:47.779381  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.779933  3542 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:47.771425  8078 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.782655  8078 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.785876  7958 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.786605  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1176) took 2387 ms. Trace:
I20260709 14:53:47.775481  8076 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.787158  3545 rpcz_store.cc:276] 0709 14:53:45.399364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.132271 (+1732907us) service_pool.cc:224] Handling call
0709 14:53:47.206446 (+ 74175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:47.296510 (+ 90064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:47.296563 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:47.297348 (+   785us) log.cc:844] Serialized 26 byte log entry
0709 14:53:47.710140 (+412792us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:47.710208 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.710318 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:47.786316 (+ 75998us) raft_consensus.cc:1685] finished
0709 14:53:47.786345 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:47.786527 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28109,"raft.queue_time_us":28839,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":12022,"thread_start_us":413851,"threads_started":2,"wal-append.queue_time_us":441918}
I20260709 14:53:47.789247  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.791508  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.792160  2677 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:47.783313  8121 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.800236  2758 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:47.789868  3545 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:47.800258  2677 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:53:47.794924  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.802177  3542 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 10: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:47.711201  8090 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.634498  7948 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.764504  7916 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:47.809477  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 313689us
I20260709 14:53:47.790601  8076 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.760361  8120 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.813266  8120 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:47.815150  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:47.815706  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:47.816238  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.817838  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:47.818028  8106 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.818764  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:47.818640  8106 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:47.820549  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 685551us
I20260709 14:53:47.815806  7799 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:47.823050  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:47.815820  7611 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.824530  7611 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:47.827777  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:47.827348  7705 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:47.830039  7705 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:47.830490  7705 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:47.830792  8112 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.544s	user 0.000s	sys 0.001s
I20260709 14:53:47.831459  7705 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.832836  7705 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:47.835948  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1496) took 1389 ms. Trace:
I20260709 14:53:47.836341  2684 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.836498  3538 rpcz_store.cc:276] 0709 14:53:46.447243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.343380 (+896137us) service_pool.cc:224] Handling call
0709 14:53:47.363372 (+ 19992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.464015 (+100643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.835664 (+371649us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54002bb454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:47.835884 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229228416}
I20260709 14:53:47.838814  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:47.839572  3538 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:47.755647  8110 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.843968  2679 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:47.846750  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:47.794518  7984 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:47.852517  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:47.853250  3545 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:47.853309  2576 leader_election.cc:400] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
I20260709 14:53:47.776011  7997 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:47.853978  2576 leader_election.cc:403] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 14:53:47.848129  8106 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.857012  2679 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.857733  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:47.858531  3538 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:47.859356  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1499) took 1392 ms. Trace:
I20260709 14:53:47.859711  3538 rpcz_store.cc:276] 0709 14:53:46.467060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.840280 (+1373220us) service_pool.cc:224] Handling call
0709 14:53:47.858328 (+ 18048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.859085 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.859298 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:47.860824  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.866521  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.867640  2679 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 14:53:47.867403  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:47.868674  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.869292  2679 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:47.870052  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.835839  7611 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.873950  2684 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.878599  2679 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:47.857265  7984 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:47.879370  2610 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:53:47.968338  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 951) took 1712 ms. Trace:
I20260709 14:53:47.969161  2610 rpcz_store.cc:276] 0709 14:53:46.255797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.779305 (+1523508us) service_pool.cc:224] Handling call
0709 14:53:47.779743 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.966389 (+186646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.967942 (+  1553us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b5400128254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:47.968238 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227858176}
I20260709 14:53:47.851912  7968 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:47.975705  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 978) took 1146 ms. Trace:
I20260709 14:53:47.976393  2755 rpcz_store.cc:276] 0709 14:53:46.829188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.873212 (+ 44024us) service_pool.cc:224] Handling call
0709 14:53:47.975426 (+1102214us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.975608 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648609152}
I20260709 14:53:47.977916  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:47.978753  2755 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:47.886883  3538 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:47.893235  8090 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.986644  7657 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:47.893445  8098 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:53:47.989619  8115 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.618s	user 0.001s	sys 0.003s
I20260709 14:53:47.896257  3541 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.991772  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1500) took 1310 ms. Trace:
I20260709 14:53:47.991839  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1626) took 2107 ms. Trace:
I20260709 14:53:47.992341  3541 rpcz_store.cc:276] 0709 14:53:46.681785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.895573 (+1213788us) service_pool.cc:224] Handling call
0709 14:53:47.896038 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.991377 (+ 95339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.991690 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:47.992571  2612 rpcz_store.cc:276] 0709 14:53:45.883929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.976817 (+1092888us) service_pool.cc:224] Handling call
0709 14:53:46.977252 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.354531 (+377279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.991543 (+637012us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400296dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:47.991757 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711745280}
I20260709 14:53:47.899566  8033 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.877228  7949 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:47.901388  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1498) took 1442 ms. Trace:
I20260709 14:53:47.902110  3543 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:47.995250  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1630) took 1753 ms (client timeout 2045 ms). Trace:
I20260709 14:53:47.996181  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1314) took 3131 ms. Trace:
W20260709 14:53:47.995766  2609 rpcz_store.cc:269] 0709 14:53:46.241866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.264348 (+ 22482us) service_pool.cc:224] Handling call
0709 14:53:47.995032 (+1730684us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54002fe9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.995186 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180441984}
I20260709 14:53:47.996923  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1320) took 1528 ms. Trace:
I20260709 14:53:47.997660  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 970) took 1073 ms. Trace:
I20260709 14:53:47.998148  3536 rpcz_store.cc:276] 0709 14:53:46.458985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.724795 (+265810us) service_pool.cc:224] Handling call
0709 14:53:47.901136 (+1176341us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b5400200154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.901321 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700565248}
I20260709 14:53:47.998073  2682 rpcz_store.cc:276] 0709 14:53:46.923985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.924121 (+   136us) service_pool.cc:224] Handling call
0709 14:53:46.924542 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:46.924965 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:46.925000 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:46.943024 (+ 18024us) log.cc:844] Serialized 26 byte log entry
0709 14:53:47.199456 (+256432us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:47.199509 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.199602 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:47.813497 (+613895us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b4800541158. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:53:47.982376 (+168879us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:47.997389 (+ 15013us) raft_consensus.cc:1685] finished
0709 14:53:47.997416 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:47.997597 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2632,"raft.run_cpu_time_us":12346,"raft.run_wall_time_us":883420,"spinlock_wait_cycles":55308288,"thread_start_us":192032,"threads_started":1,"wal-append.queue_time_us":343015}
I20260709 14:53:47.999243  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:47.999969  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:48.000351  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.000864  3536 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:48.000977  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.001508  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:48.001962  3536 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:48.002293  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.003403  2611 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:47.998103  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:48.003950  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.004576  2681 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:53:48.004623  2603 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:48.005612  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:47.904266  8110 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:47.880728  8065 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.967s	user 0.010s	sys 0.007s
I20260709 14:53:47.906384  7997 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:47.907668  8100 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:47.910739  7916 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:47.911231  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.038796  2684 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:47.916000  3545 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:47.917604  3412 leader_election.cc:400] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17.
I20260709 14:53:47.918331  8106 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 14:53:47.906586  3542 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:47.921629  7960 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:47.923784  8092 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.924486  7932 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.927152  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:47.928272  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 727577us
I20260709 14:53:47.930800  8102 consensus_queue.cc:1243] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:53:47.931269  8066 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:47.925520  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1311) took 3199 ms. Trace:
I20260709 14:53:47.936611  7976 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:47.932170  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 716939us
I20260709 14:53:48.075037  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.937829  7958 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:47.927634  2608 raft_consensus.cc:606] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.077164  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.077443  2608 raft_consensus.cc:614] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:53:47.940769  2757 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 14:53:47.942411  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:47.942873  8062 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.946777  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.079078  2679 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 14:53:48.078999  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.079420  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:47.948033  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:47.948186  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:47.952836  8114 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.705s	user 0.004s	sys 0.000s
I20260709 14:53:47.956097  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:47.964257  2678 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:47.990739  8090 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.863862  7834 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:48.097012  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:47.996626  3543 rpcz_store.cc:276] 0709 14:53:44.864741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.941782 (+2077041us) service_pool.cc:224] Handling call
0709 14:53:47.901879 (+960097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.995112 (+ 93233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.995957 (+   845us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b5400200150. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.996120 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174857472}
I20260709 14:53:48.002614  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.004633  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 946) took 2042 ms. Trace:
I20260709 14:53:48.007454  3536 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:47.995823  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:48.008327  2610 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:48.051213  8099 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:47.997457  3538 rpcz_store.cc:276] 0709 14:53:46.468257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.860468 (+1392211us) service_pool.cc:224] Handling call
0709 14:53:47.886352 (+ 25884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.996501 (+110149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.996844 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:48.019119  2681 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:48.019328  8064 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:48.019431  3547 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 14:53:48.019842  7994 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:53:48.020812  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 974) took 1499 ms (client timeout 1714 ms). Trace:
I20260709 14:53:48.025789  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1629) took 1794 ms. Trace:
W20260709 14:53:48.120755  2751 rpcz_store.cc:269] 0709 14:53:46.521549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.523313 (+  1764us) service_pool.cc:224] Handling call
0709 14:53:48.020515 (+1497202us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:48.020703 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822475648}
I20260709 14:53:48.120880  2605 rpcz_store.cc:276] 0709 14:53:46.231127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.136823 (+905696us) service_pool.cc:224] Handling call
0709 14:53:47.207956 (+ 71133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.339022 (+131066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.020695 (+681673us) spinlock_profiling.cc:243] Waited 58 ms on lock 0x7b54000ca3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:48.025649 (+  4954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133370624}
I20260709 14:53:48.122184  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.036697  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:48.041006  3412 leader_election.cc:403] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:53:48.049976  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.050482  3542 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:48.052093  2760 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:48.023808  8006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.001s	sys 0.011s
I20260709 14:53:48.064873  3546 rpcz_store.cc:276] 0709 14:53:44.725834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.124844 (+1399010us) service_pool.cc:224] Handling call
0709 14:53:47.276852 (+1152008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.343830 (+ 66978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:47.925232 (+581402us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54004b37d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:47.925428 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648164736}
I20260709 14:53:48.125240  2751 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:48.068921  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.069962  2686 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:48.078095  2604 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:48.081935  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 133842us
I20260709 14:53:48.083825  3545 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 14:53:48.127110  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.086974  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:48.008498  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 141261us
I20260709 14:53:48.128001  2757 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:48.128284  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.098381  2611 rpcz_store.cc:276] 0709 14:53:45.962510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.011925 (+1049415us) service_pool.cc:224] Handling call
0709 14:53:48.003069 (+991144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.003813 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.004423 (+   610us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b54002fe9d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:48.004571 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242339584}
I20260709 14:53:48.101444  8090 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.102128  3543 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:48.129721  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.129849  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.099274  7978 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 14:53:48.099327  8062 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:48.100421  3541 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:48.138857  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:48.099107  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.139942  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:48.140213  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.140841  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.092551  7439 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:48.107610  7989 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:48.131655  8006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.002s	sys 0.012s
I20260709 14:53:48.120255  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.157776  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:48.158231  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:48.098747  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.160145  2756 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:48.130662  2755 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:48.119729  8110 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:48.129360  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 211898us
I20260709 14:53:48.067551  8092 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:48.129336  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60570.3us
W20260709 14:53:48.167976  3411 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.714s (SENT)
I20260709 14:53:48.168552  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:48.138034  2751 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:48.134141  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:48.142625  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.106976  7991 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:48.148281  2753 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:48.111873  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1494) took 1667 ms. Trace:
I20260709 14:53:48.172320  3540 rpcz_store.cc:276] 0709 14:53:46.444444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.295443 (+850999us) service_pool.cc:224] Handling call
0709 14:53:47.295837 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:47.503179 (+207342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.111551 (+608372us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002012d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:48.111796 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402436352}
I20260709 14:53:48.162701  2607 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:48.133246  7997 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:48.170557  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.135311  8001 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:48.141307  8033 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:48.165396  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.124544  7949 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.127843  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.074667  7932 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.131330  8099 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.184485  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.139490  7958 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17.
I20260709 14:53:48.141865  7945 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 14:53:48.224853  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.199893  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.237673  2752 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 14:53:48.144556  7968 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:48.183807  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:48.201396  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:48.200960  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:48.237421  2754 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:48.257879  2603 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:48.229949  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.258724  3545 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:53:48.260133  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58802.3us
I20260709 14:53:48.261785  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.215543  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.199790  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:48.263823  3547 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:48.227063  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:48.156762  7956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.006s	sys 0.050s
W20260709 14:53:48.259123  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 75456.7us
I20260709 14:53:48.217242  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:48.265055  2605 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:48.266268  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 138497us
W20260709 14:53:48.219817  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49377.1us
W20260709 14:53:48.188902  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.202329  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:48.273001  3536 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:48.277985  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:48.139766  8064 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 14:53:48.280174  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.268282  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.295240  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.228046  7977 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:48.300303  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:48.301538  3547 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:48.193439  8121 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:48.303364  8034 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:48.182947  8007 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:48.303969  8034 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:48.305425  8098 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:48.305989  8098 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.306416  8098 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:48.203075  7912 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:48.306905  7975 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:48.307155  7912 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.307811  7912 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:48.308738  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.309031  8099 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.183892  8076 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.123s
I20260709 14:53:48.309784  8099 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:48.310184  8099 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.310228  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.304123  8007 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:48.311056  8007 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:48.312529  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:48.307463  7975 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:48.313298  2751 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:48.313623  7975 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:48.316175  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.181737  8110 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.317867  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:48.164906  7989 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.321142  7989 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:48.318997  2755 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 14:53:48.194111  8106 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:48.199824  8036 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 14:53:48.200875  8100 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:48.202234  7984 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.201807  7991 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:48.330734  8076 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.220212  8006 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:48.300359  7997 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:48.222029  7876 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:48.276369  7956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.288s	user 0.006s	sys 0.051s
I20260709 14:53:48.299539  2681 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:48.209363  8092 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.226677  8062 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:48.310639  8099 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:48.342989  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.344344  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:48.345742  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50617.1us
I20260709 14:53:48.277299  7979 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:48.278075  8033 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:48.351102  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.219362  8001 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:48.353363  3536 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 12.
I20260709 14:53:48.296245  7032 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:48.280479  2606 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:48.355823  8076 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:48.347523  8120 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 14:53:48.350965  7994 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 14:53:48.202396  7916 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 14:53:48.199983  8078 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 14:53:48.346521  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:48.337752  7690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.036s	sys 0.042s
I20260709 14:53:48.329259  8106 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.243745  8103 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 14:53:48.331274  7989 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:48.375475  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:53:48.197839  8066 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:48.279484  7949 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:48.388921  2755 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 14:53:48.340778  8051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.029s	sys 0.043s
I20260709 14:53:48.458563  8103 consensus_queue.cc:1243] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 14:53:48.390960  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:48.461330  2682 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:53:48.395437  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:48.358505  7997 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.465626  7997 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:48.397303  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.402118  2751 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:48.352676  7984 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 14:53:48.403357  7032 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:48.445192  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98939.7us
I20260709 14:53:48.477695  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.393805  7979 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:48.383122  8007 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:48.403664  7831 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.022s	sys 0.057s
I20260709 14:53:48.443521  7958 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:48.447046  7690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.036s	sys 0.050s
I20260709 14:53:48.455258  8144 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:48.446854  8106 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:53:48.458766  8051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.030s	sys 0.044s
I20260709 14:53:48.388927  2605 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:48.461087  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.462177  2606 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:48.351753  8006 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:48.364199  7956 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:48.467242  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76468us
I20260709 14:53:48.513159  2606 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:48.514011  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1541) took 1252 ms. Trace:
I20260709 14:53:48.406966  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:48.499285  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.518981  2682 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:48.514354  2606 rpcz_store.cc:276] 0709 14:53:47.261452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.512585 (+1251133us) service_pool.cc:224] Handling call
0709 14:53:48.512961 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.513714 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.513948 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:48.399552  8062 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 14:53:48.524888  2605 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:48.525463  2606 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:48.525660  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1544) took 1236 ms. Trace:
W20260709 14:53:48.491313  7831 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.022s	sys 0.061s
I20260709 14:53:48.456637  7967 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 14:53:48.526073  2605 rpcz_store.cc:276] 0709 14:53:47.289654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.524361 (+1234707us) service_pool.cc:224] Handling call
0709 14:53:48.524692 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.525384 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.525602 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:48.530368  2603 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:48.530602  2757 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:48.489295  8132 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.531208  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1542) took 1261 ms. Trace:
I20260709 14:53:48.534876  2605 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:48.534729  2603 rpcz_store.cc:276] 0709 14:53:47.270027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.515937 (+1245910us) service_pool.cc:224] Handling call
0709 14:53:48.529988 (+ 14051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.530862 (+   874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.531143 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:48.484000  7979 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:48.478427  8120 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:48.492478  7984 consensus_queue.cc:1243] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:48.492852  8090 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.537484  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 16) took 1034 ms. Trace:
I20260709 14:53:48.491742  8144 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:48.511426  8081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.013s	sys 0.013s
I20260709 14:53:48.546037  2608 rpcz_store.cc:276] 0709 14:53:47.493073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.763959 (+270886us) service_pool.cc:224] Handling call
0709 14:53:48.520057 (+756098us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54001ebc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:48.520187 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345246080}
I20260709 14:53:48.552359  7032 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:48.553274  2608 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:48.476866  8040 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 14:53:48.516309  8006 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:48.555879  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.538539  7834 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
I20260709 14:53:48.563973  2603 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 14:53:48.505756  7958 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:48.556020  8144 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:48.475456  7991 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:48.532249  7598 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 14:53:48.591919  2603 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 14:53:48.592506  2603 pending_rounds.cc:85] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 7
I20260709 14:53:48.592785  2603 pending_rounds.cc:107] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 14:53:48.593092  2603 raft_consensus.cc:2889] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:48.568881  8078 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.560396  8120 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:48.567396  7932 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:48.562624  8081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.014s	sys 0.014s
W20260709 14:53:48.609647  2684 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.028s
W20260709 14:53:48.611311  2684 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.013s	sys 0.028s
I20260709 14:53:48.510265  8001 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:48.509953  8033 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:48.546882  8132 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:48.601734  2652 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.015s	sys 0.048s
W20260709 14:53:48.619671  2652 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.016s	sys 0.048s
W20260709 14:53:48.620338  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 666216us
W20260709 14:53:48.613706  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 356872us
I20260709 14:53:48.626060  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.632397  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:48.635186  7690 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:48.635397  8141 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:48.636121  7690 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:48.636977  8100 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:48.637454  8100 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:48.637794  8142 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 14:53:48.638351  8100 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:48.640463  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:48.644433  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.035s	sys 0.036s
W20260709 14:53:48.646028  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.035s	sys 0.036s
I20260709 14:53:48.646786  8081 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:53:48.647243  8139 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 14:53:48.645366  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.638742  7690 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:48.657996  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 518081us
I20260709 14:53:48.646716  7948 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:48.642481  8121 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:48.614178  8078 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:48.669130  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 345913us
W20260709 14:53:48.670346  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 403114us
W20260709 14:53:48.674975  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:48.675089  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.680681  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:48.681166  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.686088  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.678589  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:48.664739  8100 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:48.688336  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.706010  2686 raft_consensus.cc:2410] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:53:48.691146  8065 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:48.662912  8064 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.704715  2752 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:48.671173  8121 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:48.628692  7956 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:48.718077  2755 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 14:53:48.723381  2755 pending_rounds.cc:85] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:53:48.723723  2755 pending_rounds.cc:107] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 14:53:48.724128  2755 raft_consensus.cc:2889] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:48.724488  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:48.726374  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:48.670853  7439 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:48.730717  2752 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:48.751233  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:48.720650  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.752386  2752 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:48.752811  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.753666  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:48.707970  8127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.018s
I20260709 14:53:48.705049  7949 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.706291  7960 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 14:53:48.777899  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:48.707365  8076 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:48.780172  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:48.710170  7948 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:48.710427  8007 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:48.726994  8103 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.734663  8121 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:48.795051  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 523225us
I20260709 14:53:48.706624  7916 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 14:53:48.821168  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:48.825186  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.824369  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:48.748607  8065 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:48.776389  8127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.005s	sys 0.018s
W20260709 14:53:48.941294  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.787431  7960 consensus_queue.cc:1243] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:53:48.791774  7598 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:48.845893  7690 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:48.804280  8076 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:48.880884  7975 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.005s
I20260709 14:53:48.825562  7948 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:48.801419  7949 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:48.880841  7932 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:48.732647  8064 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:48.881201  8140 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 14:53:48.881784  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60734.2us
I20260709 14:53:49.191918  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:48.941048  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.827772  7912 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:48.749157  8120 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:48.954356  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.211741  2684 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:48.955699  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:48.962364  2685 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 14:53:48.828008  8081 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 14:53:48.968292  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:48.971264  2678 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:53:48.971325  2686 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 14:53:48.968464  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 282466us
I20260709 14:53:48.920449  8156 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:48.920830  8155 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:49.178963  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 138390us
W20260709 14:53:49.179095  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:48.956128  7958 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:48.962447  7978 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.981068  8132 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.174182  8051 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:48.905299  8103 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:49.236342  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44580.9us
I20260709 14:53:49.174916  7932 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:49.176541  7975 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.266353  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:49.334038  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 969) took 1359 ms. Trace:
W20260709 14:53:49.324954  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:49.334627  2612 rpcz_store.cc:276] 0709 14:53:47.974256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.994334 (+ 20078us) service_pool.cc:224] Handling call
0709 14:53:49.266101 (+1271767us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:49.333966 (+ 67865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26994048}
W20260709 14:53:49.230970  7717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.015s	sys 0.044s
I20260709 14:53:49.278156  3545 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:48.915442  7439 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:49.280121  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.026s	sys 0.034s
W20260709 14:53:49.287108  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48261.9us
I20260709 14:53:49.287772  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:49.347007  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59329.8us
I20260709 14:53:49.225677  8078 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:49.288118  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 319951us
W20260709 14:53:49.283353  7943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.026s	sys 0.043s
I20260709 14:53:49.176839  8076 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.237160  8090 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:49.238687  2652 leader_election.cc:336] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.910s (SENT)
I20260709 14:53:49.325481  7932 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:49.292153  7705 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 14:53:49.322065  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.384336  2603 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15. Re-sending same reply.
I20260709 14:53:49.232316  8156 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:49.349349  8176 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:49.349314  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.027s	sys 0.035s
W20260709 14:53:49.352579  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:49.290728  7968 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.016s	user 0.010s	sys 0.006s
I20260709 14:53:49.331265   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 14:53:49.361656  8110 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.417351  3547 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 14:53:49.235221  7611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.003s	sys 0.056s
I20260709 14:53:49.358208  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.359375  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1326) took 1179 ms. Trace:
W20260709 14:53:49.323537  8163 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 14:53:49.431624  8170 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 14:53:49.431918  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:49.364693  3536 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:49.332253  7977 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.032s	user 0.014s	sys 0.002s
I20260709 14:53:49.433815  8175 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.434490  8175 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.436074  2677 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:49.436985  8175 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.437733  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1457) took 1609 ms. Trace:
W20260709 14:53:49.324172  8146 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 14:53:49.367357  3541 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 14:53:49.268744  8160 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:49.369480  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:49.369848  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:49.371491  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:49.285712  7978 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.324559  7945 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:49.334237  8081 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.348177  7970 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.348579  8001 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:49.322365  8169 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:49.237514  8155 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:49.361192  7943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.026s	sys 0.044s
W20260709 14:53:49.395262  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 623748us
I20260709 14:53:49.356928  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:49.491662  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 134828us
I20260709 14:53:49.492827  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:49.493232  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:49.494038  3541 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:49.472939  7032 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.502677  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.504122  2685 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:49.473445  7945 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:49.334071  7657 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 14:53:49.427300  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:49.427592  8164 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 14:53:49.429865  8098 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.099s	user 0.013s	sys 0.000s
I20260709 14:53:49.395155  7960 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.296702  8051 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.331745  8149 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 14:53:49.419044  3542 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:49.439607  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.540907  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1543) took 2256 ms. Trace:
I20260709 14:53:49.541463  2603 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:49.541395  2606 rpcz_store.cc:276] 0709 14:53:47.284921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.521840 (+1236919us) service_pool.cc:224] Handling call
0709 14:53:48.525201 (+  3361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.526361 (+  1160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.534000 (+1007639us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b540042dbd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:49.540835 (+  6835us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6654208}
W20260709 14:53:49.324924  8150 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 14:53:49.433871  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:49.444537  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.445930  8158 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.445657  8156 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.341475  7948 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.385828  8090 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.384379  8078 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.449426  8103 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.355531  8151 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 14:53:49.294221  8065 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.453038  2759 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:49.438158  2677 rpcz_store.cc:276] 0709 14:53:47.828733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.860794 (+ 32061us) service_pool.cc:224] Handling call
0709 14:53:47.881229 (+ 20435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:49.435863 (+1554634us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b54003baea0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:53:49.436560 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.437485 (+   925us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:49.437664 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1731775104}
I20260709 14:53:49.457437  3540 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:49.355443  8141 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 14:53:49.473752  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 235086us
W20260709 14:53:49.438318  7611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.004s	sys 0.056s
I20260709 14:53:49.443125  7968 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:49.528213  7916 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 14:53:49.556051  8099 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.158s	user 0.010s	sys 0.006s
I20260709 14:53:49.504542  2685 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:49.504951  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.512920  7032 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.517673  2684 pending_rounds.cc:85] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 8
I20260709 14:53:49.597916  2684 pending_rounds.cc:107] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 14:53:49.598258  2684 raft_consensus.cc:2889] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:49.578219  8149 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:49.603138  8149 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:49.604166  8149 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.605474  8149 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:49.606997  8151 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.585904  8078 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.607717  8151 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.459962  7977 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 14:53:49.610088  8006 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.032s	user 0.005s	sys 0.006s
I20260709 14:53:49.528641  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.533056  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.531344  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.522434  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:49.421058  8176 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.493521  8155 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:49.500699  7972 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:49.542970  2606 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:49.497875  8001 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:49.545109  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:49.544044  8098 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.542970  2603 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:49.552052  2611 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:53:49.638329  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 93408.7us
I20260709 14:53:49.552832  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.541025  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.510180  7984 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:49.490880  7970 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.485548  8081 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:49.536211  8106 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.009s	user 0.012s	sys 0.004s
I20260709 14:53:49.520850  8169 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:49.545601  7876 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.175s	user 0.009s	sys 0.007s
I20260709 14:53:49.503419  7944 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 14:53:49.589486  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 109722us
I20260709 14:53:49.386011  7598 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:49.529609  8160 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 14:53:49.527872  7717 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 14:53:49.563349  7976 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.521471  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.601400  2677 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:53:49.564280  8158 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.526011  7956 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.483707  8174 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:49.612983  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:49.617260  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.626223  3547 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:49.627874  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 105513us
I20260709 14:53:49.596074  2685 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.581773  8027 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.631223  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1635) took 1616 ms. Trace:
I20260709 14:53:49.629873  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:49.602417  8051 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.582516  7912 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.552922  7960 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.636751  2606 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:49.610635  8006 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.702715  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:49.742137  3541 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:53:49.665009  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:49.637241  3540 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:49.640925  3542 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:49.681126  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:49.745327  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:49.745980  3542 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:53:49.746599  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:49.496973  8110 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 14:53:49.592985  8064 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.752355  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:49.685081  2680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 980) took 2042 ms (client timeout 1539 ms). Trace:
W20260709 14:53:49.771329  2680 rpcz_store.cc:269] 0709 14:53:47.642864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.691691 (+ 48827us) service_pool.cc:224] Handling call
0709 14:53:49.684780 (+1993089us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:49.684979 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4088775936}
W20260709 14:53:49.772444  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62577us
I20260709 14:53:49.772508  2758 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:49.683246  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:49.774597  3546 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:49.776684  2605 rpcz_store.cc:276] 0709 14:53:48.014582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.542431 (+527849us) service_pool.cc:224] Handling call
0709 14:53:48.542982 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:48.543327 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:48.543367 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:48.568655 (+ 25288us) log.cc:844] Serialized 26 byte log entry
0709 14:53:48.968771 (+400116us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:48.968836 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.968933 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:49.630968 (+662035us) raft_consensus.cc:1685] finished
0709 14:53:49.630994 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:49.631150 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":360026,"threads_started":1,"wal-append.queue_time_us":602018}
I20260709 14:53:49.779013  2685 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:49.780187  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:49.781930  2686 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:49.783953  3547 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 14:53:49.784732  3546 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:49.611522  8141 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.788431  3546 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:49.781703  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:49.789520  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:49.610733  8078 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.796913  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:49.796597  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:49.634256  8033 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:53:49.672408  8168 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 14:53:49.662518  8165 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 14:53:49.661738  8171 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 14:53:49.660305  8172 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 14:53:49.682525  7997 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.198s	user 0.007s	sys 0.005s
W20260709 14:53:49.661119  8173 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 14:53:49.860518  2685 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 14:53:49.661932  8177 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 14:53:49.599746  8099 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.654553  7977 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:49.812091  2609 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:49.735854  8062 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.813061  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:49.788388  8106 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.814463  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:49.814776  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.891538  2611 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:49.813289  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.668339  7032 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.733394  7978 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.610093  8151 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.667564  8076 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:49.781630  7976 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:49.821660  8184 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.822989  7657 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:49.823137  7960 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.670022  8098 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.775669  8102 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.821241  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1629) took 1131 ms. Trace:
I20260709 14:53:49.919178  2755 rpcz_store.cc:276] 0709 14:53:48.714218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.820313 (+106095us) service_pool.cc:224] Handling call
0709 14:53:48.884231 (+ 63918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.884501 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:49.360504 (+476003us) log.cc:844] Serialized 14 byte log entry
0709 14:53:49.820632 (+460128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:49.820724 (+    92us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:49.820806 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.820921 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:49.821137 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7271,"prepare.queue_time_us":30499,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":21134,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7940,"thread_start_us":894558,"threads_started":3,"wal-append.queue_time_us":303760}
I20260709 14:53:49.920650  3547 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 14:53:49.921751  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:49.922729  2755 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:49.683887  8176 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.674244  8090 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.826882  3541 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:49.826375  8078 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 14:53:49.799934  8162 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.794442  8182 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:49.861130  2686 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:53:49.802137  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49941.5us
I20260709 14:53:49.863344  2685 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 8
I20260709 14:53:49.933437  2685 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 14:53:49.933801  2685 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:49.807193  8051 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:49.811599  8174 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:49.935683  8174 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:49.936118  2605 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 14:53:49.936750  8174 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.888700  8195 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 14:53:49.812341  8033 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:49.884119  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71171.7us
I20260709 14:53:49.825628  7943 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:49.644872  7984 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:49.811228  8183 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:49.791702  8027 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.831862  7439 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:49.940653  7439 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:49.941715  7439 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.943027  8006 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.943547  7943 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:49.944525  7943 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.947124  8033 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:49.948812  8081 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:49.949608  8121 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.101s	user 0.014s	sys 0.001s
I20260709 14:53:49.791435  7956 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.811734  7944 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 14:53:49.950104  8121 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.844487  7968 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:49.951633  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1492) took 1985 ms. Trace:
I20260709 14:53:49.951566  7968 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.952584  8121 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.953361  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:49.953933  7968 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.950125  8162 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:49.955387  8167 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.628s	user 0.001s	sys 0.003s
I20260709 14:53:49.956753  8162 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.958760  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:49.959245  7984 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:49.960173  7690 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 14:53:49.963393  7970 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.895851  7598 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.965207  7975 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:49.967235  3541 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 14:53:49.968027  7945 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:49.969244  7945 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:49.969570  8144 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.365s	user 0.008s	sys 0.005s
I20260709 14:53:49.970307  8144 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.971570  2612 raft_consensus.cc:2368] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15.
I20260709 14:53:49.809942  8141 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:49.921434  8100 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.213s	user 0.012s	sys 0.004s
I20260709 14:53:49.973667  8141 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:49.824996  8175 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 14:53:49.785724  8169 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:49.975355  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1636) took 1960 ms (client timeout 1663 ms). Trace:
W20260709 14:53:49.975885  2610 rpcz_store.cc:269] 0709 14:53:48.027669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.126780 (+ 99111us) service_pool.cc:224] Handling call
0709 14:53:49.975155 (+1848375us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:49.975279 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031736192}
I20260709 14:53:49.976749  8169 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:49.876812  7977 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:49.977239  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:49.977586  7977 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:49.977970  2610 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
W20260709 14:53:49.980356  8166 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.655s	user 0.000s	sys 0.002s
I20260709 14:53:49.980749  8144 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.982355  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 979) took 3141 ms. Trace:
I20260709 14:53:49.982882  2759 rpcz_store.cc:276] 0709 14:53:46.841130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:46.874869 (+ 33739us) service_pool.cc:224] Handling call
0709 14:53:46.875304 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:49.581588 (+2706284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.982057 (+400469us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:49.982255 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5894692224}
W20260709 14:53:49.984208  8204 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 14:53:49.984885  2759 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 14:53:49.983311  7932 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.002s
I20260709 14:53:49.986148  7932 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.986469  7984 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:49.987792  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1639) took 1917 ms. Trace:
I20260709 14:53:49.920127  8184 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:49.988515  7932 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.988948  7984 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.989701  7984 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:53:49.832036  8001 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.917716  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:49.872346  7997 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.909442  7912 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:49.927073  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:49.894366  8062 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:49.932550  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:49.934489  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1331) took 1746 ms. Trace:
I20260709 14:53:50.016315  3545 rpcz_store.cc:276] 0709 14:53:48.187931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.301483 (+113552us) service_pool.cc:224] Handling call
0709 14:53:48.301872 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:49.341503 (+1039631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.934196 (+592693us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54004b23d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:49.934420 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164317568}
I20260709 14:53:49.951964  7956 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:49.912045  7032 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:53:49.959573  2611 raft_consensus.cc:2376] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17. Re-sending same reply.
I20260709 14:53:49.991231  2606 raft_consensus.cc:2368] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17.
I20260709 14:53:49.988240  2608 rpcz_store.cc:276] 0709 14:53:48.070958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.554189 (+483231us) service_pool.cc:224] Handling call
0709 14:53:48.556317 (+  2128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:48.556672 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:48.556710 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:48.568420 (+ 11710us) log.cc:844] Serialized 26 byte log entry
0709 14:53:48.896017 (+327597us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:48.896072 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:48.896171 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:49.979698 (+1083527us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:49.987425 (+  7727us) raft_consensus.cc:1685] finished
0709 14:53:49.987460 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:49.987719 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55176960,"thread_start_us":274775,"threads_started":1,"wal-append.queue_time_us":590668}
W20260709 14:53:49.894896  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 210231us
W20260709 14:53:50.028908  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96461.7us
I20260709 14:53:49.952143  2757 rpcz_store.cc:276] 0709 14:53:47.965937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.128952 (+163015us) service_pool.cc:224] Handling call
0709 14:53:48.129318 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.531064 (+401746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.951360 (+1420296us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:49.951563 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903063936}
I20260709 14:53:49.995005   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:53:49.911239  8151 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.001628  2758 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:53:50.009560  2755 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:50.004103  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:50.040658  2610 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13. Re-sending same reply.
I20260709 14:53:49.990250  8184 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.031425  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:50.057543  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1450) took 1847 ms (client timeout 1904 ms). Trace:
W20260709 14:53:50.060420  2681 rpcz_store.cc:269] 0709 14:53:48.210044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.215008 (+  4964us) service_pool.cc:224] Handling call
0709 14:53:50.057173 (+1842165us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:50.057428 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82339712}
I20260709 14:53:50.057883  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.064967  2611 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:53:50.066354  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.070230  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.008432  7958 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.117372  2604 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:50.125779  2755 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:50.080096  2611 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:53:50.057961  8172 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 14:53:49.925472  8098 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
W20260709 14:53:50.039980  7979 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.480s	user 0.011s	sys 0.007s
I20260709 14:53:49.999768  8189 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:50.129106  8062 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:50.130421  8106 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:49.991075  8199 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.206s
I20260709 14:53:50.082643  7032 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:50.135246  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:50.137269  7997 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:50.082341  8144 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 14:53:50.138916  7958 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.083017  8092 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:53:50.084573  8151 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.173537  7968 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.175087  8051 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.013727  8121 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:49.919814  7657 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.050942  7932 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:50.145459  2758 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:49.923224  8102 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:50.146127  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.146597  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.175076  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.008669  8169 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 14:53:50.177896  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:50.177896  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 316890us
I20260709 14:53:50.177592  7979 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.036844  8033 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.189566  3545 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:50.164503  8184 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.145656  7912 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.175604  8006 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.108732  7975 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 14:53:50.127043  8149 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.521s	user 0.014s	sys 0.000s
W20260709 14:53:50.192817  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 378443us
I20260709 14:53:50.143915  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.204644  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:50.207547  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:50.217262  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:50.287186  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.187690  8120 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.290516  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:50.263777  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:50.261312  8099 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:50.298305  2751 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:50.269445  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:50.274711  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:50.280455  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.014s	sys 0.037s
I20260709 14:53:50.287185  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:50.288934  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84447.1us
W20260709 14:53:50.301383  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.014s	sys 0.037s
I20260709 14:53:50.287701  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.295924  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.132877  8155 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:50.301394  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:50.265658  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.304363  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:50.206609  7968 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.304812  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.303112  2757 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:53:50.306020  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 189700us
I20260709 14:53:50.228245  8121 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.222424  8051 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.308576  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.317289  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.306927  2605 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:50.319531  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.317003  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.297673  7439 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.332623  2757 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:50.297837  8040 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:50.333932  2681 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 14:53:50.330796  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:50.305495  8149 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.340984  2611 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:50.343801  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.345306  2678 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:50.306330  7975 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:50.283192  7912 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.350878  2610 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:53:50.352761  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:50.284608  8184 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.286679  8006 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.355338  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.361034  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:50.357578  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:50.362735  2683 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:50.354337  7989 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:50.363983  2756 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: [12.6-14.7]
I20260709 14:53:50.320060  8078 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:50.374642  2680 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:53:50.335210  8141 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:50.375947  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:50.331005  8210 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:53:50.300491  8120 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.442464  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:50.379273  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:50.443205  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.443400  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.333675  7956 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:50.439345  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.293540  7032 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.447274  2683 raft_consensus.cc:2376] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16. Re-sending same reply.
I20260709 14:53:50.339574  7690 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:50.446714  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.328321  8132 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 14:53:50.343066  8069 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 14:53:50.353165  8174 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:50.465282  2652 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.520s (SENT)
I20260709 14:53:50.492532  2678 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:53:50.494179  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.383471  7876 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:50.510900  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.500757  2757 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:50.513909  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.495595  8174 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.375952  7989 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:50.504479  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 125639us
I20260709 14:53:50.351290  7439 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.437418  7916 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:50.515363  2757 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:50.461544  7977 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:50.501468  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:50.376770  7975 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:50.450992  8141 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:50.560642  8141 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:50.561831  8036 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:50.562355  8036 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:50.562630  7912 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:50.563421  8036 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:50.564966  8036 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:50.566393  8193 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.658s	user 0.000s	sys 0.002s
I20260709 14:53:50.508235  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:50.384156  8169 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 14:53:50.537644  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:50.591035  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83198.1us
I20260709 14:53:50.592345  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.593024  2756 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:50.593122  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:50.354679  8149 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:50.544133  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78927.1us
I20260709 14:53:50.541603  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:50.574560  8207 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 14:53:50.383669  7970 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 14:53:50.558290  8184 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.542378  7916 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:50.559595  7958 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.560065  3546 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:50.532341  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:50.569448  8216 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.212s
I20260709 14:53:50.571926  8141 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:50.700176  2652 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.045s (SENT)
I20260709 14:53:50.572988  2760 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:50.505981  7956 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:50.511693  7690 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.535943  7989 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:50.383618  7960 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
W20260709 14:53:50.577687  8218 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 14:53:50.578908  8033 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.449327  8195 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 14:53:50.375133  7717 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 14:53:50.517500  8162 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:53:50.519160  8076 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.523264  7991 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.597180  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:50.603293  8132 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:50.374897  8099 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 14:53:50.489998  7978 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 14:53:50.640658  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1520) took 1024 ms. Trace:
I20260709 14:53:50.788734  3540 rpcz_store.cc:276] 0709 14:53:49.616388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.191509 (+575121us) service_pool.cc:224] Handling call
0709 14:53:50.191924 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.208090 (+ 16166us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:50.281172 (+ 73082us) log.cc:844] Serialized 14 byte log entry
0709 14:53:50.437148 (+155976us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.443950 (+  6802us) log.cc:844] Serialized 26 byte log entry
0709 14:53:50.444160 (+   210us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:50.444252 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.444359 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:50.640293 (+195934us) raft_consensus.cc:1685] finished
0709 14:53:50.640331 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:50.640567 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90540,"prepare.queue_time_us":65211,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":6600,"raft.queue_time_us":36631,"raft.run_cpu_time_us":1196,"raft.run_wall_time_us":52213,"thread_start_us":133543,"threads_started":2,"wal-append.queue_time_us":295491}
I20260709 14:53:50.725257  8151 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:50.573024  8215 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 14:53:50.702797  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.723201  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:50.802217  2681 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:50.701357  2757 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:50.615933  8208 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:50.506359  8034 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.698447  7916 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:50.759994  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 227764us
I20260709 14:53:50.814704  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.703408  7657 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:50.778416  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 416453us
I20260709 14:53:50.762807  2756 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:50.617127  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1195) took 1239 ms. Trace:
I20260709 14:53:50.720978  8120 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:50.810685  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 174379us
I20260709 14:53:50.769310  7598 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:50.817306  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.830096  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.831332  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.857322  2757 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:50.818871  3536 rpcz_store.cc:276] 0709 14:53:49.377451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.639894 (+262443us) service_pool.cc:224] Handling call
0709 14:53:49.640375 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:49.648529 (+  8154us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:50.207308 (+558779us) log.cc:844] Serialized 14 byte log entry
0709 14:53:50.337309 (+130001us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.344054 (+  6745us) log.cc:844] Serialized 26 byte log entry
0709 14:53:50.344233 (+   179us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:50.354869 (+ 10636us) log.cc:844] Serialized 14 byte log entry
0709 14:53:50.355102 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.355192 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:50.616811 (+261619us) raft_consensus.cc:1685] finished
0709 14:53:50.616848 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:50.617034 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130186,"prepare.queue_time_us":235578,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":126,"raft.queue_time_us":150347,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":53508,"thread_start_us":494023,"threads_started":2,"wal-append.queue_time_us":336012}
W20260709 14:53:50.821187  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:50.815037  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:50.869663  2681 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:50.758260  7979 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:50.832626  8185 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 14:53:50.833032  8203 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.403s	user 0.002s	sys 0.000s
I20260709 14:53:50.754662  8212 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 14:53:50.835268  8217 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:53:50.838966  8201 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.412s	user 0.000s	sys 0.002s
W20260709 14:53:50.839762  8225 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 14:53:50.726856  7975 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:50.900202  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 200046us
W20260709 14:53:50.842181  8198 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.350s	user 0.000s	sys 0.002s
I20260709 14:53:50.842779  7968 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.845847  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1203) took 1465 ms. Trace:
I20260709 14:53:50.767088  8006 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:50.847646  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1554) took 1554 ms. Trace:
I20260709 14:53:50.911649  2604 rpcz_store.cc:276] 0709 14:53:49.292895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.537304 (+244409us) service_pool.cc:224] Handling call
0709 14:53:49.537689 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:50.118159 (+580470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.847383 (+729224us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:50.847564 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256823040}
I20260709 14:53:50.913615  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:50.914088  2652 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.086s (SENT) [suppressed 29 similar messages]
I20260709 14:53:50.848326  8141 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:50.819515  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.820389  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:50.800770  8132 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:50.920226  7439 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:50.920720  8132 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:50.875182  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:50.915771  8051 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:50.923556  7956 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:50.923957  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:50.924885  2757 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:50.924923  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1335) took 1565 ms. Trace:
I20260709 14:53:50.925668  8149 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:50.925804  8174 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:50.876082  2605 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.004s
I20260709 14:53:50.926574  2605 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:53:50.927187  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 976) took 1109 ms. Trace:
I20260709 14:53:50.927508  2605 rpcz_store.cc:276] 0709 14:53:49.817837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.304042 (+486205us) service_pool.cc:224] Handling call
0709 14:53:50.927115 (+623073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:50.928685  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.929384  2605 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
W20260709 14:53:50.881770  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66797.2us
I20260709 14:53:50.932626  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:50.882766  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1514) took 1512 ms. Trace:
I20260709 14:53:50.817693  8208 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:50.888185  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.935094  8208 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:50.794806  8076 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.826726  8155 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:50.871333  7945 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.902s	user 0.016s	sys 0.000s
I20260709 14:53:50.796844  7991 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:50.885241  8213 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.370s	user 0.000s	sys 0.002s
I20260709 14:53:50.875604  8034 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:50.899761  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:50.911005  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84498.7us
I20260709 14:53:50.888442  8121 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:50.891294  7932 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:50.921260  8103 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:50.911475  3542 rpcz_store.cc:276] 0709 14:53:49.380370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.747055 (+366685us) service_pool.cc:224] Handling call
0709 14:53:49.747343 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:49.812335 (+ 64992us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:49.834771 (+ 22436us) log.cc:844] Serialized 14 byte log entry
0709 14:53:50.027461 (+192690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:50.027527 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:50.027590 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.845475 (+817885us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b54002b9dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:50.845560 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:50.845766 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":115444,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":10294,"raft.queue_time_us":94,"spinlock_wait_cycles":260287104,"thread_start_us":177694,"threads_started":1,"wal-append.queue_time_us":356895}
I20260709 14:53:50.965204  7439 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:50.902478  8230 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.934731  3543 rpcz_store.cc:276] 0709 14:53:49.370711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.370919 (+   208us) service_pool.cc:224] Handling call
0709 14:53:49.394200 (+ 23281us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:49.418771 (+ 24571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:49.418805 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:49.455101 (+ 36296us) log.cc:844] Serialized 52 byte log entry
0709 14:53:49.628966 (+173865us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:49.826016 (+197050us) log.cc:844] Serialized 14 byte log entry
0709 14:53:49.996736 (+170720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.997016 (+   280us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:50.565520 (+568504us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:50.872544 (+307024us) raft_consensus.cc:1685] finished
0709 14:53:50.882433 (+  9889us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:50.882678 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54277,"prepare.queue_time_us":180140,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":135857,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":35786,"spinlock_wait_cycles":44334720,"thread_start_us":488020,"threads_started":3,"wal-append.queue_time_us":327180}
I20260709 14:53:50.981880  8230 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:50.979223  7932 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:50.983031  7932 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:50.984985  8230 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:50.952976  8051 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:50.952174  7968 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:50.987476  7968 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:50.988745  7968 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:50.963999  7948 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.013s
I20260709 14:53:50.936213  8078 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 14:53:50.981010  8229 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:50.987226  8051 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:50.979327  8103 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:51.036468  7991 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:51.037214  8234 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 14:53:51.001796  8103 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:51.011418  8229 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:51.078769  8156 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:51.089174  8103 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:51.089731  8229 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:51.117581  7989 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:51.124960  8241 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 14:53:51.143203  8076 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:51.126605  7989 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:51.196348  8208 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:51.253958  8208 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:50.916057  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:50.936116  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.930491  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:50.906240  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3044101759999194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 14:53:50.952849  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 132525us
I20260709 14:53:50.877741  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:50.919546  2755 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:50.948946  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:50.974006  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74330.5us
I20260709 14:53:50.974455  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:50.925407  3537 rpcz_store.cc:276] 0709 14:53:49.359715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.362353 (+  2638us) service_pool.cc:224] Handling call
0709 14:53:49.362880 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:49.517284 (+154404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:49.517341 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:49.556869 (+ 39528us) log.cc:844] Serialized 26 byte log entry
0709 14:53:49.798949 (+242080us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:49.799002 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.801514 (+  2512us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:50.924685 (+1123171us) raft_consensus.cc:1685] finished
0709 14:53:50.924709 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:50.924857 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62164,"raft.queue_time_us":95112,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":7759,"thread_start_us":321264,"threads_started":2,"wal-append.queue_time_us":410502}
I20260709 14:53:50.990057  7439 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:50.994979  7948 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.001s	sys 0.013s
I20260709 14:53:51.375891  2754 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:51.353595  2610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.000s	sys 0.009s
I20260709 14:53:51.669236  7997 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.367347  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1487) took 1037 ms. Trace:
I20260709 14:53:51.674896  7997 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:51.675040  2685 rpcz_store.cc:276] 0709 14:53:50.317560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.319051 (+  1491us) service_pool.cc:224] Handling call
0709 14:53:50.319392 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.343000 (+ 23608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:50.343035 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.343585 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 14:53:50.777243 (+433658us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:50.777321 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.777444 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:50.984392 (+206948us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:51.353861 (+369469us) raft_consensus.cc:1685] finished
0709 14:53:51.353944 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:51.354684 (+   740us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23064,"raft.queue_time_us":29491,"spinlock_wait_cycles":26835968,"thread_start_us":433174,"threads_started":1,"wal-append.queue_time_us":592601}
W20260709 14:53:51.360489  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:51.679531  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:51.353281  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:51.683051  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:51.366511  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 994) took 1011 ms. Trace:
I20260709 14:53:51.350449  2680 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:51.387916  2756 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
W20260709 14:53:51.349119  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 434881us
I20260709 14:53:51.691247  7970 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.369108  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:51.692771  7970 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:51.693965  2604 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:51.376643  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.697757  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:51.350499  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1488) took 1032 ms. Trace:
I20260709 14:53:51.700459  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.360482  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:51.369380  2726 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.593s (SENT)
I20260709 14:53:51.706697  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:51.708303  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:51.708680  2604 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15. Re-sending same reply.
I20260709 14:53:51.711263  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.711612  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1001) took 1195 ms. Trace:
I20260709 14:53:51.712225  2756 rpcz_store.cc:276] 0709 14:53:50.515843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.948614 (+432771us) service_pool.cc:224] Handling call
0709 14:53:51.711377 (+762763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.713645  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.422689  7439 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:51.683894  2682 rpcz_store.cc:276] 0709 14:53:50.355736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.356541 (+   805us) service_pool.cc:224] Handling call
0709 14:53:50.356862 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:50.357119 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:50.942441 (+585322us) log.cc:844] Serialized 14 byte log entry
0709 14:53:51.364650 (+422209us) log.cc:844] Serialized 14 byte log entry
0709 14:53:51.365339 (+   689us) log.cc:844] Serialized 14 byte log entry
0709 14:53:51.365534 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:51.365609 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:51.365775 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.365955 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:51.366254 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71644,"prepare.queue_time_us":471048,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":102,"raft.queue_time_us":268624,"raft.run_cpu_time_us":654,"raft.run_wall_time_us":655,"spinlock_wait_cycles":1901440,"thread_start_us":863297,"threads_started":2,"wal-append.queue_time_us":147910}
I20260709 14:53:51.277446  8229 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:51.423740  7598 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.299s
I20260709 14:53:51.716024  2681 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 14:53:51.724485  7598 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:51.725148  7598 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:51.725766  7598 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:51.424424  7657 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.727789  7657 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:51.449733  7912 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:51.729859  7912 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:51.730562  7912 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:51.732956  8184 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:51.733652  8184 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:51.425130  7717 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:51.707213  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 338185us
I20260709 14:53:51.736104  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.710698  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:51.734964  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.000s	sys 0.010s
I20260709 14:53:51.721385  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:51.396404  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 421909us
I20260709 14:53:51.734930  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:51.453620  7984 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:51.718354  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 358530us
I20260709 14:53:51.716722  7970 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:51.746506  7997 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:51.746863  8184 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:51.749060  8184 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:51.744472  2680 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
I20260709 14:53:51.735414  7717 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:51.755312  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:51.718947  8239 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:51.454172  8169 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:51.760754  2685 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:51.763972  2682 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:51.699697  2686 rpcz_store.cc:276] 0709 14:53:50.317906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.318055 (+   149us) service_pool.cc:224] Handling call
0709 14:53:50.318425 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.318903 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:50.318936 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.319510 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 14:53:50.730993 (+411483us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:50.731076 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.731168 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:51.349546 (+618378us) raft_consensus.cc:1685] finished
0709 14:53:51.349629 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:51.350086 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25243,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9801,"thread_start_us":410720,"threads_started":1,"wal-append.queue_time_us":488736}
I20260709 14:53:51.783612  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.774614  8239 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:51.792837  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:51.793852  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 970) took 3819 ms (client timeout 2023 ms). Trace:
I20260709 14:53:51.794492  8208 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:53:51.794461  2609 rpcz_store.cc:269] 0709 14:53:47.974612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:47.997050 (+ 22438us) service_pool.cc:224] Handling call
0709 14:53:48.226889 (+229839us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:51.793289 (+3566400us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b540050ffd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:51.793617 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3568112768}
I20260709 14:53:51.798794  2603 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:51.802021  2757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.025s	sys 0.039s
W20260709 14:53:51.811982  2757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.025s	sys 0.039s
I20260709 14:53:51.809163  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:51.818286  2686 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:49.418954  3538 rpcz_store.cc:276] 0709 14:53:48.179842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.180029 (+   187us) service_pool.cc:224] Handling call
0709 14:53:48.259890 (+ 79861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:48.260149 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:48.669827 (+409678us) log.cc:844] Serialized 14 byte log entry
0709 14:53:49.358923 (+689096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:49.359006 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:49.359063 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:49.359141 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:49.359306 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21887,"prepare.queue_time_us":632412,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":411352,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":13741,"thread_start_us":1032825,"threads_started":3,"wal-append.queue_time_us":698626}
I20260709 14:53:51.819754  2685 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:53:51.782979  7984 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:51.821409  7657 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:51.830271  8027 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:51.798485  7991 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 14:53:51.831935  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 141845us
W20260709 14:53:51.837595  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:51.839620  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:51.912488  2680 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:51.881610  2604 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:53:51.913959  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 973) took 2096 ms. Trace:
I20260709 14:53:51.914407  2604 rpcz_store.cc:276] 0709 14:53:49.817077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.755575 (+1938498us) service_pool.cc:224] Handling call
0709 14:53:51.811514 (+ 55939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.913457 (+101943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.913822 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:53:51.921494  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94911.9us
I20260709 14:53:51.948330  2756 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:51.949349  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1648) took 1008 ms. Trace:
I20260709 14:53:51.949728  2756 rpcz_store.cc:276] 0709 14:53:50.940990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.801265 (+860275us) service_pool.cc:224] Handling call
0709 14:53:51.949231 (+147966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.950933  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1462) took 1055 ms. Trace:
I20260709 14:53:51.951494  2680 rpcz_store.cc:276] 0709 14:53:50.871625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.772288 (+900663us) service_pool.cc:224] Handling call
0709 14:53:51.927084 (+154796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.952595  7984 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:51.926632  8181 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.954152  2680 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:51.954579  8181 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:51.956300  2604 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:51.957547  8208 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.958110  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:51.968400  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1561) took 1159 ms. Trace:
I20260709 14:53:51.969079  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:51.969072  2604 rpcz_store.cc:276] 0709 14:53:50.805532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.955343 (+1149811us) service_pool.cc:224] Handling call
0709 14:53:51.955929 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.956890 (+   961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.965237 (+  8347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.969405  3542 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
W20260709 14:53:51.971634  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 124148us
I20260709 14:53:51.974337  3546 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:51.972316  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1469) took 1020 ms. Trace:
I20260709 14:53:51.976066  2680 rpcz_store.cc:276] 0709 14:53:50.947578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.952859 (+1005281us) service_pool.cc:224] Handling call
0709 14:53:51.953483 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.954694 (+  1211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.967736 (+ 13042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.921309  8229 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:51.958467  8208 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:51.982720  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:51.988759  7997 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:51.993000  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:51.982445  8180 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:51.985700  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.000942  7997 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:52.000948  8180 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:51.998564  2604 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:51.989025  8181 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.007095  8132 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:52.003628  7717 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.008008  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1562) took 1199 ms. Trace:
I20260709 14:53:52.012394  2604 rpcz_store.cc:276] 0709 14:53:50.806516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.973731 (+1167215us) service_pool.cc:224] Handling call
0709 14:53:51.998165 (+ 24434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.005084 (+  6919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.005359 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.015573  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1518) took 2581 ms. Trace:
I20260709 14:53:52.016081  3547 rpcz_store.cc:276] 0709 14:53:49.434042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.792392 (+358350us) service_pool.cc:224] Handling call
0709 14:53:49.792960 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:49.921218 (+128258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.014894 (+2093676us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54004c4ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.015318 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289786624}
W20260709 14:53:52.019992  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1552) took 2740 ms (client timeout 1593 ms). Trace:
W20260709 14:53:52.020560  2612 rpcz_store.cc:269] 0709 14:53:49.279968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.336379 (+ 56411us) service_pool.cc:224] Handling call
0709 14:53:52.019490 (+2683111us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.019758 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181789056}
I20260709 14:53:52.021126  7831 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:52.021624  7831 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.023092  7831 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.024574  3540 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:52.025125  7831 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:52.011232  3543 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:52.029106  2612 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:52.032699  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1572) took 1141 ms. Trace:
I20260709 14:53:52.036285  2612 rpcz_store.cc:276] 0709 14:53:50.889023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.021809 (+1132786us) service_pool.cc:224] Handling call
0709 14:53:52.022319 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.029671 (+  7352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.029845 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.034703  2756 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:52.057379  2608 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:52.059777  2604 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:53:52.060817  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1566) took 1252 ms. Trace:
I20260709 14:53:52.061250  2604 rpcz_store.cc:276] 0709 14:53:50.808104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.037941 (+1229837us) service_pool.cc:224] Handling call
0709 14:53:52.059137 (+ 21196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.060422 (+  1285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.060729 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:51.994925  8239 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.068122  2681 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:52.068795  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1464) took 1181 ms. Trace:
I20260709 14:53:52.069060  2681 rpcz_store.cc:276] 0709 14:53:50.887087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.697121 (+810034us) service_pool.cc:224] Handling call
0709 14:53:52.068733 (+371612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.071602  2681 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:52.073408  8180 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:52.074494  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.076980  2681 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 14:53:52.078131  7717 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.080513  7717 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.078706  7991 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.083587  7991 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.084312  7717 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.085513  2681 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 6
I20260709 14:53:52.085734  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1560) took 2756 ms. Trace:
I20260709 14:53:52.086282  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.087450  8180 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:52.086428  2603 rpcz_store.cc:276] 0709 14:53:49.329482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.638979 (+309497us) service_pool.cc:224] Handling call
0709 14:53:51.798125 (+2159146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.799377 (+  1252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.085397 (+286020us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b540050ffd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.085631 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4964633344}
I20260709 14:53:52.088497  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.089771  8180 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:52.090307  8180 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.090749  8180 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.093456  7439 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:52.095875  7997 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.099962  7991 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:52.078527  7916 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.226s	user 0.015s	sys 0.004s
I20260709 14:53:52.103356  7916 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.105108  2758 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:52.094554  8246 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.093s	user 0.010s	sys 0.006s
I20260709 14:53:52.107612  7916 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.108904  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 972) took 2292 ms. Trace:
I20260709 14:53:52.109452  2610 rpcz_store.cc:276] 0709 14:53:49.816814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.355897 (+539083us) service_pool.cc:224] Handling call
0709 14:53:50.356226 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.377030 (+ 20804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:50.377067 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.437877 (+ 60810us) log.cc:844] Serialized 26 byte log entry
0709 14:53:51.745787 (+1307910us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:51.745894 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.746035 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:52.093494 (+347459us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:52.108377 (+ 14883us) raft_consensus.cc:1685] finished
0709 14:53:52.108419 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.108785 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":25470,"spinlock_wait_cycles":3841280,"thread_start_us":1213419,"threads_started":1,"wal-append.queue_time_us":501818}
I20260709 14:53:52.132179  8102 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.113127  2681 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 14:53:52.135752  2681 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:52.132651  7657 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:53:52.138630  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 152617us
I20260709 14:53:52.138895  2682 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:52.141500  2686 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
W20260709 14:53:52.141674  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 150358us
I20260709 14:53:52.144412  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:52.144869  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 974) took 2327 ms. Trace:
I20260709 14:53:52.145895  7717 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:52.139652  7657 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:52.156256  8183 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:52.157028  8183 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:52.156764  7657 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.150540  7717 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:52.159389  7717 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:52.154881  8156 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.941s	user 0.015s	sys 0.006s
I20260709 14:53:52.160662  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.145406  2609 rpcz_store.cc:276] 0709 14:53:49.817339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.795921 (+1978582us) service_pool.cc:224] Handling call
0709 14:53:51.796603 (+   682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:51.797526 (+   923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:51.797556 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:51.882458 (+ 84902us) log.cc:844] Serialized 26 byte log entry
0709 14:53:52.051856 (+169398us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:52.051922 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.052040 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:52.144589 (+ 92549us) raft_consensus.cc:1685] finished
0709 14:53:52.144613 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.144772 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3659,"raft.run_cpu_time_us":529,"raft.run_wall_time_us":28641,"thread_start_us":68501,"threads_started":1,"wal-append.queue_time_us":152791}
I20260709 14:53:52.158564  7657 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:52.169545  8182 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.196455  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1628) took 3876 ms. Trace:
I20260709 14:53:52.197134  8182 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.171829  7439 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.162191  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:53:52.199594  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47530.3us
I20260709 14:53:52.200493  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:52.202651  8184 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:52.203114  8184 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.203482  8184 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:52.203665  7997 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:52.204130  7997 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.204639  7997 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:52.204813  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1482) took 2605 ms (client timeout 1741 ms). Trace:
W20260709 14:53:52.181555  2652 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.663s (SENT)
I20260709 14:53:52.181993  7977 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.182585  8208 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.183518  8106 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:52.186281  8248 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.205s	user 0.073s	sys 0.120s
W20260709 14:53:52.153613  8034 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.056s	user 0.007s	sys 0.018s
W20260709 14:53:52.188612  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 962) took 4216 ms (client timeout 3864 ms). Trace:
W20260709 14:53:52.205330  2679 rpcz_store.cc:269] 0709 14:53:49.599777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.664688 (+ 64911us) service_pool.cc:224] Handling call
0709 14:53:52.204400 (+2539712us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.204582 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480697728}
W20260709 14:53:52.155026  7976 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 2.230s	user 0.019s	sys 0.010s
W20260709 14:53:52.163913  8103 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.070s	user 0.017s	sys 0.007s
I20260709 14:53:52.196995  2751 rpcz_store.cc:276] 0709 14:53:48.319836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.474750 (+154914us) service_pool.cc:224] Handling call
0709 14:53:48.475255 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:50.298813 (+1823558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.195804 (+1896991us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54002058d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.196183 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4192066304}
W20260709 14:53:52.217113  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52371us
W20260709 14:53:52.214069  2607 rpcz_store.cc:269] 0709 14:53:47.971789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.460643 (+488854us) service_pool.cc:224] Handling call
0709 14:53:52.188033 (+3727390us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400296dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.188365 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492461952}
I20260709 14:53:52.215054  8183 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.204125  8182 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:52.228683  2683 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 14:53:52.209764  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:52.242127  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 998) took 1642 ms. Trace:
W20260709 14:53:52.247009  7932 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.263s	user 0.012s	sys 0.015s
I20260709 14:53:52.247747  7932 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:52.249166  8221 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.523s	user 0.000s	sys 0.016s
I20260709 14:53:52.251382  2605 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:52.253031  2752 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:52.255085  8214 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.255652  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1479) took 2738 ms. Trace:
I20260709 14:53:52.241573  8106 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:52.256107  2684 rpcz_store.cc:276] 0709 14:53:49.516777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.516968 (+   191us) service_pool.cc:224] Handling call
0709 14:53:49.517412 (+   444us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:49.890408 (+372996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:49.890442 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:49.893161 (+  2719us) log.cc:844] Serialized 52 byte log entry
0709 14:53:50.291022 (+397861us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:50.291072 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.291185 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:52.248052 (+1956867us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:52.255056 (+  7004us) raft_consensus.cc:1685] finished
0709 14:53:52.255085 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.255436 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84043,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":14214528,"thread_start_us":688468,"threads_started":2,"wal-append.queue_time_us":653610}
I20260709 14:53:52.211840  8180 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.242547  2683 rpcz_store.cc:276] 0709 14:53:50.595481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.596152 (+   671us) service_pool.cc:224] Handling call
0709 14:53:52.228148 (+1631996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.229364 (+  1216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.230745 (+  1381us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400444650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.237292 (+  6547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3701292416}
W20260709 14:53:52.245427  7598 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.006s
W20260709 14:53:52.272245  8236 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.346s	user 0.015s	sys 0.013s
I20260709 14:53:52.267488  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.281761  8158 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:52.263190  8106 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:52.284461  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.286602  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.287194  8103 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:53:52.288858  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1000) took 1660 ms. Trace:
I20260709 14:53:52.289425  2677 rpcz_store.cc:276] 0709 14:53:50.628571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.760539 (+131968us) service_pool.cc:224] Handling call
0709 14:53:50.800520 (+ 39981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.800822 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:50.800856 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.824906 (+ 24050us) log.cc:844] Serialized 26 byte log entry
0709 14:53:50.919472 (+ 94566us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:50.919548 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:50.919755 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:52.185035 (+1265280us) spinlock_profiling.cc:243] Waited 889 us on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:52.288190 (+103155us) raft_consensus.cc:1685] finished
0709 14:53:52.288218 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.288632 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044672,"thread_start_us":19778,"threads_started":1,"wal-append.queue_time_us":109196}
I20260709 14:53:52.288287  8103 consensus_queue.cc:1243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 14:53:52.291009  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.289435  7932 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.291958  3543 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.293211  2751 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 14:53:52.303210  8219 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.759s	user 0.005s	sys 0.012s
I20260709 14:53:52.308619  8051 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.303038  8199 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:52.308216  8180 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.300750  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.320159  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.319600  8051 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.283299  7717 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:52.328742  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 147272us
I20260709 14:53:52.255719  8214 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.281068  7598 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:52.327361  3540 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.226255  7976 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.328888  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.322037  8102 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.330703  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:52.351670  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.353636  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:52.342720  8183 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.353009  3537 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.355355  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:52.358085  2756 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.354311  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.359483  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.363481  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.367237  3541 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:53:52.368466  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:52.371737  2759 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:52.374310  8090 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 14:53:52.365475  7912 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.009s
I20260709 14:53:52.375831  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.383950  3543 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:52.352658  8102 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.377185  8090 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.400422  8081 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:52.400974  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.401257  8081 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.401793  2751 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:53:52.402990  8184 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.403766  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.405419  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.405846  8184 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.406145  2751 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:52.382586  7912 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.407466  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.408286  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:52.409400  3547 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.409649  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.410110  7976 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:52.411667  2751 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:53:52.413081  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.414108  2603 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:52.414479  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1630) took 3699 ms (client timeout 3319 ms). Trace:
I20260709 14:53:52.360971  8051 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:52.414976  2752 rpcz_store.cc:269] 0709 14:53:48.714690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:48.753303 (+ 38613us) service_pool.cc:224] Handling call
0709 14:53:52.414197 (+3660894us) spinlock_profiling.cc:243] Waited 3.48 s on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.414372 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8006798592}
I20260709 14:53:52.416918  2752 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:52.417025  8051 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.417603  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1002) took 1874 ms. Trace:
I20260709 14:53:52.417632  8051 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.418818  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.409431  8183 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:52.421113  8183 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.421777  8183 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.422811  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:52.423554  8261 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 14:53:52.426141  8066 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.426990  8066 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.427578  3543 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:53:52.427796  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.429116  8106 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.429627  3536 pending_rounds.cc:85] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 5
I20260709 14:53:52.430080  3536 pending_rounds.cc:107] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 14:53:52.429747  8106 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.430394  3536 raft_consensus.cc:2889] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:52.432263  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:52.433080  3541 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 14:53:52.433984  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.434160  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:52.434383  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:52.434751  2607 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:52.435016  2679 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.435632  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.436288  2610 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:52.436462  3546 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:52.435125  2684 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.437290  7976 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:52.437693  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.421841  2753 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:53:52.420532  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.438691  2604 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 14:53:52.433336  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.418105  2752 rpcz_store.cc:276] 0709 14:53:50.542813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.415985 (+1873172us) service_pool.cc:224] Handling call
0709 14:53:52.416475 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.417281 (+   806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.417516 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.437325  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.444224  2610 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:53:52.442858  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76492.3us
I20260709 14:53:52.413125  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 993) took 1953 ms. Trace:
I20260709 14:53:52.445855  2751 rpcz_store.cc:276] 0709 14:53:50.459851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.410689 (+1950838us) service_pool.cc:224] Handling call
0709 14:53:52.411245 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.412829 (+  1584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.413044 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.418314  8051 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.440073  8158 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:52.471877  2683 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:52.472317  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:52.472820  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.468516  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.475179  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.477504  3547 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
W20260709 14:53:52.484336  8262 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 14:53:52.484812  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.485829  2680 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:52.486918  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1636) took 2319 ms. Trace:
I20260709 14:53:52.487509  2755 rpcz_store.cc:276] 0709 14:53:50.167476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.167732 (+   256us) service_pool.cc:224] Handling call
0709 14:53:50.168194 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.376579 (+1208385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.486388 (+1109809us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54000e6354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.486668 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727393536}
I20260709 14:53:52.491003  8151 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.491683  7912 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.491535  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.491940  8151 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.493135  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.494412  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.495232  2681 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:52.495348  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.438071  7976 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.496124  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.498111  3542 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:52.499239  2677 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:52.500304  2610 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:52.503381  2755 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:52.484165  8102 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.522293  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.523285  2682 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.525357  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:52.526585  2680 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.521566  2607 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:52.529807  2752 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:52.511348  7032 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.536515  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.494441  8151 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.539965  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1647) took 1590 ms. Trace:
I20260709 14:53:52.543182  2752 rpcz_store.cc:276] 0709 14:53:50.940830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.471436 (+1530606us) service_pool.cc:224] Handling call
0709 14:53:52.512745 (+ 41309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.530471 (+ 17726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.530776 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:52.545454  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:52.545672  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.546577  3547 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:52.547626  3542 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.496889  7997 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.553357  7997 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.546270  3546 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.549904  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:52.549321  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.561136  2603 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:52.562103  7912 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:52.562160  2610 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:52.591077  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49078.6us
W20260709 14:53:52.592765  2726 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.707s (SENT)
I20260709 14:53:52.564158  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:52.594638  2681 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:52.591157  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:52.599184  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.600301  8066 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.604487  2752 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.607324  8189 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:53:52.610913  8051 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.600399  2603 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:52.563648  7717 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:52.594167  7598 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.623939  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.624370  7976 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.624788  2603 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:52.555946  7032 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:52.625861  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.627449  7032 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.631685  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.635068  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:52.647420  7717 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.648113  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.649405  8090 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:52.652611  8253 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.884s	user 0.001s	sys 0.000s
I20260709 14:53:52.653371  8205 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:52.653918  8205 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:52.654515  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1511) took 2290 ms. Trace:
I20260709 14:53:52.655668  2603 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:52.655726  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1503) took 2909 ms. Trace:
I20260709 14:53:52.657927  2753 rpcz_store.cc:276] 0709 14:53:49.746752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.943749 (+196997us) service_pool.cc:224] Handling call
0709 14:53:52.421199 (+2477450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.438558 (+ 17359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.655424 (+216866us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b54003363d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.655630 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5696642816}
I20260709 14:53:52.659044  2603 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:52.660032  2753 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 14:53:52.660491  2753 pending_rounds.cc:85] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 5
I20260709 14:53:52.660805  2753 pending_rounds.cc:107] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 14:53:52.655144  8205 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.662170  8205 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:52.663044  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 126143us
I20260709 14:53:52.661077  2753 raft_consensus.cc:2889] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:52.664091  2753 pending_rounds.cc:107] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 14:53:52.654976  2757 rpcz_store.cc:276] 0709 14:53:50.364105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.925625 (+561520us) service_pool.cc:224] Handling call
0709 14:53:50.926003 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:50.935590 (+  9587us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:53:50.935631 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:50.936170 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 14:53:51.904321 (+968151us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:51.967075 (+ 62754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:51.971261 (+  4186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:52.651792 (+680531us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:52.653984 (+  2192us) raft_consensus.cc:1685] finished
0709 14:53:52.654019 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.654389 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8884,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":23649664,"thread_start_us":843452,"threads_started":1,"wal-append.queue_time_us":795533}
I20260709 14:53:52.664624  8066 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:52.670475  2753 raft_consensus.cc:2889] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:52.661044  2603 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:52.665053  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:52.672325  7976 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:52.672921  7976 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.673322  7976 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:52.675319  8106 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.681880  8214 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:52.685981  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1502) took 2939 ms. Trace:
I20260709 14:53:52.686517  2760 rpcz_store.cc:276] 0709 14:53:49.746437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.746729 (+   292us) service_pool.cc:224] Handling call
0709 14:53:49.747071 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:50.728710 (+981639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.685453 (+1956743us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b540040ecd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.685770 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898879360}
I20260709 14:53:52.678010  8090 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:52.691556  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1513) took 2327 ms (client timeout 1870 ms). Trace:
W20260709 14:53:52.692157  2758 rpcz_store.cc:269] 0709 14:53:50.364428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.443159 (+ 78731us) service_pool.cc:224] Handling call
0709 14:53:52.691218 (+2248059us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.691464 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3702318848}
I20260709 14:53:52.703943  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1638) took 2527 ms. Trace:
I20260709 14:53:52.704660  2759 rpcz_store.cc:276] 0709 14:53:50.176580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.307783 (+131203us) service_pool.cc:224] Handling call
0709 14:53:50.339213 (+ 31430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.372493 (+2033280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.703418 (+330925us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.703828 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4645437184}
I20260709 14:53:52.707628  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1570) took 1835 ms. Trace:
W20260709 14:53:52.710682  2606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1642) took 2897 ms (client timeout 2086 ms). Trace:
W20260709 14:53:52.711301  2606 rpcz_store.cc:269] 0709 14:53:49.812697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.812968 (+   271us) service_pool.cc:224] Handling call
0709 14:53:52.710134 (+2897166us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b5400346554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.710439 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225909120}
I20260709 14:53:52.717233  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:52.708297  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.719123  8169 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:52.719619  8169 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.720032  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.708230  2608 rpcz_store.cc:276] 0709 14:53:50.872201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.018848 (+1146647us) service_pool.cc:224] Handling call
0709 14:53:52.035523 (+ 16675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.058206 (+ 22683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.707351 (+649145us) spinlock_profiling.cc:243] Waited 9.62 ms on lock 0x7b54004808d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.707518 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22119296}
I20260709 14:53:52.714395  2751 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:52.720001  8169 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:52.688542  2760 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:52.728042  2759 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:52.727679  8106 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:53:52.728752  8184 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:52.729251  8051 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:52.729866  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1466) took 1827 ms. Trace:
I20260709 14:53:52.751493  2686 rpcz_store.cc:276] 0709 14:53:50.902643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.881590 (+978947us) service_pool.cc:224] Handling call
0709 14:53:52.141290 (+259700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.141981 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.729604 (+587623us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400268c50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.729793 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388217216}
W20260709 14:53:52.714511  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1645) took 2899 ms (client timeout 2107 ms). Trace:
I20260709 14:53:52.752367  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:52.736176  2684 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:52.747141  2752 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:53:52.747376  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 129793us
W20260709 14:53:52.756318  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 163639us
W20260709 14:53:52.757941  8245 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.676s	user 0.013s	sys 0.007s
I20260709 14:53:52.759964  7690 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:52.762948  7657 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:52.765237  7657 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:52.768438  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:52.770251  8184 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:52.770809  8184 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.772050  8184 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:52.773964  8184 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.775108  8051 consensus_queue.cc:1243] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:52.775871  8051 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:52.776273  8051 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.777648  8051 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.780282  8081 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:52.787078  8151 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:53:52.728858  7598 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:53:52.752863  2611 rpcz_store.cc:269] 0709 14:53:49.814975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.167341 (+352366us) service_pool.cc:224] Handling call
0709 14:53:52.714016 (+2546675us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54001ebc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:52.714240 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167349632}
I20260709 14:53:52.812076  3542 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:52.809362  2607 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:52.810546  2753 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 14:53:52.817724  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57600.3us
I20260709 14:53:52.732275  8066 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:52.816886  8233 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:53:52.835275  7968 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 14:53:52.848860  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.746368  7997 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 14:53:52.853015  7997 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:52.853509  7997 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.853626  7834 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:52.855087  7834 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:52.855592  7834 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:52.750531  8196 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 14:53:52.750855  8214 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 14:53:52.867159  8051 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:52.869601  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.870579  2610 raft_consensus.cc:2376] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:52.872138  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.873113  2608 raft_consensus.cc:2376] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19. Re-sending same reply.
I20260709 14:53:52.874536  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:52.875798  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.876447  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:52.877293  2603 raft_consensus.cc:2376] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12. Re-sending same reply.
I20260709 14:53:52.875882  2606 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:52.883570  8239 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:52.885613  8199 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:53:52.822150  2756 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:52.823953  2679 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:52.829942  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 992) took 2373 ms. Trace:
W20260709 14:53:52.889601  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 121279us
I20260709 14:53:52.891901  3547 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:52.865669  8183 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:52.895033  2751 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 14:53:52.889741  2754 rpcz_store.cc:276] 0709 14:53:50.456977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:50.936944 (+479967us) service_pool.cc:224] Handling call
0709 14:53:50.958203 (+ 21259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:51.699617 (+741414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.829453 (+1129836us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:52.829840 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958847872}
W20260709 14:53:52.896826  3540 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 14:53:52.897532  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:52.898651  7948 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:52.905592  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:52.907593  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.911480  2752 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:52.913059  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:52.882560  8102 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:52.916710  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 164484us
I20260709 14:53:52.918026  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:52.919148  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:52.900329  8199 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:52.942584  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:51.312227   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:52.949904  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.955714  2752 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:52.960736  2754 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 14:53:52.962178  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.919091  8102 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:52.929430  8199 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:52.920799  7932 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:52.965533  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1461) took 2094 ms. Trace:
W20260709 14:53:52.970479  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65069.5us
W20260709 14:53:52.977068  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:52.946197  7032 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:52.999223  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:52.956773  8034 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:52.914490  8033 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 14:53:52.957573  8189 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:52.970075  8239 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:52.968212  8102 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:52.970197  8158 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:52.931511  7912 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 14:53:52.984624  7968 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:53.026106  8062 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:53.030936  2576 connection.cc:256] RPC call timeout handler was delayed by -1.4348393850001457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 14:53:53.031689  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1783435450000752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 14:53:53.032289  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0793435450000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
I20260709 14:53:53.015069  8239 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:53.035456  8239 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:53.041793  8179 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:53.042894  8179 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:53.043586  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.045928  2606 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 14:53:53.013712  7932 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:53.046263  8179 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:53.048332  7932 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:53.064023  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:53.065737  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:53.066807  3537 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:53.068862  3545 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: []
I20260709 14:53:53.031153  8189 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:53.010262  8214 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:53.059680  3538 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:53.060557  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.061439  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.078644  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:53.062225  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.049687  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:53.079680  2752 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:53:53.079787  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 91969.2us
I20260709 14:53:53.077353  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:53.084661  7690 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:53.085213  7690 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:53.018690  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:53.080742  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:53.089568  7690 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:53.057189  7717 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:53.073585  7948 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:53:53.106788  7948 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:53.106455  7972 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:53.107479  7972 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:53.107575  8151 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:53.107923  7972 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:53.108268  8151 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:53.111085  2608 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:53:53.117755  2754 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:53.102664  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.122507  2751 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:53.123440  3545 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 14:53:53.111965  8081 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 14:53:53.108034  7948 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:53.129598  8081 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:53.130038  8081 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:53.077944  8214 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:53.143419  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.144454  3538 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:53.145167  7834 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:53.143548  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.094846  8169 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:53.146729  2603 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:53.147212  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1517) took 1150 ms. Trace:
I20260709 14:53:53.148744  8169 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:53.147665  2755 rpcz_store.cc:276] 0709 14:53:51.997217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.540572 (+543355us) service_pool.cc:224] Handling call
0709 14:53:52.567293 (+ 26721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.635603 (+ 68310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:52.635663 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:52.675261 (+ 39598us) log.cc:844] Serialized 26 byte log entry
0709 14:53:52.943568 (+268307us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:52.943628 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.943714 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:53.146909 (+203195us) raft_consensus.cc:1685] finished
0709 14:53:53.146937 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:53.147110 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":250762,"threads_started":1,"wal-append.queue_time_us":468800}
I20260709 14:53:53.150161  7834 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:53.142102  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:53.171525  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56095.8us
W20260709 14:53:53.177166  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 421033us
I20260709 14:53:53.072947  7968 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:53.181354  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:53.074808  8158 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:53.085106  7945 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 14:53:53.186729  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.192173  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:53.157068  8214 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:53.187482  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.265410  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:53.275738  2609 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 14:53:53.278594  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.138055  8180 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.125s
I20260709 14:53:53.179450  8184 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:53.279814  3537 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:53.182365  8169 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:53.299413  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:53.309011  2603 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:53.312390  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.318810  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:53.331032  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 148330us
I20260709 14:53:53.361056  2603 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:53:53.370429  2756 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:53.324369  2754 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:53.302551  8180 consensus_queue.cc:1243] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 14:53:53.300376  8151 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:53.361764  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 399812us
I20260709 14:53:53.306635  8169 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:53.420949  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:53.422398  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.423290  3537 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:53:53.372939  8205 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.701s	user 0.017s	sys 0.004s
I20260709 14:53:53.433378  2752 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:53:53.189230  8081 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:53.433379  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:53.465665  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:53.423480  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:53.311901  8184 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:53.468535  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:53.190595  8199 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:53.477818  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.480456  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:53.421229  8169 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:53:53.483217  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.019s	sys 0.006s
I20260709 14:53:53.424721  8151 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:53:53.484193  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 477158us
I20260709 14:53:53.485759  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:52.975975  2685 rpcz_store.cc:276] 0709 14:53:50.871316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.829643 (+958327us) service_pool.cc:224] Handling call
0709 14:53:51.968014 (+138371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:52.010787 (+ 42773us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:53:52.511078 (+500291us) log.cc:844] Serialized 14 byte log entry
0709 14:53:52.951305 (+440227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:52.951404 (+    99us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:53:52.965127 (+ 13723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.965247 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:52.965451 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":292094,"prepare.queue_time_us":298365,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":37,"raft.queue_time_us":23235,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7920,"thread_start_us":543730,"threads_started":2}
W20260709 14:53:53.522210  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98848.5us
I20260709 14:53:53.532975  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:53.485314  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:53.483846  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.020s	sys 0.006s
I20260709 14:53:53.544214  2609 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:53.557062  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:53.557502  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 385128us
I20260709 14:53:53.565569  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:53.423050  7717 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 14:53:53.573406  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:53.576839  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:53.578339  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:53.541656  8199 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:53.586570  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:53.587350  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.589172  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.590142  2679 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:53:53.569288  7032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.023s	sys 0.067s
I20260709 14:53:53.591346  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.593253  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:53.590656  3540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.033s	sys 0.040s
W20260709 14:53:53.599117  3540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.034s	sys 0.040s
W20260709 14:53:53.599918  8278 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 14:53:53.600320  8196 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:53.600454  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.600910  8196 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:53.599637  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.602073  2686 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:53.602061  8196 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:53.603955  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66625.9us
I20260709 14:53:53.607024  8285 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:53.609901  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:53.610374  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1659) took 1196 ms. Trace:
I20260709 14:53:53.611074  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.604077  8196 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:53.596161  7032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.024s	sys 0.067s
I20260709 14:53:53.631703  2686 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:53.610972  2604 rpcz_store.cc:276] 0709 14:53:52.414174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.460529 (+ 46355us) service_pool.cc:224] Handling call
0709 14:53:52.503625 (+ 43096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.538514 (+ 34889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:52.538595 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:52.562464 (+ 23869us) log.cc:844] Serialized 26 byte log entry
0709 14:53:52.915216 (+352752us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:52.915274 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.915364 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:53.610039 (+694675us) raft_consensus.cc:1685] finished
0709 14:53:53.610077 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:53.610291 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10019,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":19635,"thread_start_us":352170,"threads_started":1,"wal-append.queue_time_us":735357}
I20260709 14:53:53.679517  2677 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:53.681872  3536 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:53.712914  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.713706  2679 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:53.718890  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.720924  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.725349  2685 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:53.729521  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:53.680161  8151 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:53.736171  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129575us
I20260709 14:53:53.627712  8081 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:53.730623  2609 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:53:53.761986  2686 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:53.762452  2604 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:53.704783  7032 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:53.623006  7945 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 14:53:53.776572  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.778257  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.615675  7972 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:53.788343  2603 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:53:53.789403  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 265887us
I20260709 14:53:53.797598  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:53.803506  2612 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 14:53:53.802472  2605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1569) took 2971 ms (client timeout 2002 ms). Trace:
I20260709 14:53:53.769827  8189 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:53.745756  8151 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:53.845561  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48092.4us
W20260709 14:53:53.804768  2605 rpcz_store.cc:269] 0709 14:53:50.831139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:51.352430 (+521291us) service_pool.cc:224] Handling call
0709 14:53:53.802198 (+2449768us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:53.802393 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311082240}
I20260709 14:53:53.799422  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:53.804744  8051 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.937s	user 0.013s	sys 0.002s
I20260709 14:53:53.888237  2677 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:53.788501  8034 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:53:53.897367  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 364573us
I20260709 14:53:53.781456  8158 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:53.851974  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.850710  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:53.901328  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:53.788779  7968 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:53.922367  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:53.950897  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:53.926591  2608 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 14:53:53.961767  2608 pending_rounds.cc:85] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 7
I20260709 14:53:53.872061  2679 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:53.962338  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 278889us
I20260709 14:53:53.852150  8151 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:53:53.927481  7976 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.254s	user 0.026s	sys 0.000s
I20260709 14:53:53.970620  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:53.921439  2686 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:53.974068  2608 pending_rounds.cc:107] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 14:53:54.000857  2608 raft_consensus.cc:2889] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:53.901917  8284 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.480s	user 0.002s	sys 0.000s
I20260709 14:53:53.899958  8034 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:53.896134  7717 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:53.933287  8184 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:53.997663  8239 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.962s	user 0.016s	sys 0.000s
W20260709 14:53:53.935251  8279 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.501s	user 0.000s	sys 0.002s
W20260709 14:53:54.031689  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 232402us
I20260709 14:53:53.901819  8216 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.436s
W20260709 14:53:54.023195  8281 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.535s	user 0.002s	sys 0.000s
W20260709 14:53:54.033866  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83424.8us
I20260709 14:53:53.886940  7997 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.019467  8034 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:54.035377  7690 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.945s	user 0.001s	sys 0.013s
I20260709 14:53:54.051029  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:54.040539  8287 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
I20260709 14:53:54.015352  7976 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.053515  7997 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:54.051642  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:54.048365  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151192us
I20260709 14:53:54.059986  2752 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:54.023468  8295 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 14:53:54.032178  8239 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.051872  8237 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:54.046537  7945 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:54.059929  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.039773  7948 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 14:53:54.010563  7032 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.082871  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1520) took 2031 ms. Trace:
I20260709 14:53:54.086676  2757 rpcz_store.cc:276] 0709 14:53:52.054201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.666534 (+612333us) service_pool.cc:224] Handling call
0709 14:53:52.667006 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.669286 (+  2280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:52.669321 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:52.669925 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 14:53:52.942433 (+272508us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:52.942477 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:52.942554 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.082568 (+1140014us) raft_consensus.cc:1685] finished
0709 14:53:54.082597 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.082768 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1440,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"thread_start_us":272070,"threads_started":1,"wal-append.queue_time_us":601937}
I20260709 14:53:54.058585  8305 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:54.056990  8303 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:53:54.099225  8293 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 14:53:54.099653  8205 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:54.100502  2757 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:53:54.067822  7977 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:54.101217  7945 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:54.073719  8288 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:54.114687  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 521950us
I20260709 14:53:54.102896  7976 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.053805  7690 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.112982  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1535) took 1072 ms. Trace:
I20260709 14:53:54.084038  7972 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.118398  2686 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:53:54.118192  2757 rpcz_store.cc:276] 0709 14:53:53.040416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.099709 (+1059293us) service_pool.cc:224] Handling call
0709 14:53:54.100170 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:54.112535 (+ 12365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.112900 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:54.119513  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:54.104784  8305 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:54.111495  7945 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:54.120162  8286 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 14:53:54.139508  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 985) took 1214 ms. Trace:
I20260709 14:53:54.150581  2607 rpcz_store.cc:276] 0709 14:53:52.924748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.931586 (+  6838us) service_pool.cc:224] Handling call
0709 14:53:52.932208 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.939612 (+  7404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:52.939652 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:52.940350 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 14:53:53.375719 (+435369us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:53:53.786611 (+410892us) log.cc:844] Serialized 14 byte log entry
0709 14:53:54.045330 (+258719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.045422 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.139239 (+ 93817us) raft_consensus.cc:1685] finished
0709 14:53:54.139267 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.139436 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38561,"prepare.queue_time_us":566056,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":139681,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":8340,"thread_start_us":1060923,"threads_started":3,"wal-append.queue_time_us":768743}
I20260709 14:53:54.111291  7977 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:54.133579  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 94291us
W20260709 14:53:54.120759  8285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.009s
I20260709 14:53:54.156071  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1657) took 1114 ms. Trace:
I20260709 14:53:54.156553  2757 rpcz_store.cc:276] 0709 14:53:53.042059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.121635 (+1079576us) service_pool.cc:224] Handling call
0709 14:53:54.139635 (+ 18000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.155602 (+ 15967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:54.155636 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:54.155693 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:54.155741 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.155833 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.156000 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:54.149866  7945 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:54.155284  8292 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 14:53:54.137835  8311 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:54.133546  7717 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:54.164911  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.144905  8312 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:54.158255  8285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.006s	sys 0.009s
I20260709 14:53:54.138924  8309 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:54.155169  7977 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:54.174885  8298 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 14:53:54.170099  8319 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 14:53:54.176777  8296 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 14:53:54.176884  8183 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.176791  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1009) took 2006 ms. Trace:
W20260709 14:53:54.164688  8290 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 14:53:54.132246  8288 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.164321  7972 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:54.188812  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1538) took 1013 ms. Trace:
I20260709 14:53:54.202592  8304 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:53:54.203146  8305 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.203356  7717 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.187316  8285 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:54.226898  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1527) took 1172 ms. Trace:
I20260709 14:53:54.227496  3546 rpcz_store.cc:276] 0709 14:53:53.054450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.061142 (+  6692us) service_pool.cc:224] Handling call
0709 14:53:54.226577 (+1165435us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54002eb8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.226758 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876754816}
I20260709 14:53:54.200975  8309 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.228704  8321 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:54.229705  8313 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.129s	user 0.000s	sys 0.004s
I20260709 14:53:54.229383  8321 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.230365  8309 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.201301  8312 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.232169  3546 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:53:54.232842  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1542) took 1045 ms. Trace:
I20260709 14:53:54.201680  8295 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 14:53:54.233476  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.235946  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1529) took 1180 ms. Trace:
I20260709 14:53:54.236430  3543 rpcz_store.cc:276] 0709 14:53:53.055074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.062733 (+  7659us) service_pool.cc:224] Handling call
0709 14:53:53.063209 (+   476us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:53.063773 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:53.063811 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:53.064656 (+   845us) log.cc:844] Serialized 52 byte log entry
0709 14:53:53.474261 (+409605us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:53.802783 (+328522us) log.cc:844] Serialized 14 byte log entry
0709 14:53:53.803061 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:53.803153 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.235624 (+432471us) raft_consensus.cc:1685] finished
0709 14:53:54.235650 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.235845 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39266,"prepare.queue_time_us":552713,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":105858,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":20143,"thread_start_us":661566,"threads_started":2,"wal-append.queue_time_us":681133}
I20260709 14:53:54.225669  8288 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.225768  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1656) took 1812 ms. Trace:
I20260709 14:53:54.239542  2612 rpcz_store.cc:276] 0709 14:53:52.413279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.418994 (+  5715us) service_pool.cc:224] Handling call
0709 14:53:53.803160 (+1384166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:53.823176 (+ 20016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.225465 (+402289us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003791d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.225684 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3182184192}
W20260709 14:53:54.204106  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102428us
W20260709 14:53:54.210347  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.050s (SENT)
I20260709 14:53:54.212172  2758 rpcz_store.cc:276] 0709 14:53:52.170601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.693224 (+522623us) service_pool.cc:224] Handling call
0709 14:53:52.693709 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.715325 (+ 21616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:52.715362 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:52.716068 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 14:53:53.541895 (+825827us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:53.541962 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:53.542072 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.176531 (+634459us) raft_consensus.cc:1685] finished
0709 14:53:54.176555 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.176717 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5193,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":819494,"threads_started":1,"wal-append.queue_time_us":995173}
I20260709 14:53:54.242026  3543 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:53:54.242090  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1524) took 1188 ms. Trace:
I20260709 14:53:54.227262  8285 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.249476  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:54.217245  3536 rpcz_store.cc:276] 0709 14:53:53.159698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.737865 (+578167us) service_pool.cc:224] Handling call
0709 14:53:53.779269 (+ 41404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:53.788787 (+  9518us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:53:54.063770 (+274983us) log.cc:844] Serialized 14 byte log entry
0709 14:53:54.172021 (+108251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:54.172108 (+    87us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:54.172175 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.172278 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.172491 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1352,"prepare.queue_time_us":277251,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":6198,"raft.queue_time_us":1514,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":304974,"threads_started":2}
I20260709 14:53:54.243345  3547 rpcz_store.cc:276] 0709 14:53:53.053500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.056067 (+  2567us) service_pool.cc:224] Handling call
0709 14:53:53.056572 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:53.057088 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:53.057128 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:53.057763 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 14:53:53.557279 (+499516us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:53.557347 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:53.557436 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.241845 (+684409us) raft_consensus.cc:1685] finished
0709 14:53:54.241867 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.242025 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16146,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":96,"thread_start_us":498946,"threads_started":1,"wal-append.queue_time_us":797393}
I20260709 14:53:54.246326  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:54.233167  3546 rpcz_store.cc:276] 0709 14:53:53.187817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.231361 (+1043544us) service_pool.cc:224] Handling call
0709 14:53:54.231765 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.232554 (+   789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.232776 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:54.243583  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.252820  2682 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:54.253121  7690 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:54.254014  3412 connection.cc:256] RPC call timeout handler was delayed by -3.1358839130000433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3864s for a response
W20260709 14:53:54.254747  3412 connection.cc:256] RPC call timeout handler was delayed by -2.7383751760000905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 14:53:54.254794  8291 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:54.255587  8291 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.258283  8291 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:54.261969  8196 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.007s
I20260709 14:53:54.255149  8183 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.263509  8305 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:54.263731  8183 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:54.267343  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57053.1us
I20260709 14:53:54.267335  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:54.269719  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.272444  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.218044  7972 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:54.273711  7972 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:54.275242  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.276028  2760 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:54.276544  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1528) took 1343 ms. Trace:
I20260709 14:53:54.276989  2760 rpcz_store.cc:276] 0709 14:53:52.933572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.250283 (+1316711us) service_pool.cc:224] Handling call
0709 14:53:54.276473 (+ 26190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:54.278175  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.278294  2682 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:53:54.279181  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1660) took 1202 ms. Trace:
I20260709 14:53:54.279659  2757 rpcz_store.cc:276] 0709 14:53:53.076518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.157665 (+1081147us) service_pool.cc:224] Handling call
0709 14:53:54.166533 (+  8868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:54.178665 (+ 12132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:54.178707 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:54.179402 (+   695us) log.cc:844] Serialized 26 byte log entry
0709 14:53:54.200529 (+ 21127us) raft_consensus.cc:1645] Marking committed up to 5
0709 14:53:54.202060 (+  1531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.202177 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.278952 (+ 76775us) raft_consensus.cc:1685] finished
0709 14:53:54.278974 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.279120 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7863,"raft.queue_time_us":9303,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":20618,"threads_started":1,"wal-append.queue_time_us":71076}
I20260709 14:53:54.216091  8303 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:54.202584  7976 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:54.288021  3538 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:54.290791  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1577) took 1192 ms. Trace:
I20260709 14:53:54.291428  2606 rpcz_store.cc:276] 0709 14:53:53.098168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.098312 (+   144us) service_pool.cc:224] Handling call
0709 14:53:53.108552 (+ 10240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:53.109017 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:53.109054 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:53.109608 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 14:53:53.534375 (+424767us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:53.534444 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:53.534540 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.290512 (+755972us) raft_consensus.cc:1685] finished
0709 14:53:54.290540 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.290715 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2938,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3831,"thread_start_us":377610,"threads_started":1,"wal-append.queue_time_us":920108}
I20260709 14:53:54.218634  8304 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:54.282330  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 121018us
W20260709 14:53:54.255321  3412 connection.cc:256] RPC call timeout handler was delayed by -2.6423560990000396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 14:53:54.234615  8321 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:54.294744  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45367.9us
I20260709 14:53:54.295399  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.296056  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:54.298063  3412 connection.cc:256] RPC call timeout handler was delayed by -2.5874554629999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 14:53:54.303791  3547 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:54.303858  3412 connection.cc:256] RPC call timeout handler was delayed by -2.5631554630000437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 14:53:54.304577  3412 connection.cc:256] RPC call timeout handler was delayed by -2.414655463000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 14:53:54.305096  3412 connection.cc:256] RPC call timeout handler was delayed by -2.3417554630000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 14:53:54.305540  3412 connection.cc:256] RPC call timeout handler was delayed by -2.3228554629999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
W20260709 14:53:54.305949  3412 connection.cc:256] RPC call timeout handler was delayed by -2.255356099000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 14:53:54.306391  3412 connection.cc:256] RPC call timeout handler was delayed by -2.2139554630000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 14:53:54.167845  8205 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:54.270397  8169 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.000s
I20260709 14:53:54.266690  8183 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.303213  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:54.296943  8102 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.215s	user 0.012s	sys 0.004s
W20260709 14:53:54.306898  3412 connection.cc:256] RPC call timeout handler was delayed by -2.138355463000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
I20260709 14:53:54.313766  3536 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:54.314760  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:54.293825  7976 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.316112  8314 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 14:53:54.308118  8239 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.308440  8317 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 14:53:54.284180  8233 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:54.335124  8239 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:53:54.300868  8319 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:54.335363  8233 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.336910  8317 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:54.337495  8317 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:54.327453  3412 connection.cc:256] RPC call timeout handler was delayed by -2.0456554630000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 14:53:54.326426  8106 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:54.329231  7834 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.178s	user 0.013s	sys 0.003s
I20260709 14:53:54.329963  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:54.332999  2575 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.904s (SENT)
W20260709 14:53:54.348552  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:54.339002  3412 connection.cc:256] RPC call timeout handler was delayed by -2.020455462999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
I20260709 14:53:54.345211  3547 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:53:54.349696  3412 connection.cc:256] RPC call timeout handler was delayed by -2.0141554630000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 14:53:54.343611  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:54.350817  3412 connection.cc:256] RPC call timeout handler was delayed by -1.9844554630000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
I20260709 14:53:54.350769  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:54.352010  3412 connection.cc:256] RPC call timeout handler was delayed by -1.8188554629999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23660000000000003s for a response
W20260709 14:53:54.352658  3412 connection.cc:256] RPC call timeout handler was delayed by -1.7027554630000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24950000000000003s for a response
W20260709 14:53:54.353427  3412 connection.cc:256] RPC call timeout handler was delayed by -1.6874554629999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25120000000000003s for a response
W20260709 14:53:54.354132  3412 connection.cc:256] RPC call timeout handler was delayed by -1.6154560990000846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2812s for a response
W20260709 14:53:54.354947  3412 connection.cc:256] RPC call timeout handler was delayed by -1.4642560990000675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.298s for a response
I20260709 14:53:54.347303  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:54.361191  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1521) took 2294 ms. Trace:
I20260709 14:53:54.361930  2753 rpcz_store.cc:276] 0709 14:53:52.118131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.679624 (+561493us) service_pool.cc:224] Handling call
0709 14:53:52.792477 (+112853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:52.820213 (+ 27736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.347196 (+1526983us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400184cd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.361077 (+ 13881us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258491648}
I20260709 14:53:54.366673  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.374735  3536 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:54.369807  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.376892  2681 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:54.373483  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:54.382361  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.338567  8233 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.387543  3542 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:53:54.387385  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:54.389670  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:54.390215  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.319970  8169 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.387399  2678 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:54.392246  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49450us
I20260709 14:53:54.373695  8196 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:54.344507  8102 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.377094  8184 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:53:54.389530  8214 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.091s	user 0.009s	sys 0.007s
I20260709 14:53:54.391208  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:54.398809  2681 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 14:53:54.402455  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 69531.4us
I20260709 14:53:54.404461  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.408015  8034 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.394922  8196 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:54.413261  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:54.415164  8306 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.366083  8106 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.415352  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 986) took 1457 ms. Trace:
I20260709 14:53:54.415800  7834 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.395838  8314 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:54.373874  8317 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.451514  2754 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 14:53:54.424809  8233 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:54.421671  2753 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 12.
W20260709 14:53:54.434110  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.463421  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:54.435681  2610 rpcz_store.cc:276] 0709 14:53:52.957584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.966988 (+  9404us) service_pool.cc:224] Handling call
0709 14:53:52.979206 (+ 12218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:53.007695 (+ 28489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:53.007754 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:53.059104 (+ 51350us) log.cc:844] Serialized 26 byte log entry
0709 14:53:54.240793 (+1181689us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18002d8380. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:53:54.364876 (+124083us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:54.364928 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.365013 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.414927 (+ 49914us) raft_consensus.cc:1685] finished
0709 14:53:54.414956 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.415274 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2055,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2174,"spinlock_wait_cycles":38967552,"thread_start_us":123584,"threads_started":1,"wal-append.queue_time_us":1344051}
I20260709 14:53:54.437409  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1658) took 1395 ms. Trace:
I20260709 14:53:54.466873  2752 rpcz_store.cc:276] 0709 14:53:53.042355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.579831 (+537476us) service_pool.cc:224] Handling call
0709 14:53:54.437123 (+857292us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.437316 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070817152}
W20260709 14:53:54.418670  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1525) took 1514 ms (client timeout 1870 ms). Trace:
W20260709 14:53:54.468189  2756 rpcz_store.cc:269] 0709 14:53:52.904113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:52.909861 (+  5748us) service_pool.cc:224] Handling call
0709 14:53:54.418427 (+1508566us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.418572 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554738048}
I20260709 14:53:54.444141  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1579) took 1319 ms. Trace:
I20260709 14:53:54.392292  8303 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.461607  3536 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:54.470438  2608 rpcz_store.cc:276] 0709 14:53:53.124831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.125005 (+   174us) service_pool.cc:224] Handling call
0709 14:53:53.132683 (+  7678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.014235 (+881552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.436532 (+422297us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400371c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:54.444063 (+  7531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1770771712}
I20260709 14:53:54.474375  2753 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:54.475404  2611 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:54.446663  8314 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:54.468578  8199 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.007s
I20260709 14:53:54.482795  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.465438  8233 consensus_queue.cc:1243] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:54.434633  8214 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.469029  7968 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.492936  2686 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:53:54.495919  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.496774  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.497421  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.494953  8339 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.497866  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:54.499933  8332 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 14:53:54.499607  8196 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.516202  8196 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:54.516852  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.501026  8289 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:53:54.501808  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 127220us
I20260709 14:53:54.488905  8314 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:54.506547  2610 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:54.504577  3536 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 14:53:54.509094  2604 raft_consensus.cc:2376] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17. Re-sending same reply.
I20260709 14:53:54.501088  8106 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.525535  2759 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:54.459550  8306 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.457046  8205 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.535185  8106 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:54.536289  8214 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.536518  8106 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:54.536994  8106 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:54.537185  2753 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:53:54.538143  8034 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.537992  8106 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.484184  8317 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:54.513389  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.503664  8237 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:54.451608  7834 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.511940  2755 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:54.521801  2757 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:53:54.473335  8319 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 14:53:54.502669  8102 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.530489  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:54.505959  7976 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:54.522573  7968 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:54.472923  7977 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.512584  2608 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:54.560009  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1534) took 1520 ms. Trace:
I20260709 14:53:54.589860  2754 rpcz_store.cc:276] 0709 14:53:53.040043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.594751 (+554708us) service_pool.cc:224] Handling call
0709 14:53:53.621313 (+ 26562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:54.460350 (+839037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.559640 (+ 99290us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:54.559922 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1896942976}
I20260709 14:53:54.554795  8156 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.563920  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.537209  8169 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.571834  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.571919  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:54.575659  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.579252  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.580603  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.608908  3537 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:54.609438  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:54.582526  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 186048us
I20260709 14:53:54.610440  2610 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:54.469995  8239 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:54.598392  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 478749us
I20260709 14:53:54.602007  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:54.604661  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:54.614732  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.612778  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.614020  2755 raft_consensus.cc:2376] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:53:54.621904  2677 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:54.610627  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:54.627005  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96681us
I20260709 14:53:54.626080  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.630477  2608 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:54.616916  8051 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:54.631681  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.631852  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:54.633926  3542 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:54.634884  7032 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:54.637723  8306 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.641254  8169 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:53:54.643954  8151 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.002s
I20260709 14:53:54.644572  8151 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.645884  2683 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
W20260709 14:53:54.647534  8347 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 14:53:54.648414  8151 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.621410  8289 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:54.650048  7834 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.610203  8106 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:54.650503  7945 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.653013  7945 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:54.654026  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 82236.7us
I20260709 14:53:54.649710  8289 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.521939  8339 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.669037  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.659246  2677 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:54.607272  8062 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:54.677482  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.578917  7972 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.672472  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:54.685560  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:54.687469  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:54.688258  3542 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:54.689518  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.690099  2604 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:54.697063  8306 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:54.666580  7717 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:54.688748  8062 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:54.617848  8156 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.689417  8169 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:54.706817  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.707731  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:54.711275  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.710662  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1536) took 1638 ms. Trace:
I20260709 14:53:54.717536  2754 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:54.728425  3538 rpcz_store.cc:276] 0709 14:53:53.071927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.145423 (+ 73496us) service_pool.cc:224] Handling call
0709 14:53:53.145861 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.288893 (+1143032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.710366 (+421473us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400550354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:54.710588 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2626000768}
I20260709 14:53:54.729079  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:54.719678  2607 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:54.724756  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:54.705943  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.721635  2681 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:54.704033  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1533) took 1664 ms. Trace:
I20260709 14:53:54.732340  2759 rpcz_store.cc:276] 0709 14:53:53.039649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.050288 (+ 10639us) service_pool.cc:224] Handling call
0709 14:53:53.050642 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.526140 (+1475498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.703720 (+177580us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:54.703961 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3391552896}
I20260709 14:53:54.725858  2677 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 6
I20260709 14:53:54.735569  2677 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 14:53:54.735845  2677 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:54.705708  8350 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 14:53:54.752171  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:54.752002  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:54.703469  8302 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.602s	user 0.005s	sys 0.000s
I20260709 14:53:54.762738  3545 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:54.759927  2679 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14. Re-sending same reply.
I20260709 14:53:54.710498  7997 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:54.767699  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:54.716626  8081 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.953s	user 0.015s	sys 0.000s
W20260709 14:53:54.770354  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.707430  8306 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:54.708760  7968 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.711149  8156 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.790005  3538 raft_consensus.cc:1217] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-17.11]   Dedup: 13.9->[15.10-17.11]
I20260709 14:53:54.707315  7977 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:54.721362  7972 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:54.734023  7032 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:54.788013  2685 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:53:54.771934  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.697348  8051 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 14:53:54.710371  8289 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.793099  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.794399  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:54.719467  8339 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.798084  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:54.796734  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:54.799696  2612 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:54.804291  3537 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:53:54.799161  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 91574.4us
I20260709 14:53:54.780164  7997 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:54.788576  8306 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:54.839325  3542 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: [13.8-16.9]
I20260709 14:53:54.840490  2679 raft_consensus.cc:2376] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16. Re-sending same reply.
I20260709 14:53:54.790642  3538 pending_rounds.cc:85] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 9
W20260709 14:53:54.843037  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 114092us
I20260709 14:53:54.844574  3538 pending_rounds.cc:107] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 14:53:54.844972  3538 raft_consensus.cc:2889] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:53:54.845983  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:54.846634  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.771337  8184 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 14:53:54.722112  8239 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.867973  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.715394  8169 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.857036  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.857841  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1523) took 1843 ms. Trace:
I20260709 14:53:54.858230  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:54.859387  8351 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 14:53:54.859761  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:54.867204  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:54.852188  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.881036  2605 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:54.799443  7968 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 14:53:54.881069  3543 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:54.869815  3540 rpcz_store.cc:276] 0709 14:53:53.014475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.020633 (+  6158us) service_pool.cc:224] Handling call
0709 14:53:53.073590 (+ 52957us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:53.074217 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:53.599638 (+525421us) log.cc:844] Serialized 14 byte log entry
0709 14:53:54.295044 (+695406us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b180049ff60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a
0709 14:53:54.314752 (+ 19708us) log.cc:844] Serialized 14 byte log entry
0709 14:53:54.314969 (+   217us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:54.315592 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 14:53:54.315772 (+   180us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:54.315980 (+   208us) log.cc:844] Serialized 14 byte log entry
0709 14:53:54.317105 (+  1125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:54.317291 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:54.702911 (+385620us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b24000f1238. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:53:54.857535 (+154624us) raft_consensus.cc:1685] finished
0709 14:53:54.857564 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:54.857760 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19165,"prepare.queue_time_us":893202,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":92,"raft.queue_time_us":51641,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":26837,"spinlock_wait_cycles":89187840,"thread_start_us":938884,"threads_started":2,"wal-append.queue_time_us":501553}
I20260709 14:53:54.849315  8339 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:53:54.845793  7997 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:54.857882  8183 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:54.815814  8305 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 14:53:54.903697  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.905368  2609 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:53:54.907884  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49762us
I20260709 14:53:54.798861  8180 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 14:53:54.916956  8051 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:54.917856  8051 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.920148  8306 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:54.923419  8305 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:54.923964  8305 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:54.924981  8305 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.913198  8289 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:54.926319  8305 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:54.910072  8357 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 14:53:54.927716  7032 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:54.929672  8312 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:54.936096  7968 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:53:54.937021  7968 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:54.926744  8289 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:53:54.944512  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 147836us
I20260709 14:53:54.949438  2604 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:53:54.950932  3545 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:54.952335  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:54.953187  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 218160us
I20260709 14:53:54.959358  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:54.960212  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:54.961360  8319 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 14:53:54.961905  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.968092  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:54.968408  7972 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:54.969488  7972 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:54.969897  7972 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:54.968182  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:54.970459  2608 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:54.972030  2605 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:54.961581  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:54.973107  2610 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:54.966338  2612 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
W20260709 14:53:54.979030  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 209550us
I20260709 14:53:54.922677  8306 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.938308  7968 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:54.985855  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:54.986359  2726 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.894s (SENT)
I20260709 14:53:54.986184  8289 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.987419  8051 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:54.991436  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.021551  2604 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:53:54.991734  2611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1587) took 1669 ms (client timeout 2178 ms). Trace:
W20260709 14:53:55.022030  2611 rpcz_store.cc:269] 0709 14:53:53.322328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.360230 (+ 37902us) service_pool.cc:224] Handling call
0709 14:53:54.991430 (+1631200us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:54.991623 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167635200}
I20260709 14:53:54.992367  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:54.994942  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:54.970583  7972 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:54.996047  7690 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:54.997949  8311 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:53:55.000840  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:55.000857  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.036904  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.001688  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.003548  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.004683  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.039484  2608 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 14:53:55.006652  8372 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:53:55.011890  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43906.6us
I20260709 14:53:55.016281  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.017067  2609 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:53:55.018034  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.020015  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.025517  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.025743  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.041996  2678 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:55.044450  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.045825  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.010918  8183 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:55.048028  2682 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:55.020824  8169 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:55.049842  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:55.052057  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 98162.9us
I20260709 14:53:55.057312  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.058213  2605 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:55.054605  8372 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:55.042872  8196 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:55.059742  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.060256  8372 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:55.060756  7972 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:55.062817  8183 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:55.060046  8368 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 14:53:55.030529  8339 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:55.060663  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 85479.4us
I20260709 14:53:55.084618  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:55.083386  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.088796  2608 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 14:53:55.088792  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:55.089612  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:55.072602  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.060905  2612 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:55.091673  2609 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:55.067847  2684 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:53:55.053254  8169 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:55.095690  8169 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:55.095407  8339 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:55.104029  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.055190  8051 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:55.113361  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.083950  7690 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:55.100795  8358 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:55.056777  7977 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:55.126601  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.058118  8306 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:55.134794  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:55.162366  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52178.3us
W20260709 14:53:55.118921  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 132600us
I20260709 14:53:55.166188  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:55.170130  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:55.114148  8365 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 14:53:55.171437  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.175746  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:55.175060  2608 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:55.121935  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.114882  7968 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:55.122138  8106 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:55.183868  2612 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 14:53:55.119029  8369 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.002s
I20260709 14:53:55.126910  8371 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:55.124990  7717 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:55.056826  8062 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:55.258563  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 132063us
I20260709 14:53:55.165762  8358 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:55.313740  2608 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:55.315068  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.315971  2679 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:55.115501  7690 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:55.301229  8311 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:55.302678  8306 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:55.328079  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:55.329128  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:55.331620  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 209814us
I20260709 14:53:55.332733  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.314126  7717 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:55.174973  8051 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 14:53:55.341418  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.313697  8317 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 14:53:55.379563  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.363747  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.388788  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:55.355911  8370 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:55.398309  2677 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:55.405862  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.337563  8375 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 14:53:55.412457  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100028us
I20260709 14:53:55.335220  8306 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:55.418442  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:55.419768  8357 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:55.420249  8357 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:55.420650  8357 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:55.422591  7690 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:55.423441  7690 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:55.426229  8196 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:55.426908  8196 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:55.428681  8062 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:55.429294  8196 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:55.430455  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.431205  2680 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:55.431461  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:55.432184  3538 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:55.432370  8363 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:53:55.432796  8363 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:55.415958  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:55.363086  8295 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:53:55.439036  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:55.395283  8311 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:53:55.441496  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117593us
I20260709 14:53:55.442312  2679 raft_consensus.cc:2376] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17. Re-sending same reply.
I20260709 14:53:55.444139  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:55.435597  8233 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:55.415450  8375 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:55.463654  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.462756  3538 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:55.459522  2681 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 14:53:55.466473  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1010) took 1724 ms (client timeout 1822 ms). Trace:
I20260709 14:53:55.460399  2680 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:55.470242  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:55.473163  8305 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.546s	user 0.014s	sys 0.000s
W20260709 14:53:55.475910  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62181.8us
I20260709 14:53:55.477910  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.456600  8375 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:55.483842  7997 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 14:53:55.484562  7997 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:55.487138  2610 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:53:55.466979  2685 rpcz_store.cc:269] 0709 14:53:53.736007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:53.736290 (+   283us) service_pool.cc:224] Handling call
0709 14:53:55.456108 (+1719818us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001c0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:55.456276 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2303263104}
I20260709 14:53:55.467269  8295 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 14:53:55.495106  8311 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 14:53:55.495013  8106 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:55.507565  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62341.4us
I20260709 14:53:55.508599  8106 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:55.509961  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:55.511924  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1505) took 1266 ms. Trace:
I20260709 14:53:55.512553  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1661) took 2329 ms. Trace:
I20260709 14:53:55.525666  2686 rpcz_store.cc:276] 0709 14:53:54.245767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.245952 (+   185us) service_pool.cc:224] Handling call
0709 14:53:55.511681 (+1265729us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002b3c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:55.511854 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553505152}
I20260709 14:53:55.525718  2757 rpcz_store.cc:276] 0709 14:53:53.183103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.282529 (+1099426us) service_pool.cc:224] Handling call
0709 14:53:54.298619 (+ 16090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.578131 (+279512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:55.512257 (+934126us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:55.512489 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512732032}
W20260709 14:53:55.476007  8314 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.920s	user 0.015s	sys 0.000s
I20260709 14:53:55.526118  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.530130  8184 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 14:53:55.536011  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58213.7us
I20260709 14:53:55.511215  8289 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 14:53:55.541767  8289 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:55.531471  8062 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 14:53:55.520176  8374 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 14:53:55.542934  8289 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.544299  8374 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:53:55.544782  8374 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:55.545755  8374 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.546861  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1017) took 1485 ms. Trace:
I20260709 14:53:55.547318  2683 rpcz_store.cc:276] 0709 14:53:54.061425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.061713 (+   288us) service_pool.cc:224] Handling call
0709 14:53:54.080388 (+ 18675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:54.646320 (+565932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:55.546502 (+900182us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:55.546754 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299965184}
I20260709 14:53:55.510387  7997 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:55.551002  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.535384  8314 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:55.554128  7690 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:53:55.514253  8306 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:55.514490  8196 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 14:53:55.555626  8314 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:55.556427  8295 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:53:55.511543  8051 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 14:53:55.556756  2751 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:55.556692  8081 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:55.557356  8081 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:55.557410  8051 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:55.557963  8051 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:55.558964  8051 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.560007  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1676) took 1051 ms. Trace:
I20260709 14:53:55.560439  2607 rpcz_store.cc:276] 0709 14:53:54.508791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.509446 (+   655us) service_pool.cc:224] Handling call
0709 14:53:55.559768 (+1050322us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540016d5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:55.559932 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376978176}
W20260709 14:53:55.539901  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.556589  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.505069  8317 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 14:53:55.577605  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.455741  8304 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.575632  8374 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:55.577770  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:55.586359  7912 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:55.571470  8295 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:55.587224  8380 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 14:53:55.576052  7032 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:53:55.587707  8196 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:55.589476  8363 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.452442  8358 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:55.558485  8081 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:55.576275  8314 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:55.581401  8285 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:55.601526  8196 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:55.603654  8081 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:55.537590  8169 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:55.597774  7972 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.002s
I20260709 14:53:55.573149  8367 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 14:53:55.586369  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60417.7us
I20260709 14:53:55.614916  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.610129  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.601261  8051 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:55.653913  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:55.604074  8314 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:55.662333  8285 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:53:55.668762  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50772.1us
I20260709 14:53:55.611337  8196 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:55.663050  8367 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:55.682394  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1667) took 1025 ms. Trace:
I20260709 14:53:55.683528  2684 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:55.667356  8363 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:55.676678  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.683053  2751 rpcz_store.cc:276] 0709 14:53:54.656872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:54.657079 (+   207us) service_pool.cc:224] Handling call
0709 14:53:55.682094 (+1025015us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54001ff9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:55.682314 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2019786624}
W20260709 14:53:55.707157  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:55.748798  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 341418us
I20260709 14:53:55.712179  8367 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:55.678956  8295 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:55.793889  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43493.3us
W20260709 14:53:55.788578  8353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.019s	sys 0.018s
I20260709 14:53:55.794410  8196 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:55.821623  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:55.824659  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.800478  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:55.737942  8306 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 14:53:55.829260  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:55.815936  8353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.020s	sys 0.018s
I20260709 14:53:55.816661  8352 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 14:53:55.838338  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:55.840193  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:55.846758  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:55.847910  2758 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 14
W20260709 14:53:55.852003  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:55.851733  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 425796us
I20260709 14:53:55.857158  8352 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:55.787094  8314 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:55.860939  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:55.861872  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:55.862521  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:55.868294  2610 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:55.843887  8183 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:55.882681  2605 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:53:55.858775  8306 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:53:55.898331  2610 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:53:55.855381  8169 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:55.901925  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.627s (SENT)
I20260709 14:53:55.902669  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.903136  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:55.903875  2757 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:55.904493  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:55.905323  2756 raft_consensus.cc:2376] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17. Re-sending same reply.
I20260709 14:53:55.906742  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:55.910097  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.912257  8339 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:55.843297  8285 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 14:53:55.867307  8381 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:55.920572  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:55.925269  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:55.923564  2576 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 3.221s (SENT) [suppressed 45 similar messages]
W20260709 14:53:55.934891  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:55.900729  8306 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:55.937422  8311 consensus_queue.cc:1243] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:53:55.937767  8314 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:55.938174  8381 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:55.938412  8169 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:53:55.972241  2753 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:55.972733  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 376401us
I20260709 14:53:55.931206  8184 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:55.977217  8184 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:55.961261  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:55.962883  7977 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 14:53:55.963755  2576 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.689s (SENT)
I20260709 14:53:55.983618  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.983439  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.965744  8385 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:53:55.981473  8357 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.557s	user 0.014s	sys 0.000s
I20260709 14:53:55.997633  8357 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.001194  8357 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:55.980342  2756 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:55.892582  8183 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.015367  8381 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.016721  8285 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:56.017212  8183 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:56.017436  8285 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:56.016765  8392 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 14:53:56.026132  7972 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:56.026758  7972 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.027848  7972 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.029241  7972 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:56.003794  8382 raft_consensus.cc:2764] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 14:53:56.033808  7968 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.005s
W20260709 14:53:56.007375  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43694.2us
W20260709 14:53:56.035458  7717 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.000s
I20260709 14:53:56.036060  7717 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:56.036695  2576 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.737s (SENT)
I20260709 14:53:56.037204  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.974380  8169 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.038700  8169 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:56.041239  8375 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.002s
I20260709 14:53:55.983285  7977 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:53:56.014258  2723 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.859s (SENT)
I20260709 14:53:56.046382  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:55.988543  7997 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:53:56.050364  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 396631us
W20260709 14:53:56.053912  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 377357us
I20260709 14:53:56.038517  7717 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:55.957849  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 394443us
I20260709 14:53:56.062112  2757 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:56.069461  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.013700  8156 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:53:56.073134  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58937.8us
I20260709 14:53:56.041772  8375 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.045583  8339 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.042584  8385 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.061138  2606 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:53:56.099841  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:55.984372  8306 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.107746  8306 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:56.111204  7997 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:53:56.111994  8398 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 14:53:56.102324  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65674.4us
W20260709 14:53:56.113677  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.780s (SENT)
I20260709 14:53:56.114147  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.101872  8383 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:56.018594  8285 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.116061  8383 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.116636  8383 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.056900  8233 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:56.120939  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.839s (SENT)
I20260709 14:53:56.062670  8382 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.090013  8289 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.125494  7948 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:53:56.137228  2758 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:56.138718  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 285643us
W20260709 14:53:56.162108  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.844s (SENT)
I20260709 14:53:56.162969  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.163511  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:56.122138  7997 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:56.079741  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.140169  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:56.117213  8156 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:56.139451  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71272.9us
I20260709 14:53:56.180215  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.142187  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:56.165608  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 182276us
W20260709 14:53:56.208530  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68470.3us
W20260709 14:53:56.211150  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47732.7us
I20260709 14:53:56.244998  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:56.232491  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 152875us
I20260709 14:53:56.248385  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.252817  7032 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:56.247277  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.161435  7948 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:56.164753  7690 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:56.128484  7717 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
W20260709 14:53:56.277896  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:56.288399  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 216349us
I20260709 14:53:56.289739  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.290902  7032 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.178110  8339 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:56.137068  8233 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:56.312420  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 150355us
I20260709 14:53:56.252473  8289 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:56.252130  8404 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:53:56.299674  8317 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.008s	sys 0.026s
I20260709 14:53:56.320978  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:53:56.313419  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.861s (SENT)
I20260709 14:53:56.331887  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:56.328459  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:56.329041  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 148953us
I20260709 14:53:56.265257  8393 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:56.209713  8156 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.341640  7032 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:56.167613  7977 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:56.347939  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.165045  8382 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.322630  8399 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:56.300506  7690 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.334726  8106 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:53:56.330996  8317 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.008s	sys 0.027s
I20260709 14:53:56.352443  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.361891  8393 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:56.360175  8388 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 14:53:56.343444  8295 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.359258  8375 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.361094  7997 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:53:56.331967  8289 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.386478  8403 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.413321  3542 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:56.366407  8404 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:56.368449  8169 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 14:53:56.392321  8106 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.427134  8106 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:53:56.429956  8363 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.004s
I20260709 14:53:56.408501  8373 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:56.407409  8183 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 14:53:56.437325  8183 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:56.437788  8183 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:56.388415  8062 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 14:53:56.443045  8396 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.443652  8396 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.444072  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:56.444767  2680 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:56.445923  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80009.3us
I20260709 14:53:56.447044  2650 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:56.448961  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.449935  8407 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:56.451380  2686 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:53:56.452790  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:56.453140  8381 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:56.453560  8381 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.387148  7948 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.454008  8381 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:56.454471  7948 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:56.448976  2677 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:56.426223  8404 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.457284  8404 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:56.457437  8407 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.406968  8295 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.458740  8407 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.460306  8156 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.462664  3538 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:53:56.421115  8289 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.343799  8385 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:56.463726  7912 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.869s	user 0.011s	sys 0.006s
I20260709 14:53:56.464224  7912 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:56.465840  2679 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 14:53:56.452823  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:56.468976  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.469313  3536 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:56.421769  8403 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:56.474117  8314 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
W20260709 14:53:56.423079  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 109698us
W20260709 14:53:56.423476  8051 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.763s	user 0.014s	sys 0.001s
I20260709 14:53:56.423946  8383 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:56.453596  2682 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:53:56.425022  8401 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 14:53:56.390034  8374 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:56.381351  7977 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:56.482216  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.438771  8183 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:56.424321  8393 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:56.487957  7997 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:56.482506  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.489394  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:56.442550  8402 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 14:53:56.492030  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.493067  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 994) took 1537 ms. Trace:
I20260709 14:53:56.496044  2606 rpcz_store.cc:276] 0709 14:53:54.955698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:55.000495 (+ 44797us) service_pool.cc:224] Handling call
0709 14:53:56.492788 (+1492293us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:56.492973 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296730880}
I20260709 14:53:56.492048  2679 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:56.494995  2677 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:56.527740  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:56.534874  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:56.521670  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1490) took 1083 ms. Trace:
I20260709 14:53:56.538476  3540 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:56.511932  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:56.527076  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:56.533722  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.498034  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.540022  7032 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.562033  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.562475  2610 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:56.540537  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.563763  3542 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:56.545259  3536 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:56.492889  8309 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.568753  8373 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:53:56.569079  8309 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.570057  7690 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:56.484076  8196 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.004s
I20260709 14:53:56.525115  8304 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:53:56.492733  8383 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.577840  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:56.589745  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.511803  8184 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.591063  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:56.594113  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:56.629251  2684 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:56.629457  3543 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:53:56.538537  2681 rpcz_store.cc:276] 0709 14:53:55.437818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:55.503929 (+ 66111us) service_pool.cc:224] Handling call
0709 14:53:55.504877 (+   948us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:55.568060 (+ 63183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:55.568098 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:55.576603 (+  8505us) log.cc:844] Serialized 78 byte log entry
0709 14:53:55.982041 (+405438us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:56.306070 (+324029us) log.cc:844] Serialized 14 byte log entry
0709 14:53:56.330969 (+ 24899us) log.cc:844] Serialized 14 byte log entry
0709 14:53:56.366406 (+ 35437us) log.cc:844] Serialized 14 byte log entry
0709 14:53:56.439006 (+ 72600us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b18003f4540. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:53:56.450722 (+ 11716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:56.450918 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:56.521362 (+ 70444us) raft_consensus.cc:1685] finished
0709 14:53:56.521389 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:56.521596 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180708,"prepare.queue_time_us":554804,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":304962,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":7815,"spinlock_wait_cycles":4430592,"thread_start_us":685926,"threads_started":2,"wal-append.queue_time_us":532516}
I20260709 14:53:56.595795  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.630574  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:56.596611  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.519500  8183 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:56.520222  8372 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:56.572605  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 80677.2us
I20260709 14:53:56.515136  8358 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.513581  8403 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.512090  8339 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.576972  7690 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.575479  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:56.568449  8399 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.627837  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:56.531797  7997 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:56.667273  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.497609  8374 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:56.572827  8363 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 14:53:56.688377  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:56.692229  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.741199  2684 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:56.741467  3542 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:53:56.695514  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.743221  3543 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:53:56.742697  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.728531  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1387) took 1158 ms. Trace:
I20260709 14:53:56.744715  3536 rpcz_store.cc:276] 0709 14:53:55.570522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:55.577334 (+  6812us) service_pool.cc:224] Handling call
0709 14:53:56.728306 (+1150972us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b5400317c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:56.728452 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016897280}
I20260709 14:53:56.653824  2679 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:56.744202  3547 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:53:56.637764  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.637662  8305 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:56.628782  7717 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:56.612692  7972 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:56.624336  8394 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 14:53:56.650681  3540 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
W20260709 14:53:56.678043  2576 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.628s (SENT)
W20260709 14:53:56.755703  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77743.9us
W20260709 14:53:56.756425  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.935s (SENT)
I20260709 14:53:56.678599  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.757496  3541 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:56.643254  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.679425  2680 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:56.753118  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.761559  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.658278  8295 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.654559  8400 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:56.754403  8305 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.767210  3540 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:53:56.668890  8233 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.781113  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:56.770620  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.011s	sys 0.009s
I20260709 14:53:56.696753  8309 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.659063  8396 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:56.652155  8372 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:56.795743  2679 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:53:56.782675  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.012s	sys 0.009s
W20260709 14:53:56.800236  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 544675us
I20260709 14:53:56.773257  8403 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:56.764832  7972 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.764875  8399 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:56.782192  8305 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:56.813082  2684 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:53:56.672988  2435 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:56.652675  8184 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:56.677032  8358 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.780661  8406 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:56.802280  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.702891  7690 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:56.806936  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.686013  8339 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:56.780076  8306 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.672s	user 0.016s	sys 0.000s
W20260709 14:53:56.810149  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:56.810333  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.782359  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:56.834563  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:56.835954  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53678.2us
I20260709 14:53:56.811698  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:56.815371  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:56.816300  2611 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:53:56.816687  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:56.837365  2609 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:56.817389  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:56.837958  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.818068  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:56.838861  3547 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:56.799050  8372 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:56.820603  7972 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.814145  8396 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 14:53:56.823431  8106 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.826319  8382 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.831797  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.792719  8317 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 14:53:56.835330  3411 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.822s (SENT)
I20260709 14:53:56.872824  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.843000  3540 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:53:56.808120  8403 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:56.847113  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:56.851043  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.852095  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:56.859241  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43963.8us
W20260709 14:53:56.860046  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48500us
I20260709 14:53:56.864789  2606 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 14:53:56.837380  3546 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 14:53:56.868240  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.003s	sys 0.026s
I20260709 14:53:56.871896  3541 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 14:53:56.894248  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.003s	sys 0.026s
I20260709 14:53:56.878638  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:56.895149  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 646873us
I20260709 14:53:56.897183  3536 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:56.901003  8239 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:53:56.901955  8239 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:56.904857  7717 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:56.905782  7717 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:56.808516  8427 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 14:53:56.850805  8306 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:56.916924  2686 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:56.812598  8381 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.849887  7997 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.923520  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:56.894385  8403 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:56.876874  8399 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:53:56.866227  8233 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:53:56.935732  8306 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:56.944023  8233 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.944473  8233 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:56.946158  8306 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:56.930287  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.838375  8406 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.948246  8339 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:56.916906  8404 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:56.949589  8406 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:56.891201  8363 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:53:56.950480  8184 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.951098  8404 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:56.875671  8383 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:56.952019  8383 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:56.952493  8383 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:56.940050  3537 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:56.975783  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.958777  2679 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 14:53:56.979630  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:56.981418  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:56.962744  3536 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:57.000442  3545 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:53:57.002390  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.003209  3546 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:57.003711  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:56.984426  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73424.9us
W20260709 14:53:57.007577  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:56.933311  8317 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:57.014120  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.017560  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:57.020843  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146296us
I20260709 14:53:57.024107  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:56.938545  8387 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:56.934994  8309 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 14:53:56.983736  8196 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.038019  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.037492  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.043972  3536 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:56.925384  8062 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 14:53:56.978732  8399 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:57.061298  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.053351  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:57.026525  8314 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:53:57.072711  3411 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.900s (SENT)
W20260709 14:53:57.088546  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75263.2us
I20260709 14:53:57.089763  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:57.033221  8374 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.102275  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.052651  8413 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:57.104890  2758 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 14:53:57.105122  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.069969  7977 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.007s
I20260709 14:53:57.135864  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.102144  8431 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 14:53:57.138185  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.140596  7690 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.067169  8196 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.033447  8434 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 14:53:57.026481  8317 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:56.942709  8239 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.142964  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.207535  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.259864  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:57.071271  8353 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 14:53:57.167636  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.209709  3536 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:57.214980  2756 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 14:53:57.161094  2759 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:53:57.126709  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:57.148890  2754 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:57.266312  2752 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:53:57.268106  2758 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:57.168207  8306 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.270537  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.293893  3545 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17. Re-sending same reply.
I20260709 14:53:57.271440  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.159251  8435 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.104923  8339 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 14:53:57.287304  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 160720us
I20260709 14:53:57.312356  2760 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 14:53:57.316092  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.307255  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 409584us
I20260709 14:53:57.195130  7948 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.327051  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.317914  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:57.327817  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.210116  8183 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.125381  8427 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.206s
W20260709 14:53:57.291709  8432 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 14:53:57.257335  8314 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:57.332105  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.303810  8431 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 14:53:57.283947  8363 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.164897  8381 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:57.342518  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.411653  2758 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:57.410313  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.413277  3545 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:57.039708   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:53:57.314523  7977 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:53:57.330849  3542 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:53:57.421404  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.303401  8396 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:57.351677  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.304141  8353 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:57.375142  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.399873  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.329366  8374 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.328373  8106 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:57.415751  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:57.430868  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.448170  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:53:57.453243  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.455394  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.313184  8306 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.366072  8434 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.149493  8399 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:57.467252  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.475958  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.348501  8339 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:53:57.480890  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65253.1us
W20260709 14:53:57.483711  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.486690  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.418748  8305 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.005s
I20260709 14:53:57.495440  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.496413  3540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.018s	sys 0.031s
W20260709 14:53:57.498533  3540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.018s	sys 0.031s
I20260709 14:53:57.502306  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.503544  3545 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:57.504911  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:57.509243  2753 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:57.504907  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:57.527233  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.000s	sys 0.042s
I20260709 14:53:57.526808  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:53:57.333300  8435 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.529376  2607 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 14:53:57.527777  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.001s	sys 0.042s
I20260709 14:53:57.420305  7977 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:57.528426  3542 raft_consensus.cc:2410] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:53:57.505903  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.536785  2758 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:57.372625  8385 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 14:53:57.538196  2604 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:53:57.405236  7948 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.534932  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.406646  8311 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.092s
W20260709 14:53:57.531669  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.096s (SENT)
I20260709 14:53:57.496136  8381 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:57.572129  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:57.529320  8432 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 14:53:57.421591  8314 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:57.611107  2753 raft_consensus.cc:2376] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14. Re-sending same reply.
I20260709 14:53:57.579741  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.568465  2611 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 14:53:57.411712  8183 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.579247  8062 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:57.586422  8442 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 14:53:57.588806  8233 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
I20260709 14:53:57.590422  7690 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:57.595911  8449 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:53:57.446434  8363 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:57.599736  3546 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.004s
I20260709 14:53:57.552728  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.446870  8396 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:57.602030  3541 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 14:53:57.575021  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.603297  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:57.538497  8447 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:57.631269  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:53:57.618252  3537 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:53:57.633174  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.570540  8382 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:57.630074  3542 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:57.622068  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.037s	sys 0.058s
I20260709 14:53:57.630740  3545 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:53:57.638533  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.037s	sys 0.058s
I20260709 14:53:57.610420  8381 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:57.634115  3546 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:57.485965  8374 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:57.632654  7717 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.726s	user 0.005s	sys 0.009s
I20260709 14:53:57.634981  8396 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:57.630980  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.668751  2758 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:53:57.633244  8383 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.004s
I20260709 14:53:57.675156  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.627590  8196 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:57.524168  8305 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.638190  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:57.644882  8448 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.360s	user 0.000s	sys 0.002s
I20260709 14:53:57.661273  8381 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:53:57.633255  8304 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.005s	user 0.013s	sys 0.002s
I20260709 14:53:57.680696  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.701589  3546 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 14:53:57.687644  2753 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:57.670981  8317 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:57.694818  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.628418  8233 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:57.712733  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 181126us
W20260709 14:53:57.713524  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.107s (SENT)
I20260709 14:53:57.714049  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:57.632591  7690 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.629315  8062 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:57.707804  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.712522  2678 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 14:53:57.699183  2603 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 14:53:57.718564  2603 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 6
I20260709 14:53:57.718879  2603 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 14:53:57.719156  2603 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:57.718202  3541 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:57.677116  8445 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:57.624567  8339 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:57.716254  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.254s (SENT)
I20260709 14:53:57.726482  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:57.654549  8382 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.678869  7717 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.499753  8434 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:57.725538  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.728768  7032 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:57.652086  8447 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.718286  8420 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.751617  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.686581  8383 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:57.744635  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.760393  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.742733  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.719348  8441 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:57.770491  3542 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:57.774392  3546 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:57.625141  8353 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.778651  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.788257  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.788257  3542 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 14:53:57.791699  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.792363  2758 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:53:57.794538  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78333.6us
W20260709 14:53:57.796865  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.811s (SENT)
I20260709 14:53:57.797297  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:57.800644  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.801399  2753 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:57.806281  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.787622  8306 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:57.817271  3543 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:57.814402  3541 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13. Ops: []
I20260709 14:53:57.816241  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.775013  8463 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:53:57.816218  3546 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 14:53:57.841792  8461 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 14:53:57.828311  8314 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.816551  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:57.823951  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:57.832690  2753 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:57.793802  7948 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:57.805424  8456 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 14:53:57.774927  8420 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:57.864530  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:57.863903  8363 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:57.829603  8462 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
I20260709 14:53:57.867046  3547 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:53:57.868415  8285 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:53:57.868942  8285 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:57.858708  7690 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.863829  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.860448  8387 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:57.799971  8445 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.864739  8233 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.865561  2612 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:57.868470  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.850975  8435 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:57.850351  8183 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:57.900025  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:57.900897  2612 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 10.
I20260709 14:53:57.902267  2612 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 14:53:57.880836  8470 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.903416  8470 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:53:57.904742  8464 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 14:53:57.906105  8470 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.851917  2758 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:57.907199  8444 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.631s	user 0.002s	sys 0.000s
I20260709 14:53:57.907788  8434 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.907922  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:57.909080  8468 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 14:53:57.909399  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.910089  2758 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:57.871322  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:57.911147  3541 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:57.870483  2753 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:57.912904  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.913757  2753 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:53:57.883649  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 86800.9us
I20260709 14:53:57.887629  3547 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:57.915962  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.914652  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.890147  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:57.891204  2608 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:57.815501  8441 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:53:57.925526  2603 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:57.927412  8319 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:57.928794  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:57.932158  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.933686  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.937148  2758 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 14:53:57.884393  8387 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:57.928018  8319 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:57.944346  8463 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.947963  8463 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.947866  8319 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:57.935482  8306 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:57.953399  8306 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.956352  7977 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.870009  8285 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:57.889092  8420 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:57.970746  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1696) took 1429 ms. Trace:
I20260709 14:53:57.961474  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:57.977094  2612 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:57.892999  8445 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.201792  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1019) took 1100 ms. Trace:
W20260709 14:53:58.198252  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.175369  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1018) took 1075 ms. Trace:
I20260709 14:53:58.206758  2754 rpcz_store.cc:276] 0709 14:53:57.101641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.101838 (+   197us) service_pool.cc:224] Handling call
0709 14:53:58.201440 (+1099602us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b5400060254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:58.201700 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104563072}
I20260709 14:53:58.207762  2760 rpcz_store.cc:276] 0709 14:53:57.100199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.100423 (+   224us) service_pool.cc:224] Handling call
0709 14:53:57.100983 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:57.330023 (+229040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.175033 (+845010us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:58.175292 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465963776}
W20260709 14:53:58.115314  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:58.129244  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 213887us
I20260709 14:53:57.963356  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.076444  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.209900  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:53:58.210518  2604 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
W20260709 14:53:58.209812  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.914s (SENT)
I20260709 14:53:58.173861  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1525) took 1679 ms. Trace:
I20260709 14:53:57.968488  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:58.212361  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.065s (SENT)
I20260709 14:53:57.974579  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.214159  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:57.938874  8403 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.960s	user 0.015s	sys 0.000s
I20260709 14:53:58.126758  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:58.220095  2753 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:58.210583  2758 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:53:58.221844  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:57.981022  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:58.029424  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.224448  2611 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:53:58.215175  2610 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:58.046414  2605 rpcz_store.cc:276] 0709 14:53:56.541034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.541209 (+   175us) service_pool.cc:224] Handling call
0709 14:53:56.569504 (+ 28295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:56.729355 (+159851us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:53:56.729864 (+   509us) log.cc:844] Serialized 14 byte log entry
0709 14:53:57.052522 (+322658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:57.052598 (+    76us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:53:57.052654 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:57.906535 (+853881us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:57.970306 (+ 63771us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002fe4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:57.970399 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:57.970651 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64955,"prepare.queue_time_us":10877,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":34069,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":5975,"spinlock_wait_cycles":399247360,"thread_start_us":288114,"threads_started":1,"wal-append.queue_time_us":486817}
I20260709 14:53:58.208741  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:58.225915  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.228109  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.228430  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:58.229175  2760 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:58.229525  2754 raft_consensus.cc:2410] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:58.212003  2686 rpcz_store.cc:276] 0709 14:53:56.494361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.494517 (+   156us) service_pool.cc:224] Handling call
0709 14:53:56.531390 (+ 36873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:56.926636 (+395246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.173516 (+1246880us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:58.173776 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886129536}
I20260709 14:53:57.930335  8353 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:57.929409  8374 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:53:58.247072  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.240072  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.258774  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.258185  2754 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:58.260214  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.234122  2758 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:53:58.261267  2604 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:53:58.261703  3541 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:53:57.973660  8239 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:53:58.142804  8465 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.320s	user 0.004s	sys 0.000s
W20260709 14:53:58.229954  8413 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.100s	user 0.012s	sys 0.006s
I20260709 14:53:58.252025  2605 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:53:57.972853  8306 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:57.958719  7977 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:57.968672  8309 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:58.286020  8309 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:58.287237  8396 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.602s	user 0.003s	sys 0.012s
I20260709 14:53:58.288065  8396 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.289284  8309 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.075758  8285 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:58.270355  2758 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 14:53:58.290464  2758 pending_rounds.cc:85] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 8
I20260709 14:53:58.273686  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1555) took 1087 ms. Trace:
I20260709 14:53:58.043938  8435 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.258430  8311 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:58.242360  8474 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.353s	user 0.002s	sys 0.000s
I20260709 14:53:58.248194  8339 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:57.988492  8383 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.138554  7717 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.269425  8353 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.050166  8434 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.306425  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.290781  2758 pending_rounds.cc:107] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:53:58.317185  2758 raft_consensus.cc:2889] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:58.075349  8463 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.290954  2754 rpcz_store.cc:276] 0709 14:53:57.177966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.257401 (+1079435us) service_pool.cc:224] Handling call
0709 14:53:58.257917 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.265089 (+  7172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.265388 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.323622  8434 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:57.985117  8382 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.280349  8453 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 14:53:58.290745  8372 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.425s	user 0.017s	sys 0.001s
I20260709 14:53:58.331336  8372 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.242692  8403 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.334498  8403 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:58.336002  8381 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.002s
I20260709 14:53:58.336668  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1040) took 1894 ms. Trace:
I20260709 14:53:58.336611  8381 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:58.337175  2678 rpcz_store.cc:276] 0709 14:53:56.442062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.453814 (+ 11752us) service_pool.cc:224] Handling call
0709 14:53:56.454141 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:57.718052 (+1263911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.336362 (+618310us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001d0bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:58.336592 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2886722816}
I20260709 14:53:58.339205  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1230) took 1214 ms. Trace:
I20260709 14:53:58.339664  3536 rpcz_store.cc:276] 0709 14:53:57.125039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.126844 (+  1805us) service_pool.cc:224] Handling call
0709 14:53:58.338971 (+1212127us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:58.339126 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164803968}
I20260709 14:53:58.343451  8358 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.343196  2435 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 12 to 17, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:58.347990  8062 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.083046  8427 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:58.120244  8447 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.351413  8447 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:58.351996  8447 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.354270  8447 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.356210  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.321205  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1694) took 1781 ms. Trace:
I20260709 14:53:58.357038  2606 rpcz_store.cc:276] 0709 14:53:56.540270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.541999 (+  1729us) service_pool.cc:224] Handling call
0709 14:53:56.542535 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:56.893737 (+351202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.320807 (+1427070us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:58.321123 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740379008}
I20260709 14:53:58.280095  7997 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.180761  8420 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:58.364974  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.359760  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1560) took 1180 ms. Trace:
I20260709 14:53:58.366876  2758 rpcz_store.cc:276] 0709 14:53:57.179539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.261447 (+1081908us) service_pool.cc:224] Handling call
0709 14:53:58.261783 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.359462 (+ 97679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.359697 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.368948  2758 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
W20260709 14:53:58.364086  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.313733  8239 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.345577  2435 catalog_manager.cc:5697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:58.378073  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.379127  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1026) took 1163 ms. Trace:
I20260709 14:53:58.385677  2758 rpcz_store.cc:276] 0709 14:53:57.220385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.368179 (+1147794us) service_pool.cc:224] Handling call
0709 14:53:58.368677 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.378651 (+  9974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.379024 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.387390  2758 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:58.273209  8304 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:58.350376  8427 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.399825  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.295257  8413 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:58.405267  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:58.406111  3541 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:58.408300  3546 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:58.394727  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.397138  2435 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 15, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:58.391180  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:58.388084  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1563) took 1163 ms. Trace:
I20260709 14:53:58.400283  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.302325  8383 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:58.410329  2684 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:53:58.400571  3536 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:58.411190  2758 rpcz_store.cc:276] 0709 14:53:57.224598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.386698 (+1162100us) service_pool.cc:224] Handling call
0709 14:53:58.387018 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.387790 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.388016 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.413328  2758 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:53:58.414882  3545 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 14:53:58.416229  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:53:58.385414  8443 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.893s	user 0.004s	sys 0.001s
I20260709 14:53:58.423833  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.424710  3536 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:53:58.424439  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.425980  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:58.426049  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.426970  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:58.428153  3545 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:58.427039  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:58.428910  3546 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:53:58.418486  2435 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 17, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.429598  2760 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:58.430277  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1682) took 1080 ms. Trace:
I20260709 14:53:58.430982  2435 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 13, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.432438  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1678) took 1127 ms. Trace:
I20260709 14:53:58.432881  2758 rpcz_store.cc:276] 0709 14:53:57.287054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.412652 (+1125598us) service_pool.cc:224] Handling call
0709 14:53:58.412992 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.413701 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.413900 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.433554  2435 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:58.435791  2435 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 5 to 14, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.436738  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:58.437960  2758 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:58.437707  2435 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:58.438786  8062 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:58.439188  8317 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 14:53:58.439401  7717 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.439354  8062 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:58.440234  8062 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:58.439661  2435 catalog_manager.cc:5697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 1 to 12, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:58.440418  7717 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.442610  2435 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:58.443893  7717 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.444468  2435 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 8 to 12, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:58.430645  2760 rpcz_store.cc:276] 0709 14:53:57.350427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.428983 (+1078556us) service_pool.cc:224] Handling call
0709 14:53:58.429281 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.430037 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.430220 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:53:58.446801  2435 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 14, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:58.447878  8306 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:58.449100  8306 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:53:58.449525  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.810s (SENT)
I20260709 14:53:58.291282  8435 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.453665  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:53:58.459692  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 184255us
W20260709 14:53:58.462119  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.291262  8311 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.402290  8400 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:58.398264  8447 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:53:58.455865  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:58.452859  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.454815  2758 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:53:58.454079  2760 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:53:58.476251  2684 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:53:58.460561  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.358501  7997 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.478125  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.478714  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:58.396392  7972 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 14:53:58.466424  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.455956  2759 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:58.474686  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.499867  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:53:58.382303  8309 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 14:53:58.489816  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.385210  8420 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.465595  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.449894  2435 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.499537  2758 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:58.486577  3541 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
W20260709 14:53:58.500892  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48137.5us
I20260709 14:53:58.510576  2760 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:58.460667  8372 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.500901  2677 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 14:53:58.401196  8382 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 14:53:58.403681  8412 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 14:53:58.511106  2435 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.495875  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.523268  2679 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:53:58.471462  8463 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.475839  8447 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:58.415304  8304 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:58.470219  8381 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.478289  8400 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:58.528918  2680 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:58.417682  8383 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:53:58.522424  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 162415us
I20260709 14:53:58.541285  2650 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:58.540601  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78279.8us
I20260709 14:53:58.532651  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1393) took 1640 ms. Trace:
I20260709 14:53:58.542253  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:58.500598  8441 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:53:58.534376  2435 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 13, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.542338  3538 rpcz_store.cc:276] 0709 14:53:56.892627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.946914 (+ 54287us) service_pool.cc:224] Handling call
0709 14:53:56.947349 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:57.002416 (+ 55067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:57.002473 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:57.013331 (+ 10858us) log.cc:844] Serialized 52 byte log entry
0709 14:53:57.531098 (+517767us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:57.531142 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:57.531225 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:58.115963 (+584738us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:53:58.524359 (+408396us) raft_consensus.cc:1685] finished
0709 14:53:58.524395 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:58.532560 (+  8165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33588,"raft.queue_time_us":54881,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":8612096,"thread_start_us":458144,"threads_started":1,"wal-append.queue_time_us":740682}
I20260709 14:53:58.537479  8447 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:58.575625  2682 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:53:58.540500  8396 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.571635  2680 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:53:58.528111  8486 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:53:58.573988  2435 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:58.571877  2677 pending_rounds.cc:85] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 9
W20260709 14:53:58.580679  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.580893  2677 pending_rounds.cc:107] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 14:53:58.581244  2677 raft_consensus.cc:2889] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:58.534142  8463 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.586361  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.587703  8463 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.587886  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.588563  8463 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:58.589262  2685 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:53:58.589633  8463 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.593274  8445 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.593421  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.595222  8434 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.597075  8435 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.598462  8358 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 14:53:58.605014  2435 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:53:58.605818  7997 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.606688  2435 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:58.608210  2435 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.590319  8463 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:58.617210  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.618264  8463 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.598001  8353 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.610100  2435 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.619782  8306 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:58.639959  2433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 162) took 54335 ms (client timeout 15000 ms). Trace:
I20260709 14:53:58.666091  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1023) took 1516 ms. Trace:
I20260709 14:53:58.665138  2435 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:53:58.668938  8493 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:53:58.619519  8239 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.670431  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1024) took 1463 ms. Trace:
W20260709 14:53:58.671598  8387 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.004s
I20260709 14:53:58.672277  8387 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.675117  8412 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 14:53:58.680060  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 137912us
I20260709 14:53:58.681239  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:58.682845  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 171106us
I20260709 14:53:58.684568  8306 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:58.684684  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 326273us
I20260709 14:53:58.685288  8306 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.688380  8306 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:58.666904  2757 rpcz_store.cc:276] 0709 14:53:57.151755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.207824 (+ 56069us) service_pool.cc:224] Handling call
0709 14:53:57.208377 (+   553us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:57.214662 (+  6285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:57.529115 (+314453us) log.cc:844] Serialized 14 byte log entry
0709 14:53:57.745251 (+216136us) log.cc:844] Serialized 14 byte log entry
0709 14:53:57.745411 (+   160us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:57.746056 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 14:53:57.746214 (+   158us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:57.910977 (+164763us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b18003a5e20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:53:57.911202 (+   225us) log.cc:844] Serialized 14 byte log entry
0709 14:53:58.665290 (+754088us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54003113a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c1733 00007fa16e646dbf 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c
0709 14:53:58.665424 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.665586 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:58.665663 (+    77us) raft_consensus.cc:1685] finished
0709 14:53:58.665685 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:58.665979 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146198,"prepare.queue_time_us":611690,"prepare.run_cpu_time_us":225,"prepare.run_wall_time_us":6601,"raft.queue_time_us":15237,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":33897,"spinlock_wait_cycles":42684032,"thread_start_us":463676,"threads_started":2,"wal-append.queue_time_us":218872}
I20260709 14:53:58.691641  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.570317  2684 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:53:58.697548  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 130614us
I20260709 14:53:58.632593  8427 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.697798  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.576792  8488 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 14:53:58.671001  2754 rpcz_store.cc:276] 0709 14:53:57.207434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.323495 (+1116061us) service_pool.cc:224] Handling call
0709 14:53:58.323963 (+   468us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:53:58.324455 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:58.324490 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:53:58.325305 (+   815us) log.cc:844] Serialized 78 byte log entry
0709 14:53:58.464987 (+139682us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:58.569636 (+104649us) log.cc:844] Serialized 14 byte log entry
0709 14:53:58.570162 (+   526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.570255 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:58.670109 (+ 99854us) raft_consensus.cc:1685] finished
0709 14:53:58.670137 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:58.670349 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":158970,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":67,"raft.queue_time_us":5386,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":195,"thread_start_us":209409,"threads_started":2,"wal-append.queue_time_us":258128}
I20260709 14:53:58.668009  2435 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 12, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.619359  8062 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.703742  2684 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:53:58.602710  8396 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 14:53:58.680760  8382 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:58.711230  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.681624  8412 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:58.748098  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.748829  2683 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:53:58.749532  2682 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:53:58.599447  8487 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 14:53:58.714254  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:58.756536  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.708043  8309 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:58.754285  2435 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 8, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 8 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:53:58.773715  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.699810  8239 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:58.760049  8412 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.768838  8062 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:58.769332  8305 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:58.715164  7690 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:53:58.772262  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:58.774755  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.781320  2610 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 12.
I20260709 14:53:58.769989  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:58.779743  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.782809  2605 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:58.774113  2435 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:53:58.783808  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:53:58.784197  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 74340.6us
I20260709 14:53:58.784395  2605 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:58.785941  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.787211  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:58.787302  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.788434  2611 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:53:58.777029  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.796618  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.747936  8427 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:58.715723  8382 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:58.792250  2683 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
W20260709 14:53:58.832110  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 382623us
W20260709 14:53:58.799439  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:58.832911  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.055s (SENT)
I20260709 14:53:58.833474  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.799870  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:58.778510  8387 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:58.795969  7977 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.002s
I20260709 14:53:58.804497  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.779075  8239 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:58.830554  2435 catalog_manager.cc:5697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 3 to 13, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:53:58.715428  8489 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.085s
W20260709 14:53:58.807256  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94947.8us
I20260709 14:53:58.817953  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1562) took 1609 ms. Trace:
I20260709 14:53:58.835621  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:58.852887  2756 rpcz_store.cc:276] 0709 14:53:57.208583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.327102 (+1118519us) service_pool.cc:224] Handling call
0709 14:53:58.327551 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.575977 (+248426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:58.576012 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:58.576057 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:58.576092 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.817522 (+241430us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b540040e2d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:58.817608 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:58.817857 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570790144}
I20260709 14:53:58.847330  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:58.854543  2607 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
W20260709 14:53:58.827844  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1561) took 1648 ms (client timeout 1997 ms). Trace:
W20260709 14:53:58.855531  2759 rpcz_store.cc:269] 0709 14:53:57.179836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.327815 (+147979us) service_pool.cc:224] Handling call
0709 14:53:58.827548 (+1499733us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:58.827748 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375684864}
W20260709 14:53:58.847576  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54095.1us
W20260709 14:53:58.848788  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.436s (SENT)
I20260709 14:53:58.856655  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.856794  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:58.832379  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:58.860244  2680 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 14:53:58.806034  8455 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.208s	user 0.002s	sys 0.000s
I20260709 14:53:58.869823  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.859867  2751 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15. Re-sending same reply.
W20260709 14:53:58.870985  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1551) took 1695 ms (client timeout 1640 ms). Trace:
I20260709 14:53:58.796885  8502 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:53:58.826018  8285 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.535s	user 0.014s	sys 0.001s
I20260709 14:53:58.883087  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:53:58.871418  2751 rpcz_store.cc:269] 0709 14:53:57.175894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.207305 (+ 31411us) service_pool.cc:224] Handling call
0709 14:53:58.832181 (+1624876us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:58.870897 (+ 38716us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5517568}
I20260709 14:53:58.872898  2683 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:53:58.831956  8427 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.874387  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.807971  8304 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:58.876652  2684 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:53:58.877031  8393 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:58.802922  8499 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 14:53:58.885213  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.859652  8353 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:53:58.885949  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:58.875365  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.890287  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:53:58.892647  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 175084us
I20260709 14:53:58.852097  8396 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:53:58.911844  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:58.913055  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56352.8us
I20260709 14:53:58.832361  8382 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:58.807591  8383 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:53:58.896100  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47354.4us
I20260709 14:53:58.808012  7690 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 14:53:58.859534  8488 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 14:53:58.898684  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:58.859196  8239 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:58.791034  8497 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 14:53:58.867923  2605 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:58.918682  2683 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:58.807093  8385 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 14:53:58.887485  8387 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:58.882282  8502 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:58.931897  8383 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:53:58.932988  8383 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:58.933336  8383 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:53:58.933831  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.935463  8395 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:58.935949  8395 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:58.656787  2433 rpcz_store.cc:269] 0709 14:53:04.227222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:04.252196 (+ 24974us) service_pool.cc:224] Handling call
0709 14:53:58.547229 (+54295033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:53:44.960697 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:53:44.961470 (+   773us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:53:44.961530 (+    60us) write_op.cc:435] Acquired schema lock
0709 14:53:44.961553 (+    23us) tablet.cc:662] Decoding operations
0709 14:53:45.683562 (+722009us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:53:45.683715 (+   153us) write_op.cc:641] Partition lock acquired for write op
0709 14:53:45.683758 (+    43us) tablet.cc:685] Acquiring locks for 19 operations
0709 14:53:45.685771 (+  2013us) tablet.cc:701] Row locks acquired
0709 14:53:45.685833 (+    62us) write_op.cc:260] PREPARE: finished
0709 14:53:45.686078 (+   245us) write_op.cc:270] Start()
0709 14:53:45.686282 (+   204us) write_op.cc:276] Timestamp: P: 1783608825686007 usec, L: 0
0709 14:53:45.686331 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 14:53:45.887385 (+201054us) log.cc:844] Serialized 7525 byte log entry
0709 14:53:47.200894 (+1313509us) op_driver.cc:464] REPLICATION: finished
0709 14:53:47.368468 (+167574us) write_op.cc:301] APPLY: starting
0709 14:53:47.368622 (+   154us) tablet.cc:1367] starting BulkCheckPresence
0709 14:53:47.641949 (+273327us) tablet.cc:1370] finished BulkCheckPresence
0709 14:53:47.642107 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:53:48.204070 (+561963us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:53:48.204524 (+   454us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 14:53:48.204593 (+    69us) write_op.cc:312] APPLY: finished
0709 14:53:48.205118 (+   525us) log.cc:844] Serialized 171 byte log entry
0709 14:53:49.182231 (+977113us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:49.234907 (+ 52676us) write_op.cc:454] Released schema lock
0709 14:53:49.284377 (+ 49470us) write_op.cc:341] FINISH: Updating metrics
0709 14:53:49.388526 (+104149us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:53:49.395183 (+  6657us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1530299,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":160816,"apply.run_cpu_time_us":69204,"apply.run_wall_time_us":1988137,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83509,"num_ops":19,"prepare.queue_time_us":651147,"prepare.run_cpu_time_us":25630,"prepare.run_wall_time_us":1390964,"raft.queue_time_us":209196,"raft.run_cpu_time_us":8222,"raft.run_wall_time_us":371644,"replication_time_us":1374867,"thread_start_us":1817405,"threads_started":5,"wal-append.queue_time_us":1264587,"wal-append.run_cpu_time_us":23186,"wal-append.run_wall_time_us":3095622}]]}
I20260709 14:53:58.938619  8387 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:58.936844  8395 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:58.939828  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.895352  8410 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:58.940689  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.942291  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:58.823328  8503 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 14:53:58.914711  8396 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:58.950347  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.899530  8427 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:58.949573  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59377.7us
W20260709 14:53:58.956771  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.939234  8387 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:58.934239  8383 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:58.980257  2605 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:53:58.943835  8463 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:53:58.957331  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:58.946244  8410 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:58.939476  8502 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:58.955164  8396 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:53:58.958456  8489 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.946738  8504 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:53:58.956439  8496 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:58.994130  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:58.956893  8395 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:58.934517  8498 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 14:53:58.956326  8106 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:58.988574  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.946497  8494 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 14:53:58.958060  8233 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:53:59.009122  8506 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:53:58.995114  8463 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:58.950356  8062 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 14:53:59.019727  8427 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:58.990319  8441 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:53:58.995770  8410 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:59.024863  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:58.955029  7977 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:59.002688  8383 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:58.997357  8453 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:53:59.034801  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53636.5us
I20260709 14:53:59.030373  8445 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 14:53:59.001742  8504 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:53:59.052650  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:53:59.034198  8489 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.029330  8233 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:59.041481  8106 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 14:53:59.034510  8353 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:59.088392  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1011) took 1168 ms. Trace:
W20260709 14:53:59.086112  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 829198us
W20260709 14:53:59.085825  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.097249  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:53:59.193688  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.101790  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1576) took 1973 ms. Trace:
W20260709 14:53:59.090582  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.195933  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:59.196118  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.196974  3538 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:53:59.195964  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98871.8us
I20260709 14:53:59.198740  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.083632  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1388) took 2192 ms. Trace:
I20260709 14:53:59.201350  3540 rpcz_store.cc:276] 0709 14:53:56.891112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:56.892925 (+  1813us) service_pool.cc:224] Handling call
0709 14:53:56.926224 (+ 33299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:56.926479 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:57.539081 (+612602us) log.cc:844] Serialized 14 byte log entry
0709 14:53:58.089992 (+550911us) log.cc:844] Serialized 14 byte log entry
0709 14:53:59.083087 (+993095us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:59.083156 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:59.083227 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.083369 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:59.083555 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1010188,"prepare.queue_time_us":235406,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":118,"raft.queue_time_us":1073313,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":217,"thread_start_us":1012221,"threads_started":3,"wal-append.queue_time_us":267528,"wal-append.run_cpu_time_us":3550,"wal-append.run_wall_time_us":1298579}
I20260709 14:53:59.065721  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:53:59.353189  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 287592us
I20260709 14:53:59.355182  8460 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.355343  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.356336  3536 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 14:53:59.356828  8496 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:59.080277  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1580) took 1487 ms. Trace:
I20260709 14:53:59.357496  3543 rpcz_store.cc:276] 0709 14:53:57.592964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.845335 (+252371us) service_pool.cc:224] Handling call
0709 14:53:57.880022 (+ 34687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:57.880312 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:57.935821 (+ 55509us) log.cc:844] Serialized 14 byte log entry
0709 14:53:59.079786 (+1143965us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:59.079865 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:59.079935 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.080025 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:59.080196 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1158772,"prepare.queue_time_us":59342,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1144469,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118}
I20260709 14:53:59.045930  8062 consensus_queue.cc:1243] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:53:59.194157  3542 rpcz_store.cc:276] 0709 14:53:57.128723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.799216 (+670493us) service_pool.cc:224] Handling call
0709 14:53:57.800883 (+  1667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:57.871430 (+ 70547us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:53:57.893809 (+ 22379us) log.cc:844] Serialized 14 byte log entry
0709 14:53:59.091028 (+1197219us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.091681 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 14:53:59.091864 (+   183us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:53:59.091903 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.091984 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:59.101456 (+  9472us) raft_consensus.cc:1685] finished
0709 14:53:59.101485 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:59.101695 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":690038,"prepare.queue_time_us":64131,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":81,"raft.queue_time_us":1895560,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":220,"thread_start_us":411705,"threads_started":1,"wal-append.queue_time_us":579444}
I20260709 14:53:59.191596  2603 rpcz_store.cc:276] 0709 14:53:57.920305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.045155 (+124850us) service_pool.cc:224] Handling call
0709 14:53:58.110229 (+ 65074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:58.304590 (+194361us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:58.305061 (+   471us) log.cc:844] Serialized 14 byte log entry
0709 14:53:58.569069 (+264008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:53:58.569458 (+   389us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:58.569850 (+   392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.088123 (+518273us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002fe9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:53:59.088187 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:59.088326 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":743,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":52153,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":446196608,"thread_start_us":260084,"threads_started":2,"wal-append.queue_time_us":386032}
I20260709 14:53:59.045660  8506 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:59.064792  8339 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.767s	user 0.015s	sys 0.001s
I20260709 14:53:59.372357  8339 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:59.375970  8506 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.054397  8427 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.056151  8487 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:53:59.167411  8511 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 14:53:59.384580  3538 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:59.079385  8311 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:59.082655  8463 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:59.214636  8494 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:53:59.213902  8319 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:59.118512  8375 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:53:59.264555  8400 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.703s	user 0.016s	sys 0.000s
I20260709 14:53:59.123461  8453 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.213537  8353 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.265959  8285 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.277103  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:59.072422  8501 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:53:59.296204  8447 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.711s	user 0.013s	sys 0.002s
W20260709 14:53:59.320757  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:59.329059  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.165622  8309 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:59.079959  8420 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:59.212077  7977 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.322209  8498 consensus_queue.cc:1243] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:53:59.086990  8374 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:53:59.213043  8233 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:53:59.475142  8374 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.477484  8374 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.091122  8478 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.478204  8478 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:59.479549  8503 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:53:59.480024  8503 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:59.480754  8478 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:59.214568  2755 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:53:59.465711  8285 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:53:59.483031  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1553) took 2306 ms (client timeout 1801 ms). Trace:
W20260709 14:53:59.483443  2755 rpcz_store.cc:269] 0709 14:53:57.176871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:57.177026 (+   155us) service_pool.cc:224] Handling call
0709 14:53:58.769734 (+1592708us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:53:59.482775 (+713041us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:53:59.482952 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231635072}
I20260709 14:53:59.484725  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.485054  8427 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:59.485334  2755 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15. Re-sending same reply.
I20260709 14:53:59.486407  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:53:59.253096  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60023.4us
I20260709 14:53:59.347429  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:59.090862  8183 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:53:59.489203  8183 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:59.491588  8183 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:59.349785  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.350163  7032 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.492666  7032 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.493413  8503 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.355911  8460 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.087502  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:59.495155  8489 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.495492  2612 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13. Re-sending same reply.
W20260709 14:53:59.091701  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 728902us
I20260709 14:53:59.496567  8460 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:59.060160  8306 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.285s
I20260709 14:53:59.088649  8403 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.257s
I20260709 14:53:59.311357  8473 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 14:53:59.169239  8507 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 14:53:59.389513  8410 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.331818  8382 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.354672  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:59.422991  8502 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.500985  3547 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:59.419363  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:59.494727  7032 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.394773  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.502123  2758 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:53:59.425454  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:59.426326  3545 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
W20260709 14:53:59.428458  8412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.079s	sys 0.067s
I20260709 14:53:59.429590  3537 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 14:53:59.430581  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:59.443495  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.546109  3537 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 9
I20260709 14:53:59.385877  8387 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.446117  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.450434  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.547410  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.450271  8375 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:53:59.456687  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 179745us
I20260709 14:53:59.548026  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.548367  3541 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 14:53:59.548753  2755 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:59.545861  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:59.549069  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:59.460453  8453 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.461874  8447 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.452029  8400 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.464404  8353 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.473753  7977 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.505800  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:53:59.391541  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 383172us
I20260709 14:53:59.454384  8501 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:53:59.408315  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:59.537082  3545 pending_rounds.cc:85] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 7
I20260709 14:53:59.506078  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:59.570489  3545 pending_rounds.cc:107] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 14:53:59.570502  2759 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:53:59.570921  3545 raft_consensus.cc:2889] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:59.538913  2680 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:59.546550  3537 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 14:53:59.548048  2753 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:53:59.497614  8503 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:59.509804  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:53:59.546597  3536 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:53:59.536782  2754 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:53:59.552930  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:59.552968  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 198435us
I20260709 14:53:59.710009  2759 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:53:59.714957  3546 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:53:59.716804  3537 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:59.553855  2758 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:53:59.555968  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.724241  2755 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:59.556027  2679 pending_rounds.cc:85] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 8
I20260709 14:53:59.724658  2679 pending_rounds.cc:107] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 14:53:59.724920  2679 raft_consensus.cc:2889] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:53:59.497782  8493 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.565358  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.725930  3541 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:59.536346  2751 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:59.454510  8494 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:53:59.571409  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 115955us
W20260709 14:53:59.551020  8522 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 14:53:59.405493  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:53:59.545912  8412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.079s	sys 0.067s
I20260709 14:53:59.601135  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.738785  2756 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:59.494746  8285 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:53:59.493507  8396 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.563478  8502 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:53:59.500641  8497 consensus_queue.cc:1175] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 18 status { last_received { term: 15 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 14:53:59.636019  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90245.7us
I20260709 14:53:59.746482  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:59.636864  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.747571  2757 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:53:59.550805  8375 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:53:59.706681  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:59.555462  8339 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:53:59.555214  8496 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:59.471673  8463 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:53:59.708206  2752 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:53:59.708909  2684 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:59.709053  2678 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 14:53:59.600553  8106 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 14:53:59.499071  8372 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 14:53:59.769440  3537 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 14:53:59.770402  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:59.730415  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 177596us
I20260709 14:53:59.731508  7032 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:53:59.732684  2755 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:59.734244  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.735910  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.740200  2760 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 14:53:59.773514  2603 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:53:59.773833  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.741237  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:59.707312  8400 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:53:59.741572  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:59.743593  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 233877us
I20260709 14:53:59.748970  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.742925  8487 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:53:59.781147  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1724) took 1003 ms. Trace:
I20260709 14:53:59.706615  8233 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.752907  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.754732  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:59.757858  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.760106  2605 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:53:59.741521  8399 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:53:59.717150  8478 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:53:59.765332  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:53:59.767398  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.715437  8472 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:53:59.710793  8363 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:53:59.772908  7032 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:59.774510  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.774925  2609 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:53:59.775054  8372 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:53:59.737658  8489 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.777160  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:53:59.713033  8239 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.767s	user 0.017s	sys 0.003s
I20260709 14:53:59.776932  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:53:59.821728  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44888.1us
I20260709 14:53:59.823137  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:53:59.778452  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:53:59.777822  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:53:59.825271  3537 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:53:59.825418  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47678.2us
I20260709 14:53:59.779009  2608 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 14:53:59.779680  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:59.829241  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.737483  8493 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.781598  2604 rpcz_store.cc:276] 0709 14:53:58.777541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.862711 (+ 85170us) service_pool.cc:224] Handling call
0709 14:53:58.874274 (+ 11563us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:58.874746 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:58.874781 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:58.875524 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 14:53:58.996790 (+121266us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:53:58.996837 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:58.998745 (+  1908us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:53:59.780889 (+782144us) raft_consensus.cc:1685] finished
0709 14:53:59.780916 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:53:59.781081 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30789,"raft.queue_time_us":2875,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17879,"thread_start_us":120804,"threads_started":1,"wal-append.queue_time_us":150328}
I20260709 14:53:59.749621  8521 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.786623  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.842943  2605 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:53:59.840113  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:59.763101  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:53:59.789018  3545 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:53:59.790079  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:59.767973  8305 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:53:59.800421  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:59.846663  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83539.3us
I20260709 14:53:59.802639  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:53:59.812963  2603 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:53:59.849888  3541 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:59.848336  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:59.814524  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.804025  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.829839  7032 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:59.828209  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:59.750525  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:59.830706  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.832511  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:53:59.853952  2612 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:53:59.801164  8478 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:59.855363  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:53:59.862226  2608 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:59.792217  8339 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 14:53:59.865227  8387 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:53:59.866158  8387 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:59.866628  8387 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:53:59.866796  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101503us
I20260709 14:53:59.867131  8387 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:59.867847  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:53:59.866796  2608 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14. Ops: []
I20260709 14:53:59.839701  8460 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:59.874215  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:53:59.860809  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:53:59.862267  2603 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 14:53:59.857669  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:53:59.859459  7032 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:53:59.871690  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:53:59.873032  2681 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:53:59.831080  8383 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:53:59.868438  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:53:59.882400  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:53:59.843739  8363 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:59.842417  8473 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:53:59.845763  8487 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:53:59.788662  8306 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 14:53:59.845165  8403 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.886281  3537 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:53:59.855229  8427 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:53:59.808903  8399 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.856216  8502 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:59.891546  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:53:59.831494  8489 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:53:59.855481  8410 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.860308  8285 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:53:59.840101  8239 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.889173  8383 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:53:59.869444  7977 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:53:59.907490  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:53:59.996495  2726 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.818s (SENT)
I20260709 14:53:59.860791  8382 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:53:59.957505  8487 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:00.030747  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:00.034637  2683 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:53:59.973095  8285 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:00.000897  8403 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:53:59.888762  8106 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:53:59.958959  8374 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:53:59.872195  8305 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:53:59.966547  8502 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:53:59.964411  7948 raft_consensus.cc:2764] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:53:59.958909  8450 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 14:54:00.062377  2686 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:54:00.036087  8487 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:53:59.852895  8521 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:53:59.884979  8400 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 14:53:59.850633  8494 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.013406  8427 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:53:59.963436  8470 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 14:53:59.992269  8447 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:00.095280  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 243416us
I20260709 14:54:00.040505  2612 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:54:00.059114  8502 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:00.098127  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.099094  2612 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:53:59.997438  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:00.049718  8285 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:00.116065  2755 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:00.072346  2751 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:00.054042  8374 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:00.118568  8374 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:00.052977  8106 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:00.121706  8375 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:00.123399  8106 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:00.124454  2606 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:00.016043  8410 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 14:54:00.095826  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 215006us
I20260709 14:54:00.128091  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:00.072130  8183 raft_consensus.cc:2764] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:54:00.014447  8478 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:00.139556  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 290462us
I20260709 14:54:00.146986  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:00.115664  8528 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.293s	user 0.002s	sys 0.000s
I20260709 14:54:00.077732  8396 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:00.147192  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 238097us
W20260709 14:54:00.159444  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 734804us
I20260709 14:54:00.160933  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:00.138252  8472 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 14:54:00.143219  8372 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:00.173132  8544 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:00.175443  2650 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:00.066454  8363 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.179744  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:00.148348  8410 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:00.153678  8233 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:00.114938  8427 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.142990  7977 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 14:54:00.244289  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 247874us
I20260709 14:54:00.191992  2612 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:00.260172  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:00.265916  2612 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:00.267542  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:00.182777  8524 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 14:54:00.182444  8447 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:00.166144  8498 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 14:54:00.304087  7032 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:00.315881  7032 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:00.317232  7032 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.320310  7032 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:00.189606  8478 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:00.184968  8375 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:00.376674  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99980.3us
W20260709 14:54:00.382789  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.825s (SENT)
I20260709 14:54:00.384020  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:00.389029  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:00.389815  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:00.159469  8305 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.395391  7032 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:54:00.396095  7032 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:00.396575  7032 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:54:00.290102  8306 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:00.392007  8530 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.380s	user 0.000s	sys 0.002s
I20260709 14:54:00.349596  8339 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:00.381052  8233 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:00.385767  8399 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:00.186349  8544 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.397948  8372 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:00.404769  8383 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:00.506366  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 334114us
I20260709 14:54:00.510136  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:00.453536  8531 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 14:54:00.515817  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:00.435529  8524 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 14:54:00.511449  7032 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:00.492133  8540 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 14:54:00.431551  8553 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:00.405748  8447 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
W20260709 14:54:00.492133  8535 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.323s	user 0.000s	sys 0.000s
I20260709 14:54:00.472594  8536 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:00.520247  8534 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.375s	user 0.002s	sys 0.002s
I20260709 14:54:00.519865  2612 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:00.556059  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1690) took 1101 ms. Trace:
W20260709 14:54:00.464787  8532 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.406s	user 0.002s	sys 0.000s
I20260709 14:54:00.574173  2756 rpcz_store.cc:276] 0709 14:53:59.454552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.747194 (+292642us) service_pool.cc:224] Handling call
0709 14:53:59.752339 (+  5145us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.766177 (+ 13838us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:53:59.766219 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.768822 (+  2603us) log.cc:844] Serialized 26 byte log entry
0709 14:53:59.948084 (+179262us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:59.948157 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.948274 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:00.555794 (+607520us) raft_consensus.cc:1685] finished
0709 14:54:00.555819 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:00.555975 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13282,"raft.queue_time_us":15007,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9921,"thread_start_us":172439,"threads_started":1,"wal-append.queue_time_us":326575}
I20260709 14:54:00.546540  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1402) took 1093 ms. Trace:
I20260709 14:54:00.547333  8470 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:00.576296  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1545) took 1071 ms. Trace:
I20260709 14:54:00.576694  8470 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:00.574497  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.578261  2612 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:00.576972  2679 rpcz_store.cc:276] 0709 14:53:59.505011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.511328 (+  6317us) service_pool.cc:224] Handling call
0709 14:53:59.555795 (+ 44467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:53:59.737855 (+182060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.737893 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:53:59.738553 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 14:53:59.841794 (+103241us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:53:59.841858 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.844241 (+  2383us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:00.575933 (+731692us) raft_consensus.cc:1685] finished
0709 14:54:00.575968 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:00.576204 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20624,"raft.queue_time_us":13170,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":10051,"thread_start_us":71847,"threads_started":1,"wal-append.queue_time_us":244552}
I20260709 14:54:00.547551  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.431020  8410 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.503015  8339 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.560226  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:00.602125  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:00.536397  7032 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:00.464365  8233 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:00.565872  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1579) took 1126 ms. Trace:
I20260709 14:54:00.580055  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:00.547745  8489 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:00.576256  3540 rpcz_store.cc:276] 0709 14:53:59.466787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.466968 (+   181us) service_pool.cc:224] Handling call
0709 14:53:59.546362 (+ 79394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.555672 (+  9310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.555714 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.568180 (+ 12466us) log.cc:844] Serialized 26 byte log entry
0709 14:53:59.761262 (+193082us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:53:59.761327 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.761435 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:00.546260 (+784825us) raft_consensus.cc:1685] finished
0709 14:54:00.546287 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:00.546462 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46762,"raft.queue_time_us":9501,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6824,"thread_start_us":45594,"threads_started":1,"wal-append.queue_time_us":268735}
W20260709 14:54:00.604790  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63463.3us
I20260709 14:54:00.604660  2754 rpcz_store.cc:276] 0709 14:53:59.439663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.718522 (+278859us) service_pool.cc:224] Handling call
0709 14:53:59.722862 (+  4340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.723419 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:53:59.887047 (+163628us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.534886 (+647839us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.535457 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.535634 (+   177us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:00.535672 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.535748 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:00.565066 (+ 29318us) raft_consensus.cc:1685] finished
0709 14:54:00.565111 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:00.565744 (+   633us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13123,"prepare.queue_time_us":351555,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2528,"raft.queue_time_us":253599,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":32870,"thread_start_us":766341,"threads_started":3,"wal-append.queue_time_us":498731}
I20260709 14:54:00.606344  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:00.577185  8470 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:00.433056  8306 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.552871  7948 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:00.659051  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.660506  2604 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:00.664080  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:00.665623  2612 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:00.505296  8537 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.140s
W20260709 14:54:00.668458  2760 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.000s	sys 0.012s
W20260709 14:54:00.669516  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 109426us
W20260709 14:54:00.669651  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.003s	sys 0.008s
I20260709 14:54:00.708652  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.668201  7948 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:00.709611  2604 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:00.709658  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.004s	sys 0.012s
W20260709 14:54:00.709794  2760 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.000s	sys 0.016s
I20260709 14:54:00.713102  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.711114  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:00.716182  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.608633  8233 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:00.719869  2604 raft_consensus.cc:2376] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16. Re-sending same reply.
I20260709 14:54:00.723315  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.550745  8487 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.552277  8536 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:00.726208  7032 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 14:54:00.621470  8521 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:00.724340  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.019s	sys 0.045s
W20260709 14:54:00.742607  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.020s	sys 0.046s
W20260709 14:54:00.743557  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 875781us
I20260709 14:54:00.746145  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:00.746896  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.014s	sys 0.007s
W20260709 14:54:00.749804  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.015s	sys 0.008s
I20260709 14:54:00.749702  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:00.753289  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 569237us
I20260709 14:54:00.720515  7948 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:00.780339  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:00.777016  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 195652us
I20260709 14:54:00.779011  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:00.787683  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:00.752629  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.008s	sys 0.016s
I20260709 14:54:00.792254  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:00.797163  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.022s
W20260709 14:54:00.797658  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 699184us
I20260709 14:54:00.730543  8557 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:00.800546  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:00.807955  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:00.815905  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.817754  2612 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:00.817987  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 211810us
I20260709 14:54:00.815255  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:00.820297  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:00.823231  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:00.824101  3536 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
W20260709 14:54:00.828467  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:00.688732  8372 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.855966  8363 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:54:00.856464  8363 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:00.808503  8559 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:00.857780  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:00.847987  8552 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 14:54:00.810109  8453 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:00.868425  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.873337  2612 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:54:00.825973  8427 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:00.854604  8553 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:00.814992  7977 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 14:54:00.898242  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:00.899675  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:00.900779  2612 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:00.921615  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:00.868177  7972 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 14:54:00.921576  8554 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 14:54:00.994589  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:00.883805  8453 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:00.984880  8538 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.466s	user 0.000s	sys 0.002s
W20260709 14:54:00.995437  2724 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.547s (SENT) [suppressed 24 similar messages]
I20260709 14:54:00.995426  3545 raft_consensus.cc:2376] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14. Re-sending same reply.
I20260709 14:54:00.993340  3536 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:00.922044  8544 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:00.996133  8552 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 14:54:01.020210  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.028812  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.030398  3543 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:01.009431  8488 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:01.034384  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:01.021546  8374 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.902s	user 0.012s	sys 0.004s
I20260709 14:54:00.906782  8363 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:00.907408  2612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:00.903148  8553 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.048266  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.026793  8306 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.061764  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:00.985028  8567 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.182s
I20260709 14:54:01.077284  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1691) took 1583 ms. Trace:
I20260709 14:54:01.077886  2757 rpcz_store.cc:276] 0709 14:53:59.493766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.748304 (+254538us) service_pool.cc:224] Handling call
0709 14:53:59.771712 (+ 23408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.783583 (+ 11871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.783610 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.820375 (+ 36765us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.069943 (+249568us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:00.070003 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.070100 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.076825 (+1006725us) raft_consensus.cc:1685] finished
0709 14:54:01.076857 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.077187 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8156,"raft.queue_time_us":110,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":9596,"thread_start_us":206746,"threads_started":1,"wal-append.queue_time_us":598261}
I20260709 14:54:01.090304  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.080248  8363 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:01.076257  8487 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:01.103924  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 331299us
I20260709 14:54:01.043702  8374 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:01.103746  8556 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
I20260709 14:54:01.133591  8553 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.130872  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.130810  8306 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.157948  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:01.160530  8306 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.163527  8306 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:01.096585  8536 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.142902  8374 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:01.143687  8563 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 14:54:01.144078  8565 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.190949  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1547) took 1685 ms. Trace:
I20260709 14:54:01.191668  2681 rpcz_store.cc:276] 0709 14:53:59.505647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.505815 (+   168us) service_pool.cc:224] Handling call
0709 14:54:01.190585 (+1684770us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400376754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.190818 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379778304}
I20260709 14:54:01.209751  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:01.147428  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1580) took 1707 ms. Trace:
I20260709 14:54:01.212924  2759 rpcz_store.cc:276] 0709 14:53:59.439893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.720651 (+280758us) service_pool.cc:224] Handling call
0709 14:53:59.721131 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.721746 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.721791 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.747573 (+ 25782us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.724635 (+977062us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800066b80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:00.855563 (+130928us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:00.855603 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.855679 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.146930 (+291251us) raft_consensus.cc:1685] finished
0709 14:54:01.146963 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.147312 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":730,"raft.queue_time_us":125,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":15187,"spinlock_wait_cycles":24870656,"thread_start_us":130493,"threads_started":1,"wal-append.queue_time_us":1386414}
I20260709 14:54:01.152405  8501 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:01.187680  8502 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.047s	user 0.010s	sys 0.006s
I20260709 14:54:01.220871  8233 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.125248  8564 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:01.094094  8579 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.225471  8537 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:01.153348  8548 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 14:54:01.124526  8591 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:01.231266  8591 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.232247  8536 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.232882  8536 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.096021  8562 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:54:01.142481  8566 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.235934  8399 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.235756  8566 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.236552  8399 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.237365  2681 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
I20260709 14:54:01.238351  8566 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.157193  2609 raft_consensus.cc:1217] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 15.8->[16.9-19.11]   Dedup: 19.11->[]
I20260709 14:54:01.238903  2612 raft_consensus.cc:606] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:01.239102  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:01.144685  8460 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.262s	user 0.013s	sys 0.003s
I20260709 14:54:01.222158  8564 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.239941  8460 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:01.240715  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:01.241075  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.241477  2757 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:01.241761  3546 raft_consensus.cc:2376] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16. Re-sending same reply.
I20260709 14:54:01.241977  8564 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:01.243891  8565 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:01.244719  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1569) took 2980 ms (client timeout 2047 ms). Trace:
I20260709 14:54:01.245108  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1603) took 1459 ms. Trace:
W20260709 14:54:01.245203  2752 rpcz_store.cc:269] 0709 14:53:58.264590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.365233 (+100643us) service_pool.cc:224] Handling call
0709 14:54:01.244435 (+2879202us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000d0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.244631 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817826176}
W20260709 14:54:01.204802  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.239132  2609 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: [16.9-19.11]
I20260709 14:54:01.247166  2753 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:01.247577  8382 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:01.248004  8382 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:01.195968  8374 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.155943  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1733) took 1397 ms. Trace:
W20260709 14:54:01.195343  8285 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.052s	user 0.014s	sys 0.003s
I20260709 14:54:01.199694  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1055) took 1445 ms. Trace:
W20260709 14:54:01.196659  8573 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.296s	user 0.002s	sys 0.000s
I20260709 14:54:01.205117  8518 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:01.249776  2607 rpcz_store.cc:276] 0709 14:53:59.758822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.796635 (+ 37813us) service_pool.cc:224] Handling call
0709 14:53:59.833369 (+ 36734us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 14:53:59.845115 (+ 11746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.845156 (+    41us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 14:53:59.863649 (+ 18493us) log.cc:844] Serialized 104 byte log entry
0709 14:54:00.178179 (+314530us) spinlock_profiling.cc:243] Waited 8.89 ms on lock 0x7b18004b1e40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:00.377475 (+199296us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:00.443377 (+ 65902us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.443845 (+   468us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.806152 (+362307us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.806420 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.806636 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.155600 (+348964us) raft_consensus.cc:1685] finished
0709 14:54:01.155629 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.155864 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68365,"prepare.queue_time_us":159362,"prepare.run_cpu_time_us":435,"prepare.run_wall_time_us":5448,"raft.queue_time_us":799977,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":65787,"spinlock_wait_cycles":20449024,"thread_start_us":560605,"threads_started":2,"wal-append.queue_time_us":798431}
I20260709 14:54:01.250103  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:01.226222  8537 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.207477  8504 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:01.211716  8583 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 14:54:01.253968  8399 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.217952  8502 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:01.144548  8555 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
W20260709 14:54:01.234586  2680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1538) took 2350 ms (client timeout 2151 ms). Trace:
W20260709 14:54:01.291146  2680 rpcz_store.cc:269] 0709 14:53:58.884305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.884500 (+   195us) service_pool.cc:224] Handling call
0709 14:54:01.234324 (+2349824us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.234499 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310787456}
I20260709 14:54:01.293182  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:01.294492  2680 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:01.239358  2612 raft_consensus.cc:614] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:01.244388  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:01.193801  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1598) took 1444 ms. Trace:
I20260709 14:54:01.266805  8374 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:01.206246  8503 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.469s	user 0.016s	sys 0.001s
W20260709 14:54:01.165571  2651 connection.cc:256] RPC call timeout handler was delayed by -1.0938802779999151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
I20260709 14:54:01.250061  2683 rpcz_store.cc:276] 0709 14:53:59.754569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.755394 (+   825us) service_pool.cc:224] Handling call
0709 14:53:59.764329 (+  8935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:00.044982 (+280653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.199351 (+1154369us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400423954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.199607 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606672768}
I20260709 14:54:01.250742  2759 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:01.250754  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.252269  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:01.182181  8387 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.314s	user 0.016s	sys 0.002s
I20260709 14:54:01.251520  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.387760  2605 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:01.388412  3545 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:54:01.249029  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1601) took 1475 ms. Trace:
I20260709 14:54:01.251994  2686 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 14:54:01.242599  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:01.265728  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:01.248996  8382 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.269084  2752 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:01.391314  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 125767us
I20260709 14:54:01.204850  8453 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:01.245738  2610 rpcz_store.cc:276] 0709 14:53:59.786189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.852222 (+ 66033us) service_pool.cc:224] Handling call
0709 14:53:59.852586 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.853056 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:53:59.885911 (+ 32855us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.108210 (+222299us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.117814 (+  9604us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.118043 (+   229us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:00.118315 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.121944 (+  3629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.122064 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.244858 (+1122794us) raft_consensus.cc:1685] finished
0709 14:54:01.244885 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.245040 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30739,"prepare.queue_time_us":91610,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":110,"raft.queue_time_us":4795,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":14633,"spinlock_wait_cycles":233856,"thread_start_us":103806,"threads_started":1,"wal-append.queue_time_us":901981}
I20260709 14:54:01.185702  3543 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
W20260709 14:54:01.212420  8478 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.830s	user 0.007s	sys 0.008s
I20260709 14:54:01.302429  8504 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:01.285079  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:01.297739  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.453544  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:01.196144  8493 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.302173  2604 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:01.454471  2759 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:01.267654  8518 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:01.317762  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.014s	sys 0.028s
I20260709 14:54:01.234233  8591 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:01.352751  3547 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.025s	sys 0.021s
I20260709 14:54:01.284655  8233 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.383116  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.547665  3546 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:01.296193  2606 rpcz_store.cc:276] 0709 14:53:59.749329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.749631 (+   302us) service_pool.cc:224] Handling call
0709 14:53:59.749984 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:00.124881 (+374897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.193350 (+1068469us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.193692 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860744448}
I20260709 14:54:01.390144  2609 rpcz_store.cc:276] 0709 14:53:59.773383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.846525 (+ 73142us) service_pool.cc:224] Handling call
0709 14:54:01.157007 (+1310482us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:01.247393 (+ 90386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.248813 (+  1420us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001eb750. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.248960 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012620032}
I20260709 14:54:01.304942  8564 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.273294  8285 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:01.549225  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.520535  8518 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:01.579218  8285 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:01.579429  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1692) took 2085 ms. Trace:
W20260709 14:54:01.547040  8559 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.001s
I20260709 14:54:01.352671  8565 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:01.460083  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 73120us
I20260709 14:54:01.457448  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:01.583714  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 126427us
I20260709 14:54:01.453953  2607 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:01.249442  8579 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:01.559845  8470 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.009s
I20260709 14:54:01.381789  8497 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.559571  8382 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:01.536885  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.020s	sys 0.034s
W20260709 14:54:01.620023  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.23726e+06us
W20260709 14:54:01.532467  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 247477us
I20260709 14:54:01.539644  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:01.588375  8590 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 14:54:01.456760  8381 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.639842  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:01.639523  8381 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.640733  2757 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:01.549225  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:01.643543  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1589) took 1883 ms (client timeout 1851 ms). Trace:
I20260709 14:54:01.644158  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1540) took 2746 ms. Trace:
I20260709 14:54:01.645154  2751 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 14:54:01.644603  2684 rpcz_store.cc:276] 0709 14:53:58.897559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.897789 (+   230us) service_pool.cc:224] Handling call
0709 14:53:58.921181 (+ 23392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:53:59.764389 (+843208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.643888 (+1879499us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400343854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.644090 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495978240}
I20260709 14:54:01.646052  8374 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:01.646637  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:01.647881  8239 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.571640  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.648694  2609 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:54:01.574272  8589 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 14:54:01.521162  8478 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 14:54:01.577046  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.397069  8387 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.453974  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1520) took 1588 ms. Trace:
I20260709 14:54:01.657228  2685 rpcz_store.cc:276] 0709 14:53:59.865637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.396385 (+530748us) service_pool.cc:224] Handling call
0709 14:54:00.436842 (+ 40457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:00.437483 (+   641us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:54:00.437528 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.461536 (+ 24008us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.717343 (+255807us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:00.717403 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.717496 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.402214 (+684718us) raft_consensus.cc:1685] finished
0709 14:54:01.453700 (+ 51486us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.453891 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112702,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":216,"thread_start_us":142419,"threads_started":1,"wal-append.queue_time_us":234697}
W20260709 14:54:01.545060  8106 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.421s	user 0.015s	sys 0.003s
I20260709 14:54:01.545522  7977 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:01.589452  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1060) took 1796 ms. Trace:
I20260709 14:54:01.661383  2686 rpcz_store.cc:276] 0709 14:53:59.792649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.063993 (+271344us) service_pool.cc:224] Handling call
0709 14:54:01.234522 (+1170529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:01.390327 (+155805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.589143 (+198816us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400445050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.589368 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2646280064}
I20260709 14:54:01.590302  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1720) took 2816 ms. Trace:
I20260709 14:54:01.587381  8592 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:01.598482  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.580109  2760 rpcz_store.cc:276] 0709 14:53:59.494111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.883421 (+389310us) service_pool.cc:224] Handling call
0709 14:53:59.962955 (+ 79534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:00.015255 (+ 52300us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:00.714557 (+699302us) log.cc:844] Serialized 14 byte log entry
0709 14:54:01.018212 (+303655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:01.018286 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:01.018355 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.579061 (+560706us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.579137 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.579343 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1618,"prepare.queue_time_us":158632,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":122,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":5652,"spinlock_wait_cycles":119533952,"thread_start_us":989269,"threads_started":2,"wal-append.queue_time_us":516282}
I20260709 14:54:01.599853  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:01.600378  8062 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:01.604453  8569 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.696s	user 0.006s	sys 0.005s
I20260709 14:54:01.605194  8597 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:01.605921  8582 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 14:54:01.606783  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:01.532377  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79626us
I20260709 14:54:01.672308  2683 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:54:01.672978  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:01.611068  8470 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.534389  2605 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
W20260709 14:54:01.602442  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.580817  8559 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:01.545125  2752 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:01.628870  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.688494  7977 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:01.689007  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.678572  8470 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.689785  7977 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:01.632977  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.691398  3547 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.662937  2611 rpcz_store.cc:276] 0709 14:53:58.774133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.789302 (+ 15169us) service_pool.cc:224] Handling call
0709 14:53:58.882455 (+ 93153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:58.944744 (+ 62289us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:53:59.153928 (+209184us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b18003d0ea0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:53:59.403700 (+249772us) log.cc:844] Serialized 14 byte log entry
0709 14:53:59.557887 (+154187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.568319 (+ 10432us) log.cc:844] Serialized 26 byte log entry
0709 14:53:59.568486 (+   167us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:53:59.568521 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:53:59.568595 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:00.597574 (+1028979us) raft_consensus.cc:1685] finished
0709 14:54:00.620733 (+ 23159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.590090 (+969357us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b5400128250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.590233 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90110,"prepare.queue_time_us":118100,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":111199,"raft.queue_time_us":62271,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":4315,"spinlock_wait_cycles":197676800,"thread_start_us":367964,"threads_started":2,"wal-append.queue_time_us":371268,"wal-append.run_cpu_time_us":10552,"wal-append.run_wall_time_us":1813424}
I20260709 14:54:01.691594  2609 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 13.
I20260709 14:54:01.692868  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.693672  2611 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:54:01.695407  7948 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.002s
I20260709 14:54:01.678452  8387 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:01.643970  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.452610  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.645628  2751 pending_rounds.cc:85] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 8
I20260709 14:54:01.570334  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.697507  8478 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:54:01.698387  3543 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:01.699491  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.634814  8593 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:01.644125  2755 rpcz_store.cc:269] 0709 14:53:59.759627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.773559 (+ 13932us) service_pool.cc:224] Handling call
0709 14:54:01.643311 (+1869752us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54002058d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.643459 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784552832}
I20260709 14:54:01.701068  2751 pending_rounds.cc:107] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260709 14:54:01.701068  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101309us
I20260709 14:54:01.701669  2751 raft_consensus.cc:2889] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:01.655645  3536 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:01.666659  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 17) took 1359 ms. Trace:
I20260709 14:54:01.587534  8591 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 14:54:01.667649  8062 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:01.677026  8597 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.717666  8503 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:01.718222  8597 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:01.718322  8503 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:01.679332  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:01.679646  8600 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 14:54:01.668432  8536 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.681375  2682 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:01.667899  8106 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.625918  8497 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.692971  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.759254  2609 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:01.695983  7948 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:01.700212  3543 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.704032  2612 rpcz_store.cc:276] 0709 14:54:00.339522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.901668 (+562146us) service_pool.cc:224] Handling call
0709 14:54:01.650314 (+748646us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.650456 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449795840}
W20260709 14:54:01.677464  8603 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 14:54:01.705862  2606 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:01.761710  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.699396  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1581) took 2259 ms. Trace:
I20260709 14:54:01.763036  2758 rpcz_store.cc:276] 0709 14:53:59.440485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.724540 (+284055us) service_pool.cc:224] Handling call
0709 14:53:59.725014 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.725551 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:00.073140 (+347589us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.732860 (+659720us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.745972 (+ 13112us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.746260 (+   288us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:00.746355 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.746488 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.460091 (+713603us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:01.546649 (+ 86558us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:54:01.697702 (+151053us) raft_consensus.cc:1685] finished
0709 14:54:01.699178 (+  1476us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.699329 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71388,"prepare.queue_time_us":389810,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":18663,"raft.run_cpu_time_us":6115,"raft.run_wall_time_us":1849764,"spinlock_wait_cycles":167233536,"thread_start_us":868272,"threads_started":2,"wal-append.queue_time_us":715948}
I20260709 14:54:01.582022  8564 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:01.633925  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1849302539999371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 14:54:01.775359  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.776605  8504 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.777172  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:01.777201  8504 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.779887  8504 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.785183  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.786301  3538 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:01.678400  8584 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 14:54:01.718938  8062 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.792392  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1522) took 1237 ms. Trace:
I20260709 14:54:01.792896  2679 rpcz_store.cc:276] 0709 14:54:00.554747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.581506 (+ 26759us) service_pool.cc:224] Handling call
0709 14:54:00.611662 (+ 30156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:00.614394 (+  2732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:00.614446 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.661537 (+ 47091us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.856218 (+194681us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:00.856266 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.856344 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.792065 (+935721us) raft_consensus.cc:1685] finished
0709 14:54:01.792094 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.792306 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9655,"raft.queue_time_us":55838,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":30085,"thread_start_us":87343,"threads_started":1,"wal-append.queue_time_us":576890}
I20260709 14:54:01.719241  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.795246  2610 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:01.796022  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1693) took 2301 ms. Trace:
I20260709 14:54:01.796489  2753 rpcz_store.cc:276] 0709 14:53:59.494428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.970293 (+475865us) service_pool.cc:224] Handling call
0709 14:54:01.246934 (+1276641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.247648 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.795769 (+548121us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000d0550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.795946 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2813405952}
I20260709 14:54:01.798095  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1694) took 2303 ms. Trace:
I20260709 14:54:01.719549  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1600) took 2294 ms. Trace:
W20260709 14:54:01.799028  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.799000  3542 rpcz_store.cc:276] 0709 14:53:59.425483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.427217 (+  1734us) service_pool.cc:224] Handling call
0709 14:53:59.427710 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.495624 (+ 67914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.495663 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.496312 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 14:53:59.716670 (+220358us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:53:59.716722 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.711322 (+1994600us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b5400200dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.711413 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.711504 (+    91us) raft_consensus.cc:1685] finished
0709 14:54:01.711524 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.719476 (+  7952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59077,"raft.queue_time_us":1444,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"spinlock_wait_cycles":155140864,"thread_start_us":212840,"threads_started":1,"wal-append.queue_time_us":253112,"wal-append.run_cpu_time_us":6890,"wal-append.run_wall_time_us":1690500}
W20260709 14:54:01.801131  8550 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.288s	user 0.000s	sys 0.002s
I20260709 14:54:01.801842  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.804104  8374 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:54:01.804603  8374 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:01.805543  8374 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.714445  8593 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.807410  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.808393  3542 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:54:01.809367  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.809602  8593 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.720917  8403 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:01.723033  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.810550  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.815407  3542 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:01.815591  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:01.816510  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.817029  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:01.817770  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.819053  8536 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:01.819499  8536 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:01.819881  8536 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:01.820366  8062 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.820744  8501 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.823916  2610 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:01.824621  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.825901  3546 raft_consensus.cc:2410] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:01.826395  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.724459  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1046) took 3382 ms. Trace:
I20260709 14:54:01.725466  8488 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.827184  2678 rpcz_store.cc:276] 0709 14:53:58.360614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:58.371668 (+ 11054us) service_pool.cc:224] Handling call
0709 14:53:58.399879 (+ 28211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.771089 (+1371210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.724138 (+1953049us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400269654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.724375 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2833964416}
I20260709 14:54:01.827502  8488 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.798555  2751 rpcz_store.cc:276] 0709 14:53:59.544762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.117181 (+572419us) service_pool.cc:224] Handling call
0709 14:54:01.644608 (+1527427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.702256 (+ 57648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.797859 (+ 95603us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002058d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.798025 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3511870208}
I20260709 14:54:01.827119  8399 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 14:54:01.830031  8399 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:54:01.830384  8399 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:01.713981  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:01.733472  2605 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:01.830964  8564 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:01.831283  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117432us
I20260709 14:54:01.831624  8564 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:01.831415  8399 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.750365  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:01.832778  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:01.833935  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.834465  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:01.747041  2611 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:01.834805  2607 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:54:01.756011  2760 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 14:54:01.835220  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1607) took 2016 ms. Trace:
I20260709 14:54:01.667115  8592 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.831813  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.832101  8564 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:01.836146  8062 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.826501  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:01.840512  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63429.6us
I20260709 14:54:01.836347  8592 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:01.841244  2611 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:54:01.837010  8564 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.859251  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1615) took 1800 ms. Trace:
I20260709 14:54:01.836561  2605 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
W20260709 14:54:01.860057  8572 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.805s	user 0.003s	sys 0.000s
I20260709 14:54:01.861409  2605 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:01.842461  8537 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.836762  8504 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.864141  8504 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:01.865792  8339 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:01.867093  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:01.868069  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1606) took 1091 ms. Trace:
I20260709 14:54:01.868494  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1696) took 2314 ms. Trace:
W20260709 14:54:01.869248  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1406) took 2141 ms (client timeout 2135 ms). Trace:
I20260709 14:54:01.843482  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:01.870080  2760 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.836943  8597 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:01.871097  8353 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:01.871526  8597 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:01.871949  8597 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:01.872944  8353 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:01.874285  8353 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:54:01.869750  3541 rpcz_store.cc:269] 0709 14:53:59.728037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.733644 (+  5607us) service_pool.cc:224] Handling call
0709 14:54:01.869015 (+2135371us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54004b64d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.869140 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176015744}
I20260709 14:54:01.876909  3541 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 10. Current term is 11. Ops: []
W20260709 14:54:01.877679  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1617) took 1739 ms (client timeout 2135 ms). Trace:
I20260709 14:54:01.873338  8592 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:01.878126  2604 rpcz_store.cc:269] 0709 14:54:00.138854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.722808 (+583954us) service_pool.cc:224] Handling call
0709 14:54:01.877498 (+1154690us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540050ffd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.877613 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317581184}
I20260709 14:54:01.845206  3545 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 14:54:01.879725  2604 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:01.845402  8502 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.846757  8488 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:01.837337  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47267.3us
I20260709 14:54:01.868409  3540 rpcz_store.cc:276] 0709 14:54:00.777222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.777473 (+   251us) service_pool.cc:224] Handling call
0709 14:54:01.866853 (+1089380us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800105998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:01.867994 (+  1141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969984}
I20260709 14:54:01.869015  2756 rpcz_store.cc:276] 0709 14:53:59.554495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.576557 (+1022062us) service_pool.cc:224] Handling call
0709 14:54:00.576992 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:00.613623 (+ 36631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:00.613669 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:00.661146 (+ 47477us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.806728 (+145582us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:00.806980 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.868024 (+1061044us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.868088 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.868150 (+    62us) raft_consensus.cc:1685] finished
0709 14:54:01.868164 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.868426 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19240,"raft.queue_time_us":3788,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":42535,"spinlock_wait_cycles":74470272,"thread_start_us":61319,"threads_started":1,"wal-append.queue_time_us":304343}
I20260709 14:54:01.883787  3540 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 14:54:01.844759  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:01.851269  2677 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:01.837265  8399 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:54:01.887562  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1549) took 2138 ms (client timeout 1982 ms). Trace:
I20260709 14:54:01.887975  2683 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:01.888532  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1521) took 1334 ms. Trace:
I20260709 14:54:01.889061  2605 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 14:54:01.851447  8374 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 14:54:01.888075  2677 rpcz_store.cc:269] 0709 14:53:59.749424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.750179 (+   755us) service_pool.cc:224] Handling call
0709 14:54:01.887315 (+2137136us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.887450 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4593783680}
I20260709 14:54:01.888940  2682 rpcz_store.cc:276] 0709 14:54:00.554338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.557699 (+  3361us) service_pool.cc:224] Handling call
0709 14:54:00.608411 (+ 50712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.741645 (+1133234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.888241 (+146596us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:01.888466 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467041536}
I20260709 14:54:01.893592  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:01.894376  2607 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:54:01.894618  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:01.859681  2611 rpcz_store.cc:276] 0709 14:54:00.059445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.840536 (+1781091us) service_pool.cc:224] Handling call
0709 14:54:01.840881 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.858866 (+ 17985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.859176 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:01.895443  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:01.874679  8353 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:01.896775  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:01.881075  2760 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:01.838439  3542 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:54:01.836612  8562 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:01.869218  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:01.876570  8576 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:01.827817  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:01.921999  2606 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:01.914894  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:01.922595  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96225.2us
I20260709 14:54:01.723852  3547 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:01.924105  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:01.924567  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:01.924881  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:01.925590  3547 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:01.925639  3546 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [13.8-17.10]
I20260709 14:54:01.835726  2608 rpcz_store.cc:276] 0709 14:53:59.818697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.873812 (+ 55115us) service_pool.cc:224] Handling call
0709 14:53:59.874370 (+   558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.886332 (+ 11962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:53:59.886372 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:53:59.933719 (+ 47347us) log.cc:844] Serialized 26 byte log entry
0709 14:54:00.191269 (+257550us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:00.380872 (+189603us) log.cc:844] Serialized 14 byte log entry
0709 14:54:00.596769 (+215897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:00.596890 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:01.800283 (+1203393us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:01.817289 (+ 17006us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b18003a43e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:01.835009 (+ 17720us) raft_consensus.cc:1685] finished
0709 14:54:01.835032 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:01.835164 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93562,"prepare.queue_time_us":172927,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":48128,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":1798,"spinlock_wait_cycles":46494464,"thread_start_us":374279,"threads_started":2,"wal-append.queue_time_us":367772}
I20260709 14:54:01.897617  2682 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:01.901374  2609 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 14:54:01.901695  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 66865.6us
I20260709 14:54:01.901855  7032 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:01.713374  8381 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:01.741483  8503 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:01.903522  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:01.933645  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1621) took 1766 ms. Trace:
I20260709 14:54:01.931820  2609 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:01.856061  2610 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 12. Current term is 13. Ops: []
I20260709 14:54:01.968394  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:02.087149  2680 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:01.980780  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:02.088326  2684 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:54:02.081730  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:02.080050  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:02.782917  2607 rpcz_store.cc:276] 0709 14:54:00.167104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.842657 (+1675553us) service_pool.cc:224] Handling call
0709 14:54:01.894011 (+ 51354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.897458 (+  3447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.933337 (+ 35879us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54002ff3d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:01.933557 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91949440}
W20260709 14:54:02.814909  2603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.014s	sys 0.017s
W20260709 14:54:02.815608  2603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.014s	sys 0.018s
I20260709 14:54:02.799041  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1020) took 1627 ms. Trace:
I20260709 14:54:02.816851  2610 rpcz_store.cc:276] 0709 14:54:01.145828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.855421 (+709593us) service_pool.cc:224] Handling call
0709 14:54:01.855844 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.772416 (+916572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.772885 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:02.083312  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:01.896806  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:02.089002  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 152332us
W20260709 14:54:02.188076  3411 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.606s (SENT)
W20260709 14:54:02.089677  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 164894us
I20260709 14:54:02.881352  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:02.082875  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:02.082079  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:02.083264  3538 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:02.884694  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1282) took 1180 ms. Trace:
I20260709 14:54:02.885274  3538 rpcz_store.cc:276] 0709 14:54:01.704227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.784787 (+ 80560us) service_pool.cc:224] Handling call
0709 14:54:02.884324 (+1099537us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:01.903571  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1274) took 1006 ms. Trace:
I20260709 14:54:02.160480  3543 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:01.926769  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:02.887390  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1428) took 1261 ms. Trace:
I20260709 14:54:02.887992  3543 rpcz_store.cc:276] 0709 14:54:01.625894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.699150 (+ 73256us) service_pool.cc:224] Handling call
0709 14:54:02.887184 (+1188034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:02.888937  8183 raft_consensus.cc:2764] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 14:54:02.099493  8062 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:54:02.245936  8319 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.015s
W20260709 14:54:02.891424  8319 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.007s	sys 0.018s
I20260709 14:54:01.897197  8353 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:02.893478  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1062) took 1794 ms. Trace:
I20260709 14:54:02.894935  8353 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:02.896726  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1067) took 1279 ms. Trace:
I20260709 14:54:02.897481  2684 rpcz_store.cc:276] 0709 14:54:01.617740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.647894 (+ 30154us) service_pool.cc:224] Handling call
0709 14:54:01.648247 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.806228 (+1157981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.895710 (+ 89482us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:02.896371 (+   661us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735233408}
I20260709 14:54:02.160758  8363 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:01.928625  7032 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:02.899834  8363 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:02.900642  8363 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:01.938213  8400 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 14:54:02.904474  8400 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 14:54:02.905222  8400 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:02.161106  8372 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 14:54:02.907205  8559 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:54:02.908231  8559 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:02.103440  8494 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.030s	sys 0.011s
I20260709 14:54:02.913215  8062 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:02.913823  8494 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.051s	sys 0.022s
I20260709 14:54:02.914232  8062 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:02.014364  8504 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:02.917117  2681 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:01.897888  8506 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:02.887017  3542 rpcz_store.cc:276] 0709 14:54:00.896650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.833142 (+936492us) service_pool.cc:224] Handling call
0709 14:54:01.833472 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.903204 (+ 69732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:01.903510 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:02.896818  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 813512us
I20260709 14:54:02.919248  8506 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:01.938925  8537 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 14:54:02.922117  8537 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:02.922746  8537 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:01.922303  8478 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:02.930105  8478 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:01.939090  8562 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:01.916935  8596 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:02.957583  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.05947e+06us
I20260709 14:54:02.959362  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:02.958942  8596 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:02.960799  8631 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:54:02.023698  8603 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 14:54:02.963366  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 775204us
I20260709 14:54:02.963891  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:02.020565  8605 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 14:54:02.217324  8579 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:02.968091  8579 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:02.099159  8599 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:02.726897  2677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.016s	sys 0.043s
I20260709 14:54:02.728160  8561 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:02.729341  8630 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:02.971127  2677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.016s	sys 0.045s
I20260709 14:54:02.971383  8599 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:02.971563  8561 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:02.971834  8630 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:02.973678  8599 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:02.633313  2751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.004s	sys 0.008s
W20260709 14:54:02.597647  2651 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.011s	sys 0.011s
W20260709 14:54:02.975550  2751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.012s	sys 0.011s
W20260709 14:54:02.668702  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.007s	sys 0.003s
W20260709 14:54:02.633809  2758 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.000s	sys 0.012s
W20260709 14:54:02.976260  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.012s	sys 0.008s
W20260709 14:54:02.976528  2758 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.004s	sys 0.018s
W20260709 14:54:02.618620  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:02.978186  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1416) took 2172 ms. Trace:
I20260709 14:54:02.968780  8579 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:02.978709  3537 rpcz_store.cc:276] 0709 14:54:00.805901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.826878 (+ 20977us) service_pool.cc:224] Handling call
0709 14:54:00.831537 (+  4659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:00.855771 (+ 24234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:00.855799 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:01.726414 (+870615us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b2c000e3d48. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:54:01.726795 (+   381us) log.cc:844] Serialized 26 byte log entry
0709 14:54:02.977283 (+1250488us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:02.977449 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.977648 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:02.977701 (+    53us) raft_consensus.cc:1685] finished
0709 14:54:02.977715 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.978078 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6649,"raft.queue_time_us":169294,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6235,"spinlock_wait_cycles":32844032,"thread_start_us":1060967,"threads_started":1,"wal-append.queue_time_us":732693}
I20260709 14:54:02.980688  8579 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:02.667814  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:02.981454  8578 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:02.540449  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:02.983407  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:02.984535  8578 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:02.985224  8578 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:02.975838  2651 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.012s	sys 0.025s
W20260709 14:54:02.267686  8305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.007s	sys 0.020s
W20260709 14:54:02.986748  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.36972e+06us
W20260709 14:54:02.987326  8305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.007s	sys 0.020s
W20260709 14:54:02.270511  8473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.854s	user 0.009s	sys 0.018s
W20260709 14:54:02.459869  8558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.001s	sys 0.040s
I20260709 14:54:02.561256  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:02.676956  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1741) took 1475 ms. Trace:
W20260709 14:54:02.320873  8608 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.259s	user 0.009s	sys 0.000s
W20260709 14:54:02.327082  8611 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 14:54:02.392549  8374 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
W20260709 14:54:02.451110  8518 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.872s	user 0.000s	sys 0.016s
W20260709 14:54:02.448051  8597 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.576s	user 0.015s	sys 0.000s
W20260709 14:54:02.397061  8399 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.001s
W20260709 14:54:02.436313  8564 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.003s
W20260709 14:54:02.407583  8536 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.587s	user 0.016s	sys 0.000s
W20260709 14:54:02.762615  7977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.008s	sys 0.004s
W20260709 14:54:02.762890  2757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.007s	sys 0.012s
I20260709 14:54:02.634220  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:02.766170  7948 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:02.766360  8306 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:02.767846  8595 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:02.765556  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.015s	sys 0.031s
W20260709 14:54:02.770992  8412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.008s	sys 0.019s
W20260709 14:54:02.771317  2611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.002s	sys 0.019s
I20260709 14:54:02.604048  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:02.763252  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.007s	sys 0.010s
I20260709 14:54:02.604060  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:02.782675  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.007s	sys 0.010s
I20260709 14:54:02.437942  8592 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:02.783341  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 879781us
I20260709 14:54:02.783560  8488 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:02.784279  8106 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:02.784515  8489 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:02.773138  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1750) took 1549 ms. Trace:
W20260709 14:54:02.778296  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.002s	sys 0.026s
W20260709 14:54:02.548673  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 679356us
I20260709 14:54:02.433720  8497 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:02.641542  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:03.069980  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.017s	sys 0.033s
I20260709 14:54:02.787770  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1418) took 1949 ms. Trace:
I20260709 14:54:02.796756  8487 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:02.803728  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1754) took 1271 ms. Trace:
W20260709 14:54:02.849917  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56562.9us
I20260709 14:54:02.861089  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1030) took 1156 ms. Trace:
I20260709 14:54:03.073359  2610 rpcz_store.cc:276] 0709 14:54:01.702609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.817905 (+1115296us) service_pool.cc:224] Handling call
0709 14:54:02.818504 (+   599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.855935 (+ 37431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:02.855975 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.858797 (+  2822us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:02.858877 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.858959 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.859138 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:03.074766  8305 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.075543  8597 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.076254  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1031) took 1373 ms. Trace:
I20260709 14:54:02.097954  8553 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.077558  2754 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:03.077778  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.003s	sys 0.026s
I20260709 14:54:03.077451  8553 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:02.874598  8638 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:02.910153  8372 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:02.955396  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1751) took 1711 ms. Trace:
W20260709 14:54:02.950156  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69430.8us
I20260709 14:54:02.968751  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1029) took 1371 ms. Trace:
I20260709 14:54:02.910020  8559 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:02.932610  8478 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.047183  7032 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:02.960879  8562 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:02.976114  8630 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:02.975569  8631 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:03.047952  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:03.060537  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 73977.2us
W20260709 14:54:03.065130  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.008s	sys 0.010s
W20260709 14:54:03.057119  2757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.007s	sys 0.013s
I20260709 14:54:03.056613  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1609) took 3234 ms. Trace:
I20260709 14:54:03.075915  8305 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:02.894176  2680 rpcz_store.cc:276] 0709 14:54:01.099365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.304425 (+205060us) service_pool.cc:224] Handling call
0709 14:54:01.351906 (+ 47481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.790953 (+1439047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.892769 (+101816us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400377154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:02.893156 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351069568}
W20260709 14:54:03.070957  2611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.003s	sys 0.019s
W20260709 14:54:03.070102  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.007s	sys 0.011s
W20260709 14:54:03.063978  7977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.009s	sys 0.004s
I20260709 14:54:03.078776  8597 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.080811  2610 rpcz_store.cc:276] 0709 14:54:01.703050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:03.074788 (+1371738us) service_pool.cc:224] Handling call
0709 14:54:03.075389 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:03.075845 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:03.075876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.075937 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:03.075973 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.076032 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.076189 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:03.067099  2609 rpcz_store.cc:276] 0709 14:54:01.224001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.931219 (+707218us) service_pool.cc:224] Handling call
0709 14:54:01.931587 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.772366 (+840779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.772885 (+   519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:02.989370  2605 rpcz_store.cc:276] 0709 14:54:01.201198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.891288 (+690090us) service_pool.cc:224] Handling call
0709 14:54:01.891608 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.891872 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:01.910584 (+ 18712us) log.cc:844] Serialized 14 byte log entry
0709 14:54:02.580374 (+669790us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b18004100e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:02.675624 (+ 95250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.675757 (+   133us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:02.676011 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.676203 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.676601 (+   398us) inbound_call.cc:177] Queueing success response
0709 14:54:02.680338 (+  3737us) spinlock_profiling.cc:243] Waited 8.13 ms on lock 0x7b18004100e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
Metrics: {"prepare.queue_time_us":18243,"prepare.run_cpu_time_us":6660,"prepare.run_wall_time_us":132653,"raft.queue_time_us":85158,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":236,"spinlock_wait_cycles":78612736,"thread_start_us":78040,"threads_started":1,"wal-append.queue_time_us":758312}
I20260709 14:54:03.090175  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1535) took 1135 ms. Trace:
W20260709 14:54:03.108584  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52627us
I20260709 14:54:03.071187  8489 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.060258  8596 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:03.089010  8412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.008s	sys 0.020s
I20260709 14:54:03.070755  3546 rpcz_store.cc:276] 0709 14:54:00.808789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.826685 (+1017896us) service_pool.cc:224] Handling call
0709 14:54:01.925196 (+ 98511us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:01.939727 (+ 14531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.724561 (+784834us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b54002ba7d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:02.756545 (+ 31984us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201271040}
I20260709 14:54:03.102308  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1624) took 1046 ms. Trace:
I20260709 14:54:03.073041  2608 rpcz_store.cc:276] 0709 14:54:01.531909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.802039 (+1270130us) service_pool.cc:224] Handling call
0709 14:54:02.802635 (+   596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.803157 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:02.803188 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.803238 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:02.803271 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.803329 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.803476 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:03.125458  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 564563us
I20260709 14:54:03.127526  2683 rpcz_store.cc:276] 0709 14:54:01.937862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.603205 (+665343us) service_pool.cc:224] Handling call
0709 14:54:03.072824 (+469619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53760}
I20260709 14:54:03.131630  8489 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:03.079054  8463 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:54:03.088387  2607 rpcz_store.cc:276] 0709 14:54:01.597131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.813263 (+1216132us) service_pool.cc:224] Handling call
0709 14:54:02.813936 (+   673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:02.814625 (+   689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:02.814665 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:02.822735 (+  8070us) log.cc:844] Serialized 26 byte log entry
0709 14:54:02.857816 (+ 35081us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:02.857879 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.861108 (+  3229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:02.968431 (+107323us) raft_consensus.cc:1685] finished
0709 14:54:02.968463 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.968652 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5813,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":198,"thread_start_us":34280,"threads_started":1,"wal-append.queue_time_us":129190}
I20260709 14:54:03.143158  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:03.152760  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.078944  8576 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:03.135592  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.21502e+06us
W20260709 14:54:03.151513  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 163676us
I20260709 14:54:03.139498  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1701) took 1509 ms. Trace:
I20260709 14:54:03.158067  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.157092  2758 rpcz_store.cc:276] 0709 14:54:01.630636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.765987 (+135351us) service_pool.cc:224] Handling call
0709 14:54:01.766498 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.766806 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:02.977276 (+1210470us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.067029 (+ 89753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.091199 (+ 24170us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:03.126402 (+ 35203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.135270 (+  8868us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.139373 (+  4103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":874,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":7376,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":38185,"thread_start_us":1195665,"threads_started":2}
I20260709 14:54:03.158402  8400 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:03.161005  8473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.662s	user 0.009s	sys 0.040s
I20260709 14:54:03.162376  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1703) took 1321 ms. Trace:
W20260709 14:54:03.163736  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 200146us
I20260709 14:54:03.163897  8400 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:54:03.165069  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.166976  8375 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 14:54:03.167788  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1623) took 1265 ms. Trace:
I20260709 14:54:03.168593  8375 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:03.169111  8375 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:03.171391  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1526) took 1279 ms. Trace:
I20260709 14:54:03.171902  2677 rpcz_store.cc:276] 0709 14:54:01.892155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.895003 (+  2848us) service_pool.cc:224] Handling call
0709 14:54:01.895305 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.895528 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:02.971885 (+1076357us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.139754 (+167869us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.146613 (+  6859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.152124 (+  5511us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:03.170962 (+ 18838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.171082 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.171317 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2817,"prepare.queue_time_us":500668,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1090,"raft.queue_time_us":12680,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":11308,"thread_start_us":1153378,"threads_started":2}
I20260709 14:54:03.173988  8643 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.174888  8643 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.178030  8643 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.178582  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1423) took 2075 ms. Trace:
I20260709 14:54:03.179203  3545 rpcz_store.cc:276] 0709 14:54:01.103452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.880681 (+777229us) service_pool.cc:224] Handling call
0709 14:54:01.881017 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:01.916409 (+ 35392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:01.916446 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:01.946780 (+ 30334us) log.cc:844] Serialized 26 byte log entry
0709 14:54:02.804136 (+857356us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:03.152197 (+348061us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:03.177971 (+ 25774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.178109 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:03.178169 (+    60us) raft_consensus.cc:1685] finished
0709 14:54:03.178187 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.178518 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202974,"raft.queue_time_us":40917,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":43116,"spinlock_wait_cycles":51000192,"thread_start_us":925878,"threads_started":1,"wal-append.queue_time_us":580618}
I20260709 14:54:03.183820  8504 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:03.140316  3540 rpcz_store.cc:276] 0709 14:54:01.903700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.885597 (+981897us) service_pool.cc:224] Handling call
0709 14:54:02.886402 (+   805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.942257 (+ 55855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:02.942322 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.947435 (+  5113us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:02.947512 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.947637 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.949540 (+  1903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:03.185256  8504 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:03.185987  8504 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:03.187220  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45584.4us
I20260709 14:54:03.189896  8319 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:03.191121  8453 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:03.190745  8319 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:03.192876  8453 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:03.193431  8453 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:03.088129  2612 rpcz_store.cc:276] 0709 14:54:01.224227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:02.597603 (+1373376us) service_pool.cc:224] Handling call
0709 14:54:02.598651 (+  1048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.612897 (+ 14246us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:02.613913 (+  1016us) log.cc:844] Serialized 14 byte log entry
0709 14:54:02.793444 (+179531us) log.cc:844] Serialized 14 byte log entry
0709 14:54:02.924108 (+130664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.924324 (+   216us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:02.927822 (+  3498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:02.927935 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:02.935302 (+  7367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30609,"prepare.queue_time_us":109799,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":1876,"raft.queue_time_us":56479,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":27757,"spinlock_wait_cycles":49480960,"thread_start_us":264143,"threads_started":2,"wal-append.queue_time_us":436241}
I20260709 14:54:03.194918  8453 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.197031  8453 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:03.190969  8487 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.201239  8487 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.201467  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1523) took 2634 ms. Trace:
I20260709 14:54:03.196774  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:03.202024  2681 rpcz_store.cc:276] 0709 14:54:00.569624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.239386 (+669762us) service_pool.cc:224] Handling call
0709 14:54:01.251804 (+ 12418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:02.917974 (+1666170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.201096 (+283122us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:03.201389 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3827662208}
I20260709 14:54:03.154510  8285 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:03.203413  8537 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.168286  3541 rpcz_store.cc:276] 0709 14:54:01.902362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.903688 (+  1326us) service_pool.cc:224] Handling call
0709 14:54:01.904300 (+   612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:01.904633 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:01.904673 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:01.905372 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 14:54:03.135362 (+1229990us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:03.150483 (+ 15121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.167417 (+ 16934us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:03.167478 (+    61us) raft_consensus.cc:1685] finished
0709 14:54:03.167493 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.167686 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1175292,"threads_started":1,"wal-append.queue_time_us":960751}
I20260709 14:54:03.163053  2751 rpcz_store.cc:276] 0709 14:54:01.840876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.841391 (+   515us) service_pool.cc:224] Handling call
0709 14:54:01.841737 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.841967 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:01.842277 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.056121 (+1213844us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.098983 (+ 42862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.109721 (+ 10738us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:03.145630 (+ 35909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.162099 (+ 16469us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.162305 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3777,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":47,"raft.queue_time_us":5089,"raft.run_cpu_time_us":3749,"raft.run_wall_time_us":102982,"spinlock_wait_cycles":13835008,"thread_start_us":1205545,"threads_started":2,"wal-append.queue_time_us":585435}
I20260709 14:54:03.211278  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:03.213048  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69088.4us
I20260709 14:54:03.213958  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.214848  8646 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.215972  8646 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.216733  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:03.218519  8637 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 14:54:03.219148  8536 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.222127  8488 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.223757  8487 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:03.224371  8487 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:03.224504  8494 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:03.224869  8487 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.225183  8488 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.226007  8497 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:03.218650  8183 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:03.105912  2603 rpcz_store.cc:276] 0709 14:53:59.819493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:59.880674 (+ 61181us) service_pool.cc:224] Handling call
0709 14:53:59.881265 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:53:59.886582 (+  5317us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:54:01.773422 (+1886840us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b1800218e60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf
0709 14:54:01.773736 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 14:54:02.844924 (+1071188us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:02.877529 (+ 32605us) log.cc:844] Serialized 26 byte log entry
0709 14:54:02.939419 (+ 61890us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:02.969816 (+ 30397us) log.cc:844] Serialized 14 byte log entry
0709 14:54:02.970243 (+   427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.036546 (+ 66303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:03.048590 (+ 12044us) raft_consensus.cc:1685] finished
0709 14:54:03.048624 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.053656 (+  5032us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107993,"prepare.queue_time_us":210748,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":128,"raft.queue_time_us":9839,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":336,"spinlock_wait_cycles":65847936,"thread_start_us":1146479,"threads_started":3,"wal-append.queue_time_us":1211118}
I20260709 14:54:03.139452  8502 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:03.228379  8502 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:03.228859  8305 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.229029  8561 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.193727  8319 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.230331  8478 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.178732  8372 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.232817  8576 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.225045  8494 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:03.235103  8494 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.211301  8553 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.226405  2760 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:03.236997  2608 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 14:54:03.229807  8502 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.214143  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.226183  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.250705  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1756) took 1330 ms. Trace:
I20260709 14:54:03.077061  8564 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.251428  2608 rpcz_store.cc:276] 0709 14:54:01.919917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:03.178149 (+1258232us) service_pool.cc:224] Handling call
0709 14:54:03.230263 (+ 52114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:03.248667 (+ 18404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.250566 (+  1899us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:03.184968  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:03.239969  8558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.434s	user 0.002s	sys 0.043s
I20260709 14:54:03.174583  8506 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.269089  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:03.272351  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58370.1us
I20260709 14:54:03.274448  8399 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.278667  3537 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:03.280766  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:03.285595  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120839us
I20260709 14:54:03.283975  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.283228  8506 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:54:03.281940  8487 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.287506  8631 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:03.253234  7948 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.290333  8631 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.297662  8638 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:03.298337  8487 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.275714  8646 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.276829  8536 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.226230  8595 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.151583  8518 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.303790  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1738) took 2103 ms. Trace:
W20260709 14:54:03.308130  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 123310us
W20260709 14:54:03.329924  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126799us
I20260709 14:54:03.282806  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:03.332568  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 135578us
I20260709 14:54:03.313257  3545 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:03.306542  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.170657  8375 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.319777  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1600) took 1785 ms. Trace:
I20260709 14:54:03.287249  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:03.322013  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 357244us
I20260709 14:54:03.315765  8638 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.348395  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:03.360755  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.363364  7948 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.364513  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:03.365473  2758 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:03.367261  3546 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.369007  8472 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:03.369560  8472 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:03.369977  8472 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:03.334668  8518 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.370971  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.371552  8638 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:03.373268  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:03.374639  3542 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:03.375061  8487 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.375883  8487 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.376466  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:03.376025  8399 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.367115  8557 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 14:54:03.378456  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 95780us
I20260709 14:54:03.335476  2604 rpcz_store.cc:276] 0709 14:54:01.200297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.880686 (+680389us) service_pool.cc:224] Handling call
0709 14:54:01.880981 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.881193 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:01.884855 (+  3662us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.303025 (+1418170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.303154 (+   129us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:03.303293 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.303438 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.303686 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":692,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":46690,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":86587,"thread_start_us":1417479,"threads_started":2,"wal-append.queue_time_us":1391078}
I20260709 14:54:03.375883  8576 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.332306  8631 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:03.383677  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77339us
I20260709 14:54:03.385051  8631 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:03.387260  8565 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:03.388861  8565 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:03.391319  8521 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:03.391947  8521 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:03.392124  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.393395  8521 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.396538  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.352654  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1755) took 1433 ms. Trace:
I20260709 14:54:03.397682  3537 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 14:54:03.399031  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.395161  8557 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:03.386605  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.407244  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.409776  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.398810  8631 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:03.411314  2681 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:03.412292  8631 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:03.414000  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 56432.6us
I20260709 14:54:03.366506  7948 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.424957  3541 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:03.438165  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.378535  8487 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:03.417775  8353 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.004s
I20260709 14:54:03.398013  2610 rpcz_store.cc:276] 0709 14:54:01.919218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:03.144930 (+1225712us) service_pool.cc:224] Handling call
0709 14:54:03.192700 (+ 47770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:03.193165 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:03.193695 (+   530us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.328266 (+134571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.328409 (+   143us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:03.352326 (+ 23917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.352435 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.352581 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":928,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":884,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":174,"thread_start_us":131214,"threads_started":1}
I20260709 14:54:03.418272  8518 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:03.457335  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 240786us
I20260709 14:54:03.458225  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1742) took 2256 ms. Trace:
W20260709 14:54:03.461977  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59816.1us
I20260709 14:54:03.350054  2757 rpcz_store.cc:276] 0709 14:54:01.534525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.649984 (+115459us) service_pool.cc:224] Handling call
0709 14:54:01.650410 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:01.651270 (+   860us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:54:01.667193 (+ 15923us) log.cc:844] Serialized 14 byte log entry
0709 14:54:01.845058 (+177865us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b180035d6a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:02.098897 (+253839us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:02.099754 (+   857us) log.cc:844] Serialized 52 byte log entry
0709 14:54:02.099985 (+   231us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:02.100285 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.251038 (+1150753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.259433 (+  8395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:03.275761 (+ 16328us) raft_consensus.cc:1685] finished
0709 14:54:03.275796 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.319655 (+ 43859us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42523,"prepare.queue_time_us":81493,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":116,"raft.queue_time_us":230685,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":13366,"spinlock_wait_cycles":59041152,"thread_start_us":1406609,"threads_started":2,"wal-append.queue_time_us":597734,"wal-append.run_cpu_time_us":3742,"wal-append.run_wall_time_us":1048817}
I20260709 14:54:03.448823  8564 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.475365  2677 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:03.466056  8487 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:03.470276  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0112758860000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2489s for a response
I20260709 14:54:03.466411  8106 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:03.469223  8553 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:03.505353  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.505892  8339 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:03.509307  8537 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:03.510952  8592 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:03.460477  8504 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:03.512763  8503 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.423s	user 0.008s	sys 0.023s
I20260709 14:54:03.512925  8504 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:03.513036  8472 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.515918  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:03.516968  2756 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:03.517632  2679 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:03.484359  8375 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.517333  8559 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:03.521226  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.521991  2610 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14. Re-sending same reply.
I20260709 14:54:03.523249  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.524004  2604 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:03.525700  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1748) took 2302 ms. Trace:
W20260709 14:54:03.526554  8641 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.235s	user 0.000s	sys 0.004s
I20260709 14:54:03.522250  8559 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:03.526935  8487 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.527587  8472 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:03.530978  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 49442.7us
W20260709 14:54:03.532967  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1281) took 1828 ms (client timeout 1646 ms). Trace:
I20260709 14:54:03.460750  2611 rpcz_store.cc:276] 0709 14:54:01.201486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.895527 (+694041us) service_pool.cc:224] Handling call
0709 14:54:01.895856 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:01.903367 (+  7511us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:54:03.131691 (+1228324us) log.cc:844] Serialized 14 byte log entry
0709 14:54:03.352896 (+221205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:03.358258 (+  5362us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:03.415272 (+ 57014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.443832 (+ 28560us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.458129 (+ 14297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30967,"prepare.queue_time_us":295095,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":6000,"raft.queue_time_us":28614,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":12080,"thread_start_us":1297038,"threads_started":2,"wal-append.queue_time_us":324426}
W20260709 14:54:03.535343  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 206508us
W20260709 14:54:03.536500  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.537834  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:03.539130  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:03.539479  8578 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.006s
I20260709 14:54:03.511010  8521 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.504572  8489 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.543370  8399 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.543740  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:03.545747  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 153808us
I20260709 14:54:03.526172  2606 rpcz_store.cc:276] 0709 14:54:01.223454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.922899 (+699445us) service_pool.cc:224] Handling call
0709 14:54:01.923316 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:02.964381 (+1041065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:02.964464 (+    83us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:02.964549 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:02.964658 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.525164 (+560506us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003c8554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:03.525237 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:03.525443 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392988416}
I20260709 14:54:03.555434  2754 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:03.580000  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.556011  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:03.469310  8557 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:03.547219  8643 consensus_queue.cc:1175] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 19 status { last_received { term: 19 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 19 index: 10 } }
I20260709 14:54:03.563217  2758 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:03.571460  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:03.587270  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:03.572943  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:03.558357  3536 rpcz_store.cc:269] 0709 14:54:01.703940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.757616 (+ 53676us) service_pool.cc:224] Handling call
0709 14:54:03.532471 (+1774855us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54004b5ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:03.532702 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377465600}
I20260709 14:54:03.552464  8559 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:03.575484  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:03.592419  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 205992us
I20260709 14:54:03.592046  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.598104  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.578799  8372 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.581496  8599 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:03.588362  8536 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:03.614238  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.618462  2757 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 11.
I20260709 14:54:03.618922  8564 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:54:03.619356  8374 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.619853  8652 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:03.620415  8652 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.620059  8374 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.620906  8652 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:03.621012  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129942us
I20260709 14:54:03.621287  8575 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 14:54:03.622206  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:03.623724  8657 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 14:54:03.624416  2576 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:54:03.624543  8659 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:54:03.625126  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59334.9us
I20260709 14:54:03.626089  8559 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.628369  8559 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:03.629768  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.631137  2686 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:03.623840  8463 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:03.626088  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:03.635123  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 160327us
W20260709 14:54:03.637583  8062 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.722s	user 0.018s	sys 0.001s
I20260709 14:54:03.640251  8472 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:03.643078  8575 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:03.644593  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:03.644973  8661 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 14:54:03.647209  8599 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.566639  8503 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.650761  3541 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:03.653335  2756 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:03.654242  8565 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:03.654618  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.654771  2685 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:03.656013  7977 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 14:54:03.657161  8579 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.676s	user 0.013s	sys 0.004s
I20260709 14:54:03.653363  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:03.579438  8578 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.658407  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.659174  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:03.659687  8374 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.665086  8062 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.663712  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:03.666023  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62333.1us
I20260709 14:54:03.666519  8339 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:03.668298  8489 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:03.670855  8557 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.672783  8487 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.675119  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.661082  8574 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:03.679491  8655 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 14:54:03.681412  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.699227  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.680706  8564 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:03.706219  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 117141us
I20260709 14:54:03.691730  7977 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:03.723150  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.719012  2610 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:54:03.698272  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53755.1us
W20260709 14:54:03.711575  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65011.6us
W20260709 14:54:03.712474  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86233us
W20260709 14:54:03.713263  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.714232  3542 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:03.688234  8659 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:03.729758  8659 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.697098  8575 consensus_queue.cc:1243] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:03.733657  2678 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:03.735268  8564 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.697688  8579 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.741026  8579 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.744300  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:03.746114  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.707089  8463 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:03.710126  8374 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:03.699062  8339 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:03.718288  3536 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:54:03.755106  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:03.756127  2683 pending_rounds.cc:85] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 10
I20260709 14:54:03.758229  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.760874  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.782047  2604 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:03.765651  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:03.767091  8668 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:03.767791  8487 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:54:03.768193  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 104657us
I20260709 14:54:03.768911  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.769482  8578 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.773126  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:03.773988  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 148028us
I20260709 14:54:03.775861  8062 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.777347  8372 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:03.778475  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:03.779865  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:03.784366  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89580.7us
I20260709 14:54:03.716660  8574 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.803755  8574 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.805254  7948 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.807101  2611 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:03.801781  8487 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:03.809787  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:03.809963  8487 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.810889  8592 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.811367  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.812458  2603 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:03.788143  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:03.814141  8497 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:03.780661  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.814736  8497 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:03.815248  2685 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:03.694854  8565 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.780545  2683 pending_rounds.cc:107] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 14:54:03.783866  8339 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.728281  8595 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.800006  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:03.794580  2756 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:03.801075  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.795967  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.803067  8631 raft_consensus.cc:2764] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 14:54:03.811884  8592 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.815946  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:03.794178  2607 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
W20260709 14:54:03.788166  8494 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.552s	user 0.006s	sys 0.009s
W20260709 14:54:03.754139  3411 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.489s (SENT)
W20260709 14:54:03.837071  8363 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.935s	user 0.013s	sys 0.006s
I20260709 14:54:03.753557  8374 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.785207  8504 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:03.903173  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.796265  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.823191  2683 raft_consensus.cc:2889] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:03.785168  8375 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:03.819630  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:03.799098  8372 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:03.831993  8565 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.806100  7948 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:03.853811  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1533) took 1954 ms. Trace:
I20260709 14:54:03.855063  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:03.857262  8591 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:03.857877  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61988.3us
I20260709 14:54:03.819828  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:03.865815  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:03.858345  8557 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:03.878948  8494 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.815935  8497 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.879151  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:03.879624  8575 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.948830  8579 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:03.948750  8575 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.950074  8579 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:03.950448  8494 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.950981  8591 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:03.951838  8575 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.953272  8591 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:03.879048  8183 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.882946  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.954758  2606 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:03.900226  8363 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:03.954418  8183 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:03.955004  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 75940.8us
I20260709 14:54:03.883188  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.955926  2608 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.956918  8363 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:03.957382  8658 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:03.957691  8363 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:54:03.897980  3546 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.003s
I20260709 14:54:03.869130  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:03.960078  3546 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:03.837774  8643 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 14:54:03.909613  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:03.915230  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 95746.6us
I20260709 14:54:03.853819  8578 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.912654  8521 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 14:54:03.922756  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:03.925820  8339 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:03.926452  8518 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:03.927079  8463 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:03.928252  7977 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:03.984040  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 159993us
I20260709 14:54:03.929570  8667 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:03.930197  8504 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:03.939322  2757 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:03.935467  2678 rpcz_store.cc:276] 0709 14:54:01.899764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:01.899958 (+   194us) service_pool.cc:224] Handling call
0709 14:54:02.082503 (+182545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:03.734306 (+1651803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:03.841219 (+106913us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400377654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:03.853685 (+ 12466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3716797824}
I20260709 14:54:03.907292  8374 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:03.939579  8630 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 14:54:03.911224  8576 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:03.942406  8557 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.976341  8575 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:03.955854  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:03.889959  2723 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.982s (SENT)
W20260709 14:54:03.954456  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 225761us
I20260709 14:54:03.913763  8536 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:03.959672  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.958582  8363 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 14:54:03.964356  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:03.966912  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 182763us
W20260709 14:54:03.967507  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58048.4us
I20260709 14:54:03.985009  8463 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:03.919174  8062 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:03.973412  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.980129  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:03.980805  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 171241us
I20260709 14:54:03.978992  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:04.034804  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70468.6us
W20260709 14:54:03.954479  8478 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.004s
I20260709 14:54:04.032864  8339 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:04.007185  8518 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:04.037887  8518 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.042322  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:04.043843  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 176327us
I20260709 14:54:04.045836  8400 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:04.046636  8400 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:04.047855  8400 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.048745  8372 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.049975  8285 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:04.050390  8372 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:04.051432  8671 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 14:54:04.033342  8363 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:04.051680  8504 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.052480  8363 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:04.055169  8537 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.003984  2607 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.056250  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:04.057281  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:03.979722  8562 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:04.026867  8557 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.058971  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.059211  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.059935  2680 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:04.060108  2681 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:04.060952  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.061071  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:03.933410  2605 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:54:03.950577  8579 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:04.062927  8579 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.064004  8658 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:54:03.956949  8183 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:03.964672  8643 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:04.065181  8643 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 10 FOLLOWER]: Advancing to term 11
I20260709 14:54:04.033479  8062 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:04.066524  8062 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:03.991299  8638 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.002s
I20260709 14:54:04.067215  8638 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.058101  8562 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:04.068320  8562 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:04.023061  7977 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:04.070152  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:04.072582  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116846us
I20260709 14:54:04.072790  8399 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:04.073357  8399 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:04.073719  8399 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:04.074436  8676 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 14:54:04.023878  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.069813  8638 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.024277  2610 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.076267  8638 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 14:54:04.027199  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:04.025343  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:03.984110  8503 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:04.004106  8631 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:04.077785  2753 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:04.078123  8631 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:04.079377  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.079190  8631 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.080044  2756 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.080672  8631 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:04.081393  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.082307  2757 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:04.084252  3537 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.085072  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.086009  3542 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:04.017004  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 201220us
I20260709 14:54:04.087160  3540 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 14:54:04.087625  3540 pending_rounds.cc:85] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 6
I20260709 14:54:04.087868  3540 pending_rounds.cc:107] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 14:54:04.088093  3540 raft_consensus.cc:2889] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:04.064296  8579 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:04.089367  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.089675  8599 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:04.008975  8667 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:04.093899  2758 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:04.090325  2751 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:04.089412  2760 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:04.073625  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.087492  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:04.091043  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45808us
I20260709 14:54:04.096457  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.107736  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.027642  8536 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:04.012271  8578 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.111675  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:04.109779  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.115152  2608 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:04.117880  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.118356  2609 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.119313  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.120165  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.111775  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.115423  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.033753  8463 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:04.116149  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.122895  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:04.124738  2723 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.151s (SENT)
I20260709 14:54:04.125726  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:04.119547  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:04.129372  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.099040  8667 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:04.136991  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.137780  3537 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:04.139616  3536 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:04.108825  8183 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:04.137645  2607 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 14:54:04.149391  3540 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:04.114202  8599 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:04.114691  8536 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:04.173280  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:04.161636  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.160454  2611 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:04.102196  8575 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.162936  8285 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:04.164400  8592 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.166312  8453 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:04.167397  8339 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:54:04.167922  3543 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 14:54:04.168401  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54838.8us
I20260709 14:54:04.170015  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.170413  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.169679  8537 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:04.172848  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.122071  8591 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:04.177398  2753 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.105880  8503 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 14:54:04.106096  8595 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 14:54:04.105903  8574 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.136741  8576 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:04.150080  8183 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:04.186163  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:04.200767  8575 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:04.289376  8183 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:04.262007  7948 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:04.290423  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.187919  8106 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 14:54:04.188771  8062 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.189528  2757 raft_consensus.cc:2376] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18. Re-sending same reply.
I20260709 14:54:04.192591  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.166380  8506 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 14:54:04.142841  8659 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 14:54:04.194797  3538 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.113951  8652 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:04.210529  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:04.221421  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134063us
W20260709 14:54:04.219565  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.204639  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.228592  2682 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:04.228526  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.223304  3537 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:04.242677  2681 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:04.243672  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.175823  8472 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:54:04.319104  8472 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:04.319816  8472 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:54:04.231241  2685 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:04.203536  8559 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.004s
I20260709 14:54:04.208305  8578 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:04.252135  2752 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [16.10-17.11]
W20260709 14:54:04.256228  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83487.4us
I20260709 14:54:04.324708  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:04.256933  2603 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:04.256997  8489 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:04.327596  8489 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:04.328197  8489 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:04.247020  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:04.194993  8677 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 14:54:04.226568  2756 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:04.334849  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.267573  3540 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:04.338090  2684 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:54:04.251482  8536 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.283016  8319 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:04.285269  8285 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:04.287882  8557 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:04.343921  8536 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.345861  8557 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:04.293916  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.347709  2751 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 14:54:04.304690  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.351076  2756 raft_consensus.cc:2368] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19.
I20260709 14:54:04.352363  8557 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:04.353313  8557 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.356823  8557 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:04.357160  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.358217  3540 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 14:54:04.364352  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.373349  8659 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:04.374012  8659 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:04.367624  8561 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:04.375272  8561 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:04.375042  8659 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:04.376618  8659 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.376470  8561 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.378328  8561 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.378381  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.392154  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.393863  8497 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:04.394591  8497 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:04.395974  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:04.393962  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.401612  8653 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:04.401983  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.382243  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.367230  8487 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:04.406492  2752 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:04.407907  8666 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:04.408479  8666 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:04.408583  8574 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.408156  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.409857  2752 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:04.409646  8666 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.411190  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.411749  8592 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:04.412374  8592 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:04.398433  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.423635  2757 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:04.423952  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 99386.5us
W20260709 14:54:04.427840  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.493602  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.494246  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.495054  2680 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:04.473780  8375 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:04.495877  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:04.510059  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 145836us
I20260709 14:54:04.517644  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:04.510314  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.453256  8559 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:04.530328  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134495us
I20260709 14:54:04.516412  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.529847  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.537077  2753 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:04.539039  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.540658  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:04.541324  2686 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:04.541428  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45695.7us
I20260709 14:54:04.542284  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:04.544296  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.545285  2753 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:04.548321  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.550877  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:04.552768  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 434996us
I20260709 14:54:04.553758  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.516851  8503 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:04.553328  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.532354  8339 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:04.536774  8666 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.559930  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.534585  8319 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:04.567447  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:04.563652  8503 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:04.570797  8372 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.520s	user 0.014s	sys 0.003s
I20260709 14:54:04.443351  8592 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.564865  8652 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:04.556259  7977 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:04.559058  8506 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:04.539137  8375 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:04.572115  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.625744  2756 raft_consensus.cc:2376] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15. Re-sending same reply.
I20260709 14:54:04.572489  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:04.628057  2603 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:04.573501  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:04.629932  2611 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:54:04.489899  8487 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.573972  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:04.573930  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:04.575870  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:04.580289  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:04.583988  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:04.594697  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.587587  3541 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:04.611086  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:04.603830  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.594702  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:04.573879  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:04.631543  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61329.9us
I20260709 14:54:04.633908  2753 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:04.785342  2759 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
I20260709 14:54:04.588294  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.786522  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.784982  2681 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:04.788447  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.615953  2752 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:04.621604  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:04.798429  2751 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:04.806623  3536 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.018s	sys 0.045s
W20260709 14:54:04.808221  3536 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.018s	sys 0.045s
W20260709 14:54:04.807956  2726 connection.cc:256] RPC call timeout handler was delayed by -4.0858413639998616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 14:54:04.571581  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:04.813588  2754 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:54:04.800184  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:04.784924  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.016s	sys 0.032s
W20260709 14:54:04.818032  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.017s	sys 0.032s
W20260709 14:54:04.818413  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 556588us
W20260709 14:54:04.819223  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:04.821393  2607 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 14:54:04.609928  8374 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.580s	user 0.016s	sys 0.008s
W20260709 14:54:04.827054  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:04.829493  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 241305us
W20260709 14:54:04.619763  8686 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 14:54:04.833572  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.817848  2685 raft_consensus.cc:2376] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17. Re-sending same reply.
I20260709 14:54:04.584626  8574 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:04.591012  8399 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.013s
W20260709 14:54:04.574368  8643 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.004s
I20260709 14:54:04.837251  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:04.622503  8579 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.014s
W20260709 14:54:04.898114  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64641.8us
I20260709 14:54:04.623142  8506 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:04.893548  3537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.056s	sys 0.067s
I20260709 14:54:04.576498  8062 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.899444  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:04.910034  3537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.057s	sys 0.067s
I20260709 14:54:04.910069  2604 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 14:54:04.893149  2609 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:04.900426  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:04.821256  8400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.027s	sys 0.044s
I20260709 14:54:04.905637  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.575398  8503 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:04.620421  7977 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:04.910647  2604 pending_rounds.cc:85] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 9
I20260709 14:54:04.924954  2604 pending_rounds.cc:107] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 14:54:04.925372  2604 raft_consensus.cc:2889] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:04.571498  8339 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:04.586585  8667 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:04.938349  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.886204  8575 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:54:04.941561  2723 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.552s (SENT)
I20260709 14:54:04.941563  3538 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:04.593051  8559 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.315s
I20260709 14:54:04.914036  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:04.947353  2685 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:54:04.951673  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.965474  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:54:04.926964  8400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.028s	sys 0.044s
I20260709 14:54:04.969360  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:04.975848  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61964.7us
I20260709 14:54:04.977730  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.971992  2678 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 14:54:04.913657  8688 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:04.980479  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:04.982450  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.906201  8652 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:54:04.972271  2723 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.575s (SENT)
I20260709 14:54:04.931908  8578 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:04.988466  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:04.991215  2686 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:04.963785  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:04.887760  8374 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:05.009032  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 437730us
I20260709 14:54:04.954192  8062 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:05.011087  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.020979  2753 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:04.914038  8565 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:05.016153  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:05.013377  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:04.906422  8399 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:04.914515  8643 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:05.007543  8372 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:54:04.988035  8319 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:05.041548  2607 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:04.913019  8574 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:05.067385  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.056s	sys 0.104s
I20260709 14:54:04.983727  8575 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:05.025575  8688 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:05.085088  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:05.050477  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.051731  8412 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:05.092866  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:05.052454  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.054230  8487 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:05.055517  8691 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 14:54:05.055354  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.055941  8565 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:05.059150  2760 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 17. Ops: []
I20260709 14:54:05.059885  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:05.061365  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 89157.3us
I20260709 14:54:05.110283  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:05.029636  2678 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:05.011937  8561 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.633s	user 0.025s	sys 0.008s
W20260709 14:54:05.069931  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.056s	sys 0.104s
W20260709 14:54:05.046062  8562 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.977s	user 0.012s	sys 0.005s
I20260709 14:54:05.025002  8400 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:54:05.047523  8363 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.994s	user 0.016s	sys 0.000s
I20260709 14:54:05.125639  8363 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:04.986444  8579 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:05.129779  8579 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:05.130398  8579 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:05.078678  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.137622  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.139331  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:05.140033  8591 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:05.015754  8375 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:05.151008  8487 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:54:05.156570  2607 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:05.079459  2684 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:54:04.912423  8497 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:05.076938  3411 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.032s	sys 0.010s
W20260709 14:54:05.176414  3411 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.032s	sys 0.010s
W20260709 14:54:05.176918  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 646214us
I20260709 14:54:05.098433  2685 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:05.047420  8372 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:05.158584  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1630) took 1075 ms. Trace:
I20260709 14:54:05.181819  3546 rpcz_store.cc:276] 0709 14:54:04.082805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.083291 (+   486us) service_pool.cc:224] Handling call
0709 14:54:04.083713 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:04.084046 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:54:04.533271 (+449225us) log.cc:844] Serialized 14 byte log entry
0709 14:54:05.119030 (+585759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:05.119128 (+    98us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:05.158231 (+ 39103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.158338 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.158496 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188539,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":20778,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":1008126,"threads_started":2}
I20260709 14:54:05.035979  8062 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:05.179749  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56721.7us
I20260709 14:54:05.155666  8363 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:05.090997  8412 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:05.189563  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:05.191149  8472 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.871s	user 0.017s	sys 0.000s
I20260709 14:54:05.115430  8537 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 14:54:05.193146  8339 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.076107  8506 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:05.195304  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.196635  8319 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:05.174450  8562 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:05.202997  8372 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:05.203837  8591 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:05.211925  8690 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 14:54:05.220613  8575 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:05.223752  2753 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:05.075455  7977 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:05.226464  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:05.238116  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.243471  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:05.236609  8518 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:05.243800  8659 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.867s	user 0.016s	sys 0.000s
I20260709 14:54:05.245260  2754 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:05.251312  3540 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 14:54:05.258616  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.251619  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:05.275566  2609 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:05.287686  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 222551us
I20260709 14:54:05.294250  2759 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:05.297355  2606 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:05.257560  3546 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:05.312031  2679 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:05.327874  2607 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:05.301230  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1054) took 1088 ms. Trace:
I20260709 14:54:05.330367  2758 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:05.305608  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:05.270984  8183 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.917s	user 0.011s	sys 0.005s
I20260709 14:54:05.349733  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1082) took 1616 ms. Trace:
I20260709 14:54:05.329537  2760 rpcz_store.cc:276] 0709 14:54:04.212909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.214969 (+  2060us) service_pool.cc:224] Handling call
0709 14:54:04.216024 (+  1055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.107980 (+891956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.300813 (+192833us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:05.301133 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938486144}
I20260709 14:54:05.331518  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:05.408937  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60101.1us
W20260709 14:54:05.261024  8697 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 14:54:05.303586  8666 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.728s	user 0.012s	sys 0.000s
I20260709 14:54:05.356315  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:05.420230  2759 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:54:05.287142  8339 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 14:54:05.409298  2683 rpcz_store.cc:276] 0709 14:54:03.733691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:03.747068 (+ 13377us) service_pool.cc:224] Handling call
0709 14:54:03.755761 (+  8693us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:03.981491 (+225730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:03.981546 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:04.009099 (+ 27553us) log.cc:844] Serialized 52 byte log entry
0709 14:54:04.540731 (+531632us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:04.540797 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:04.542980 (+  2183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:05.349449 (+806469us) raft_consensus.cc:1685] finished
0709 14:54:05.349477 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.349655 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":666,"raft.queue_time_us":15652,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":23899,"thread_start_us":497194,"threads_started":1,"wal-append.queue_time_us":1223686}
I20260709 14:54:05.256441  8399 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:05.439709  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 83529.3us
I20260709 14:54:05.409431  2755 raft_consensus.cc:2376] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11. Re-sending same reply.
I20260709 14:54:05.459236  2753 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:54:05.452572  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 375665us
I20260709 14:54:05.263753  8667 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:05.292488  8537 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 14:54:05.465440  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.470053  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.297768  8578 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:05.472263  2758 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:05.474363  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.475401  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:05.479312  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65508.2us
I20260709 14:54:05.346997  8651 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 14:54:05.461572  8652 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:05.497246  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:05.422155  8183 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:05.490058  2760 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:05.299178  8518 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:05.450942  8631 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:05.325917  8561 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.152s
W20260709 14:54:05.503144  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92798.3us
W20260709 14:54:05.503414  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 443652us
I20260709 14:54:05.471182  2759 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:05.246551  8412 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:05.255864  8472 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:05.519397  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.515863  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:05.469990  8536 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:05.289963  8591 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:05.518673  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.520328  2758 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:05.516422  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1604) took 1063 ms. Trace:
W20260709 14:54:05.542276  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 243649us
I20260709 14:54:05.490748  8643 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:05.577095  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:05.580974  3540 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:05.577759  3545 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:54:05.505448  8652 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:05.573999  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45644.5us
I20260709 14:54:05.572345  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.590553  2755 raft_consensus.cc:2376] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16. Re-sending same reply.
I20260709 14:54:05.591435  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1063) took 1214 ms. Trace:
I20260709 14:54:05.572996  8705 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:05.576869  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:05.540030  2759 rpcz_store.cc:276] 0709 14:54:04.453409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.465057 (+1011648us) service_pool.cc:224] Handling call
0709 14:54:05.516343 (+ 51286us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:05.589476  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73689us
I20260709 14:54:05.594130  8487 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:05.522716  8631 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:05.522310  8659 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:05.595264  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.026s	sys 0.062s
I20260709 14:54:05.518800  8574 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:05.596055  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98960.7us
W20260709 14:54:05.599742  8635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.006s	sys 0.009s
I20260709 14:54:05.579705  8561 consensus_queue.cc:1243] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 14:54:05.599478  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.027s	sys 0.062s
I20260709 14:54:05.597369  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.611641  2760 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:05.604700  8659 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:05.613323  8518 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:05.614159  8659 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.615993  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.617161  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1076) took 1014 ms. Trace:
I20260709 14:54:05.617607  2756 rpcz_store.cc:276] 0709 14:54:04.602589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.615639 (+1013050us) service_pool.cc:224] Handling call
0709 14:54:05.617069 (+  1430us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:05.603928  8635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.006s	sys 0.009s
I20260709 14:54:05.626577  8591 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.599565  8536 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:05.606130  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1448) took 1254 ms. Trace:
I20260709 14:54:05.636183  3537 rpcz_store.cc:276] 0709 14:54:04.351760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.353149 (+  1389us) service_pool.cc:224] Handling call
0709 14:54:04.353730 (+   581us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:04.354446 (+   716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:04.354493 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:04.355322 (+   829us) log.cc:844] Serialized 26 byte log entry
0709 14:54:04.910700 (+555378us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:04.910747 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:04.910854 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:05.605866 (+695012us) raft_consensus.cc:1685] finished
0709 14:54:05.605892 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.606057 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1892,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":174,"thread_start_us":554853,"threads_started":1,"wal-append.queue_time_us":767980}
I20260709 14:54:05.591903  2755 rpcz_store.cc:276] 0709 14:54:04.376511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.514472 (+1137961us) service_pool.cc:224] Handling call
0709 14:54:05.591333 (+ 76861us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.519739  8565 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:05.518503  8666 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 14:54:05.628443  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.511909  8651 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:05.613070  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.583757  8374 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.597268  8652 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:05.631850  2751 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.832s	user 0.009s	sys 0.005s
I20260709 14:54:05.632360  2685 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:05.410090  8688 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:05.635013  8489 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.290s	user 0.012s	sys 0.003s
I20260709 14:54:05.639220  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:05.688417  2755 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:05.653225  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.604384  8705 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:05.644122  8698 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 14:54:05.660725  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:05.662199  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118365us
I20260709 14:54:05.605466  8562 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.611105  8412 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:05.668949  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73805.7us
I20260709 14:54:05.670369  2756 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:05.702900  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1609) took 1145 ms. Trace:
I20260709 14:54:05.597976  8487 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:05.680127  2751 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:05.703320  2756 rpcz_store.cc:276] 0709 14:54:04.557701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.618501 (+1060800us) service_pool.cc:224] Handling call
0709 14:54:05.702798 (+ 84297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.704641  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:05.685564  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.705349  2756 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:05.637277  8062 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:05.687086  3536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.009s	sys 0.016s
I20260709 14:54:05.690512  8489 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:05.708576  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1087) took 1098 ms. Trace:
W20260709 14:54:05.708786  3536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.010s	sys 0.016s
I20260709 14:54:05.709097  2751 rpcz_store.cc:276] 0709 14:54:04.605865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.621138 (+ 15273us) service_pool.cc:224] Handling call
0709 14:54:05.703858 (+1082720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.710522  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:05.711083  8489 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:05.695477  2759 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:05.711707  2751 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:05.712972  8487 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:05.713300  2759 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 14:54:05.714058  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1059) took 1499 ms. Trace:
I20260709 14:54:05.715269  2682 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:05.637606  8339 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 14:54:05.715931  8374 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:05.716879  8374 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:05.717618  8705 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:05.718592  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1085) took 1113 ms. Trace:
I20260709 14:54:05.719355  2751 rpcz_store.cc:276] 0709 14:54:04.605280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.710069 (+1104789us) service_pool.cc:224] Handling call
0709 14:54:05.712412 (+  2343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.722069  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1721) took 1390 ms. Trace:
W20260709 14:54:05.722540  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 88943us
I20260709 14:54:05.724826  8183 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:05.728909  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:05.678483  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1086) took 1072 ms. Trace:
I20260709 14:54:05.729503  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:05.729821  2760 rpcz_store.cc:276] 0709 14:54:04.605541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.612678 (+1007137us) service_pool.cc:224] Handling call
0709 14:54:05.678377 (+ 65699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.731166  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:05.731491  3537 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 14:54:05.715533  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1637) took 1543 ms. Trace:
I20260709 14:54:05.731696  2760 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:05.730000  3540 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:05.732491  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1062) took 1356 ms. Trace:
I20260709 14:54:05.637753  8578 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:05.732074  3536 rpcz_store.cc:276] 0709 14:54:04.171635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.172478 (+   843us) service_pool.cc:224] Handling call
0709 14:54:04.190599 (+ 18121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:04.191177 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:04.818465 (+627288us) log.cc:844] Serialized 14 byte log entry
0709 14:54:05.715088 (+896623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:05.715160 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:05.715235 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.715323 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.715466 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5583,"prepare.queue_time_us":81598,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":16251,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":111,"thread_start_us":1507100,"threads_started":2}
I20260709 14:54:05.729285  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1640) took 1276 ms. Trace:
I20260709 14:54:05.700786  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:05.734313  2685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:54:05.739004  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:05.739509  2685 raft_consensus.cc:606] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:05.732960  2760 rpcz_store.cc:276] 0709 14:54:04.376159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.730939 (+1354780us) service_pool.cc:224] Handling call
0709 14:54:05.731338 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.732191 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.732427 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.714421  2759 rpcz_store.cc:276] 0709 14:54:04.223552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.712425 (+1488873us) service_pool.cc:224] Handling call
0709 14:54:05.712844 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.713727 (+   883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.713976 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.722630  2751 rpcz_store.cc:276] 0709 14:54:04.332105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.720552 (+1388447us) service_pool.cc:224] Handling call
0709 14:54:05.721104 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.721416 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:05.721471 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:05.721554 (+    83us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:05.721633 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.721738 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.721957 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:05.740162  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 135260us
I20260709 14:54:05.750056  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:05.708693  8712 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:05.755443  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:05.701419  8643 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:05.761682  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:05.762450  3540 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:05.737651  3543 rpcz_store.cc:276] 0709 14:54:04.450530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.450750 (+   220us) service_pool.cc:224] Handling call
0709 14:54:04.489352 (+ 38602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:04.489829 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:04.489866 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:04.612152 (+122286us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b18003f3ca0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:05.147627 (+535475us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b2c000e3d48. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:54:05.148329 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 14:54:05.501003 (+352674us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:05.501081 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.501201 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:05.726370 (+225169us) raft_consensus.cc:1685] finished
0709 14:54:05.726420 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.726714 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":24329,"spinlock_wait_cycles":96592640,"thread_start_us":328454,"threads_started":1,"wal-append.queue_time_us":454786}
I20260709 14:54:05.671082  8591 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 14:54:05.761873  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:05.777781  3537 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:05.752259  8183 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:05.752492  8062 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:05.779256  2756 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:05.760288  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.814913  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1723) took 1234 ms. Trace:
W20260709 14:54:05.815371  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.815337  2756 rpcz_store.cc:276] 0709 14:54:04.580865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.704279 (+1123414us) service_pool.cc:224] Handling call
0709 14:54:05.814805 (+110526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.819366  8503 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 14:54:05.792822  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1614) took 1214 ms. Trace:
I20260709 14:54:05.820484  2751 rpcz_store.cc:276] 0709 14:54:04.578392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.744016 (+1165624us) service_pool.cc:224] Handling call
0709 14:54:05.744384 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.748986 (+  4602us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:05.789143 (+ 40157us) log.cc:844] Serialized 14 byte log entry
0709 14:54:05.792016 (+  2873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:05.792116 (+   100us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:05.792256 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.792365 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.792627 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39977,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":166,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"thread_start_us":13165,"threads_started":2,"wal-append.queue_time_us":2226}
I20260709 14:54:05.823683  2751 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:05.824666  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1617) took 1245 ms. Trace:
I20260709 14:54:05.825022  2751 rpcz_store.cc:276] 0709 14:54:04.579265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.822928 (+1243663us) service_pool.cc:224] Handling call
0709 14:54:05.823411 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.824356 (+   945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.824603 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.827936  2751 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:05.828776  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1618) took 1249 ms. Trace:
I20260709 14:54:05.829301  2751 rpcz_store.cc:276] 0709 14:54:04.579574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.826409 (+1246835us) service_pool.cc:224] Handling call
0709 14:54:05.827688 (+  1279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.828445 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.828704 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.859136  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:05.860219  2603 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:05.798985  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1615) took 1220 ms. Trace:
I20260709 14:54:05.907172  2753 rpcz_store.cc:276] 0709 14:54:04.578676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.038702 (+460026us) service_pool.cc:224] Handling call
0709 14:54:05.798696 (+759994us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400236ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:05.798900 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266927360}
I20260709 14:54:05.942134  2603 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:54:05.944185  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 241703us
I20260709 14:54:05.759017  8710 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:05.952097  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:05.967520  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:05.969069  3543 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:05.976091  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 279009us
W20260709 14:54:05.999172  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:06.000640  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1606) took 1436 ms. Trace:
I20260709 14:54:06.001391  2759 rpcz_store.cc:276] 0709 14:54:04.529698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.748736 (+1219038us) service_pool.cc:224] Handling call
0709 14:54:05.762706 (+ 13970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.948330 (+185624us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:05.948392 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:05.965495 (+ 17103us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:05.965588 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:05.965719 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:05.965931 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7276,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6066}
I20260709 14:54:05.978608  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:06.015075  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.016309  2603 raft_consensus.cc:2368] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:06.022207  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:06.027462  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55923.5us
W20260709 14:54:06.020466  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49327.9us
I20260709 14:54:06.029470  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:05.945998  8631 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:06.044457  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65964.1us
I20260709 14:54:06.050431  3536 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:06.051489  2759 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:06.052628  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1075) took 1450 ms. Trace:
I20260709 14:54:06.053197  2759 rpcz_store.cc:276] 0709 14:54:04.602299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.008817 (+1406518us) service_pool.cc:224] Handling call
0709 14:54:06.043037 (+ 34220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.052181 (+  9144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.052537 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:06.010182  2650 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.920s (SENT) [suppressed 31 similar messages]
I20260709 14:54:06.060334  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.062009  2755 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:06.062978  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1724) took 1481 ms. Trace:
I20260709 14:54:06.063558  2755 rpcz_store.cc:276] 0709 14:54:04.581165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.638395 (+1057230us) service_pool.cc:224] Handling call
0709 14:54:06.062873 (+424478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:05.998056  8503 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.011201  8688 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.119071  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.112054  3543 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 14:54:06.120707  3543 pending_rounds.cc:85] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 10
I20260709 14:54:06.121222  3543 pending_rounds.cc:107] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 14:54:06.121806  3543 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:06.011808  8643 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:06.023696  8705 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.144214  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.145882  2604 raft_consensus.cc:2368] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:06.056156  8565 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:06.055377  8591 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:06.121531  8504 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.161154  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:06.172374  3536 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 19
W20260709 14:54:06.173761  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 144380us
I20260709 14:54:06.182328  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.184005  2603 raft_consensus.cc:2368] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:06.136556  2759 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:06.191810  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1725) took 1584 ms. Trace:
I20260709 14:54:06.192477  2759 rpcz_store.cc:276] 0709 14:54:04.607954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.054445 (+1446491us) service_pool.cc:224] Handling call
0709 14:54:06.135839 (+ 81394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.191331 (+ 55492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.191715 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:06.124578  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 114436us
W20260709 14:54:06.194470  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53986.8us
I20260709 14:54:06.198426  3540 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
W20260709 14:54:06.199373  8716 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 14:54:06.199826  7977 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:54:06.200614  8714 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 14:54:06.201822  2755 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:06.203852  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.205634  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.207792  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.210307  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.159070  8565 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:06.200521  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:06.173478  8463 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 2.025s	user 0.027s	sys 0.006s
W20260709 14:54:06.222572  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:06.171557  8503 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.139626  8688 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.226135  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.148926  8643 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.227936  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1616) took 1649 ms. Trace:
I20260709 14:54:06.229528  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.153271  8705 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.235409  2756 rpcz_store.cc:276] 0709 14:54:04.578994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.816153 (+1237159us) service_pool.cc:224] Handling call
0709 14:54:05.816518 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.816822 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:05.817138 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 14:54:06.227451 (+410313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:06.227533 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:06.227600 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.227698 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:06.227866 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3013,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":48,"raft.queue_time_us":4900,"thread_start_us":405116,"threads_started":1}
W20260709 14:54:06.239082  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 178855us
I20260709 14:54:06.241262  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.221920  8565 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:06.221809  8183 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:06.180505  8504 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.251850  2759 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:06.258464  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:06.259318  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49118.4us
I20260709 14:54:06.260757  2759 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:06.261270  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:06.224500  8463 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.257196  8536 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:54:06.278805  8579 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.102s	user 0.013s	sys 0.000s
W20260709 14:54:06.280570  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1056) took 1698 ms (client timeout 1582 ms). Trace:
W20260709 14:54:06.265218  8719 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 14:54:06.299654  2610 rpcz_store.cc:269] 0709 14:54:04.582213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.583706 (+  1493us) service_pool.cc:224] Handling call
0709 14:54:06.280288 (+1696582us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.280468 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749875968}
I20260709 14:54:06.300841  8579 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:06.283118  2612 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 14:54:06.268708  8643 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:06.302438  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1619) took 1722 ms. Trace:
W20260709 14:54:06.268272  3541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1628) took 2212 ms (client timeout 1920 ms). Trace:
W20260709 14:54:06.303021  3541 rpcz_store.cc:269] 0709 14:54:04.056109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.056983 (+   874us) service_pool.cc:224] Handling call
0709 14:54:06.267983 (+2211000us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b5400550854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.268131 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189334272}
I20260709 14:54:06.304250  8705 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:06.302875  2751 rpcz_store.cc:276] 0709 14:54:04.579877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.830165 (+1250288us) service_pool.cc:224] Handling call
0709 14:54:05.830612 (+   447us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:05.952363 (+121751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:05.952414 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:06.013272 (+ 60858us) log.cc:844] Serialized 78 byte log entry
0709 14:54:06.225428 (+212156us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:06.253926 (+ 28498us) log.cc:844] Serialized 14 byte log entry
0709 14:54:06.254221 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.254289 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:06.302147 (+ 47858us) raft_consensus.cc:1685] finished
0709 14:54:06.302171 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:06.302370 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115652,"prepare.queue_time_us":28293,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":135469,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":33971,"thread_start_us":102500,"threads_started":1,"wal-append.queue_time_us":239451}
I20260709 14:54:06.243377  3540 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:06.270406  8708 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 14:54:06.265218  8718 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 14:54:06.305250  8708 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:06.277307  8503 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:06.278494  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1059) took 1142 ms. Trace:
I20260709 14:54:06.306722  2607 rpcz_store.cc:276] 0709 14:54:05.136067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.138950 (+  2883us) service_pool.cc:224] Handling call
0709 14:54:06.278244 (+1139294us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.278385 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37527424}
I20260709 14:54:06.306428  8708 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:06.307928  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.308331  8708 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:06.308744  8579 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:06.309407  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.310487  2726 leader_election.cc:400] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:06.279850  8688 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:06.273393  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:06.252946  3543 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:06.302235  3542 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:06.279798  8363 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 14:54:06.286078  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1641) took 1692 ms (client timeout 2009 ms). Trace:
W20260709 14:54:06.313201  8372 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.047s	user 0.011s	sys 0.003s
W20260709 14:54:06.287389  8412 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.001s
I20260709 14:54:06.314548  2686 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:06.295714  8504 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:06.299082  8652 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.007s
I20260709 14:54:06.316323  8652 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:06.317267  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:06.317900  8472 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:06.311842  8710 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:06.318894  8652 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:06.313303  3538 rpcz_store.cc:269] 0709 14:54:04.594196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.594362 (+   166us) service_pool.cc:224] Handling call
0709 14:54:06.285799 (+1691437us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54004c4ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.285972 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331666816}
I20260709 14:54:06.290619  3536 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:06.312805  8363 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:06.320317  8717 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:06.320703  8591 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:06.310767  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.274129  8183 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 14:54:06.312610  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.311528  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.293104  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:06.326073  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:06.323086  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.327013  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:06.328226  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.331391  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.262264  2759 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.282620  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:06.333645  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51122.2us
I20260709 14:54:06.339633  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.340461  3546 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:06.344067  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:06.344877  3536 raft_consensus.cc:2376] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17. Re-sending same reply.
I20260709 14:54:06.349452  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:06.349776  2610 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:06.351930  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.353463  2604 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:54:06.356547  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.332801  8564 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:06.358690  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.360167  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.361334  2685 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:06.363071  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.367218  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.320855  8717 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:06.373517  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.369406  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.370051  2683 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 14:54:06.375084  2683 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 9
I20260709 14:54:06.375465  2683 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 14:54:06.375864  2683 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:06.285983  8463 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:06.337759  8472 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:06.358167  8564 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.343039  8702 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.346168  8652 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:06.387409  8717 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.388433  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.354444  8659 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 14:54:06.391572  8659 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:06.392140  8707 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.729s	user 0.004s	sys 0.008s
I20260709 14:54:06.392920  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.395833  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.396111  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.354159  8578 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:06.399616  8578 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 14:54:06.400174  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1605) took 1925 ms (client timeout 2006 ms). Trace:
W20260709 14:54:06.400913  2757 rpcz_store.cc:269] 0709 14:54:04.475040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.536674 (+ 61634us) service_pool.cc:224] Handling call
0709 14:54:06.399873 (+1863199us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.400084 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3211305344}
I20260709 14:54:06.383123  8503 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:06.404062  8503 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:06.405319  8503 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.407481  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.409202  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.411198  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.374528  2684 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:06.377839  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46601.6us
I20260709 14:54:06.332559  8183 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:06.338567  8688 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:06.345551  7977 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:06.372550  2759 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:06.382776  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.402156  8578 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:06.417811  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.425457  2685 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:06.426945  8631 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:06.382951  8715 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 14:54:06.385828  8700 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.963s	user 0.007s	sys 0.007s
I20260709 14:54:06.386206  8702 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.376881  8591 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 14:54:06.430168  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.386610  8652 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:06.431054  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.434017  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1778) took 1599 ms. Trace:
I20260709 14:54:06.505676  2612 rpcz_store.cc:276] 0709 14:54:04.834705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.834944 (+   239us) service_pool.cc:224] Handling call
0709 14:54:06.282713 (+1447769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.302567 (+ 19854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.433698 (+131131us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400530050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.433925 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328488320}
I20260709 14:54:06.434336  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:06.435346  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1082) took 1831 ms (client timeout 1736 ms). Trace:
I20260709 14:54:06.435590  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1453) took 1209 ms. Trace:
I20260709 14:54:06.507929  3542 rpcz_store.cc:276] 0709 14:54:05.225936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.226090 (+   154us) service_pool.cc:224] Handling call
0709 14:54:06.435385 (+1209295us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400225454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.435525 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2318278528}
I20260709 14:54:06.436478  8183 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:06.453675  8643 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:06.385648  8564 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:06.439555  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56927.3us
I20260709 14:54:06.450402  2726 leader_election.cc:400] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:06.524367  2726 leader_election.cc:403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 14:54:06.454061  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:06.525386  2607 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:06.479682  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.528296  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.530071  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.481212  2759 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:54:06.531437  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1622) took 1246 ms. Trace:
I20260709 14:54:06.531888  2759 rpcz_store.cc:276] 0709 14:54:05.285111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.194071 (+908960us) service_pool.cc:224] Handling call
0709 14:54:06.240462 (+ 46391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.531016 (+290554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.531357 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:06.501770  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:06.507859  2754 rpcz_store.cc:269] 0709 14:54:04.604373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.919519 (+315146us) service_pool.cc:224] Handling call
0709 14:54:06.435130 (+1515611us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400570b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.435265 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466965888}
I20260709 14:54:06.313788  8372 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:06.517165  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.552986  2680 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:06.518611  2604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:06.555517  2604 raft_consensus.cc:606] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:06.555773  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:06.557533  8375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.392s	user 0.028s	sys 0.022s
W20260709 14:54:06.558252  8375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.028s	sys 0.022s
I20260709 14:54:06.479127  8715 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:06.563840  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:06.561019  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 207468us
I20260709 14:54:06.510074  8688 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.511052  8658 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.573098  2612 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:06.515132  8659 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:06.380678  8363 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:06.592022  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.600591  2612 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:06.598178  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.599182  3541 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
W20260709 14:54:06.560738  8667 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 14:54:06.497428  8631 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:06.594025  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:06.618158  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.571276  8715 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:06.568794  8504 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.626446  2678 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:54:06.630752  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:06.631039  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 328854us
I20260709 14:54:06.639052  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:06.640909  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.642462  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.575115  8472 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:06.648417  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146803us
I20260709 14:54:06.648862  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.577582  8506 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:06.587023  8375 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.648346  2607 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:06.658885  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:06.658890  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:06.674724  2753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.025s
I20260709 14:54:06.621482  8536 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:06.591678  8575 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
W20260709 14:54:06.676285  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 226464us
W20260709 14:54:06.687325  2753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.013s	sys 0.025s
I20260709 14:54:06.596616  8688 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:06.617952  8659 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.635213  8643 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 14:54:06.701191  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:06.672878  8708 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:06.596964  8658 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:06.652869  8504 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:06.742146  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99714.8us
W20260709 14:54:06.738047  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1051) took 2161 ms (client timeout 2041 ms). Trace:
W20260709 14:54:06.745510  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85030.9us
I20260709 14:54:06.750602  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:06.750490  2609 rpcz_store.cc:269] 0709 14:54:04.566748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.567013 (+   265us) service_pool.cc:224] Handling call
0709 14:54:06.728073 (+2161060us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:06.728286 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746634112}
W20260709 14:54:06.738049  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 303834us
I20260709 14:54:06.751647  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:06.735123  2605 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:06.739629  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 121126us
I20260709 14:54:06.754531  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:06.755743  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 201630us
W20260709 14:54:06.757915  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:06.758641  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:06.688416  8537 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:06.733263  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 159851us
W20260709 14:54:06.697541  8062 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.750s	user 0.007s	sys 0.006s
I20260709 14:54:06.696056  8463 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:06.665292  8652 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:06.837486  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86271.1us
I20260709 14:54:06.847434  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:06.848740  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:06.700512  8518 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:06.850212  2604 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:06.852916  2752 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 11: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:06.850445  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96056.2us
I20260709 14:54:06.852960  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:06.854458  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:06.703213  8374 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:06.737269  8741 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:06.876781  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:06.765564  8537 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:06.836596  8062 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:06.698705  8375 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:06.699874  8536 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:06.857270  8518 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:06.861379  8713 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:06.743953  8734 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.151s
I20260709 14:54:06.916491  2760 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:06.887782  8537 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:06.911242  8742 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.240s	user 0.001s	sys 0.001s
I20260709 14:54:06.897845  8741 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:06.927829  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:06.929162  2759 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:06.925082  2610 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.574s	user 0.014s	sys 0.001s
W20260709 14:54:06.926031  2684 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.002s
I20260709 14:54:06.932408  2610 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:06.932579  2684 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:06.933660  2612 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:06.933990  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.936340  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.937275  2684 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:06.951885  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:06.912163  8518 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:06.954124  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:06.915079  8643 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:06.911239  8372 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:06.849678  8472 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:06.917039  8578 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:06.974982  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:06.977483  2759 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:06.915674  8745 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:06.925256  8736 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:06.921715  8732 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:06.930996  8702 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:06.994326  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:06.972909  8713 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:06.975926  8712 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.200s	user 0.016s	sys 0.000s
I20260709 14:54:06.977169  8578 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:06.940627  8559 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 14:54:06.969357  8753 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:07.027063  8578 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:07.031140  2680 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:07.016358  8759 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:07.034296  8759 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.035523  2759 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:07.024389  8712 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:06.990414  8736 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:07.045804  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.072135  2757 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 19
I20260709 14:54:07.045924  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1446) took 2769 ms. Trace:
I20260709 14:54:07.073200  3545 rpcz_store.cc:276] 0709 14:54:04.276180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.276446 (+   266us) service_pool.cc:224] Handling call
0709 14:54:04.277014 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.583922 (+1306908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.045583 (+1461661us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002026d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.045830 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990871808}
W20260709 14:54:07.015908  8729 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.357s	user 0.002s	sys 0.000s
I20260709 14:54:07.050087  2758 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:07.051229  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.053047  2751 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:07.053478  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.055015  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.020067  8710 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:07.075579  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 215211us
I20260709 14:54:07.079558  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.080838  2755 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:07.056497  2759 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:54:07.059698  3547 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:07.091419  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.026182  8708 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:07.114104  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:07.039819  8745 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:07.046471  8733 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.190s	user 0.001s	sys 0.001s
W20260709 14:54:07.047699  8737 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 14:54:07.044019  8743 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 14:54:07.040932  8750 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:07.055156  8749 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.043424  8631 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.077034  8688 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.068177  8659 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.120877  8753 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:07.044724  8704 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 14:54:07.023945  8713 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.004300  8732 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.132773  8536 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:07.158793  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 408285us
W20260709 14:54:07.140586  2679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.006s	sys 0.003s
W20260709 14:54:07.172227  2679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.006s	sys 0.004s
W20260709 14:54:07.152454  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.005s
W20260709 14:54:07.174232  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.005s	sys 0.005s
W20260709 14:54:07.173935  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 319549us
W20260709 14:54:07.140277  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 280149us
W20260709 14:54:07.141988  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.000s	sys 0.008s
I20260709 14:54:07.190975  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:07.141369  8750 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:07.212502  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:07.119819  8708 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.160140  8752 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:07.149143  8658 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.153369  8753 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:07.152386  8463 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:07.214882  8703 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:07.238539  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1722) took 2658 ms. Trace:
I20260709 14:54:07.239176  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:07.135195  8745 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.237532  3546 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:07.254758  3543 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:07.191732  8757 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:07.129441  8741 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:07.242938  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 123297us
W20260709 14:54:07.191908  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.000s	sys 0.013s
I20260709 14:54:07.193132  3537 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:07.264945  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:07.256599  8591 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.192672  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.248766  2753 rpcz_store.cc:276] 0709 14:54:04.580544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.908778 (+1328234us) service_pool.cc:224] Handling call
0709 14:54:05.909438 (+   660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:05.965639 (+ 56201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:05.965694 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:06.019739 (+ 54045us) log.cc:844] Serialized 26 byte log entry
0709 14:54:06.693797 (+674058us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:06.693889 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:06.694018 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:07.238241 (+544223us) raft_consensus.cc:1685] finished
0709 14:54:07.238273 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:07.238464 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55838,"raft.queue_time_us":61373,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":12751,"thread_start_us":545639,"threads_started":1,"wal-append.queue_time_us":752921}
I20260709 14:54:07.266408  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1779) took 1407 ms. Trace:
I20260709 14:54:07.267920  2605 rpcz_store.cc:276] 0709 14:54:05.858952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.069425 (+210473us) service_pool.cc:224] Handling call
0709 14:54:06.729900 (+660475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.761887 (+ 31987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.266173 (+504286us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b5400135950. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:07.266342 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1517593472}
W20260709 14:54:07.248936  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.004s	sys 0.009s
I20260709 14:54:07.266387  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:07.269134  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.004s	sys 0.009s
W20260709 14:54:07.251394  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59202.7us
I20260709 14:54:07.216456  8750 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:07.258879  8374 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:07.259840  8763 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:07.270081  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:07.269556  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 698183us
W20260709 14:54:07.272011  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79443.5us
I20260709 14:54:07.275081  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.265753  8591 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.275070  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:07.281610  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.286024  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:07.286859  3536 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:07.244771  8753 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:07.287665  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 114871us
I20260709 14:54:07.288103  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.152101  8749 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:07.301837  8731 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.407s	user 0.001s	sys 0.002s
W20260709 14:54:07.302769  8760 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:54:07.303006  3543 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:07.284531  8559 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 14:54:07.306325  8565 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.050s	user 0.016s	sys 0.001s
I20260709 14:54:07.306531  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1652) took 1201 ms. Trace:
I20260709 14:54:07.307029  3540 rpcz_store.cc:276] 0709 14:54:06.105800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.305800 (+200000us) service_pool.cc:224] Handling call
0709 14:54:06.306281 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:06.704573 (+398292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:54:06.752749 (+ 48176us) log.cc:844] Serialized 14 byte log entry
0709 14:54:06.989990 (+237241us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:07.003520 (+ 13530us) log.cc:844] Serialized 52 byte log entry
0709 14:54:07.010528 (+  7008us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:07.010576 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.010667 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:07.306238 (+295571us) raft_consensus.cc:1685] finished
0709 14:54:07.306264 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:07.306461 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":202,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":25609,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":12194,"thread_start_us":509481,"threads_started":2,"wal-append.queue_time_us":299522}
I20260709 14:54:07.308351  2758 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:07.308624  8497 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:07.310025  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1639) took 1004 ms. Trace:
I20260709 14:54:07.310494  2604 rpcz_store.cc:276] 0709 14:54:06.305874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.693308 (+387434us) service_pool.cc:224] Handling call
0709 14:54:06.693723 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.853034 (+159311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.309761 (+456727us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.309959 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359295360}
I20260709 14:54:07.311827  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:07.312130  8643 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:07.312814  8643 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:07.313490  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:07.155876  8736 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.315649  8736 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:07.316092  8732 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.316880  8713 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.291742  8765 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.319201  8765 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.321985  8765 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.311949  2607 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 14:54:07.323801  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48854.5us
I20260709 14:54:07.325006  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:07.315099  8463 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.336782  3536 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:07.348407  2755 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
W20260709 14:54:07.295586  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.443678  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1285) took 1715 ms. Trace:
I20260709 14:54:07.443797  2757 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:07.444442  3547 rpcz_store.cc:276] 0709 14:54:05.728694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.730706 (+  2012us) service_pool.cc:224] Handling call
0709 14:54:07.443390 (+1712684us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004b64d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:07.443580 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3004988800}
W20260709 14:54:07.448625  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 123697us
I20260709 14:54:07.448729  3542 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:07.443825  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 168879us
W20260709 14:54:07.331032  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48179.9us
I20260709 14:54:07.450665  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.324728  8651 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:07.318672  8749 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.452335  2679 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:07.284495  8374 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.322041  8565 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.457396  8591 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.458312  8734 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:07.448297  8732 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:07.469861  8768 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 14:54:07.339156  8757 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:07.462965  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 106942us
I20260709 14:54:07.472098  8732 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.466169  8704 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:07.469741  8559 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:07.471827  8734 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:07.478191  8734 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.459761  8503 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 14:54:07.471027  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.0539e+06us
I20260709 14:54:07.452821  8752 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:07.492112  8752 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:07.492655  8752 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:54:07.452304  8715 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.800s	user 0.010s	sys 0.006s
W20260709 14:54:07.502043  2723 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.664s (SENT)
I20260709 14:54:07.502655  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.499197  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:07.454018  8319 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:07.509753  2754 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:07.510774  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.453248  8741 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:07.504690  8536 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.515214  8775 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:07.518193  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:07.525354  8745 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.526244  8770 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:07.508231  8658 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 14:54:07.559645  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92339.3us
I20260709 14:54:07.530421  8631 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.564881  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.501991  8412 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.527043  8754 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:07.507678  8715 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.546456  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1771) took 3373 ms. Trace:
I20260709 14:54:07.497088  8651 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:07.555569  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.529554  8374 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.529616  8565 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.566097  2757 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:07.562423  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:07.500347  8503 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 14:54:07.517128  8732 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.576676  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:07.577243  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 126635us
I20260709 14:54:07.558624  8741 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.577567  3547 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:07.577250  8732 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.569475  2606 rpcz_store.cc:276] 0709 14:54:04.173518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.173688 (+   170us) service_pool.cc:224] Handling call
0709 14:54:04.187321 (+ 13633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:05.316010 (+1128689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.546099 (+2230089us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540016e9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.546366 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548944256}
I20260709 14:54:07.571928  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.578684  8732 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.559847  8559 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:07.580137  8559 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:07.580729  8559 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:07.581418  8741 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.581984  8732 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.578702  8770 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:07.584277  8770 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.590159  2723 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.593439  2755 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:07.557576  8536 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:07.597163  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.604744  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:07.592135  8537 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.005s
I20260709 14:54:07.606554  2677 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:07.607722  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 110120us
I20260709 14:54:07.595126  8775 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:07.614616  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:07.615760  2606 raft_consensus.cc:606] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.616349  2606 raft_consensus.cc:614] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 14:54:07.627331  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:07.628875  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.569820  8412 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.569509  8631 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.639048  2726 leader_election.cc:400] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:07.611591  8565 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:07.642580  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1066) took 3265 ms. Trace:
I20260709 14:54:07.614337  8775 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:07.663199  2760 rpcz_store.cc:276] 0709 14:54:04.377490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.742726 (+1365236us) service_pool.cc:224] Handling call
0709 14:54:05.756899 (+ 14173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.919708 (+1162809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.642260 (+722552us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.642497 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666103680}
W20260709 14:54:07.645186  8701 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.502s	user 0.006s	sys 0.020s
W20260709 14:54:07.646059  8518 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.659s	user 0.006s	sys 0.008s
I20260709 14:54:07.716591  2683 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:07.614992  8643 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.621308  8537 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:07.719717  8578 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.002s
I20260709 14:54:07.732816  2604 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:07.733832  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:07.741122  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:07.712339  8775 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:07.739558  8565 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:07.747300  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126559us
I20260709 14:54:07.743409  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:07.749524  2605 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:07.729233  8772 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.754473  3546 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:07.745010  8790 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 14:54:07.756999  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:07.758235  8782 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 14:54:07.715319  8463 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 14:54:07.774363  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1649) took 1820 ms. Trace:
I20260709 14:54:07.777150  3541 rpcz_store.cc:276] 0709 14:54:05.954328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.304574 (+350246us) service_pool.cc:224] Handling call
0709 14:54:06.304932 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:06.620870 (+315938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.773979 (+1153109us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54003d86d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.774267 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676078464}
I20260709 14:54:07.728148  8518 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:07.780903  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.740783  8578 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.790660  2684 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:07.774919  8745 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:54:07.777930  8536 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.762307  8565 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.798621  3545 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
W20260709 14:54:07.807539  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 57944us
I20260709 14:54:07.808535  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.777665  8772 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.811717  8715 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.812482  8772 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.813236  8772 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:07.813882  8772 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.720077  8651 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:07.819240  8783 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 14:54:07.800848  8506 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.096s	user 0.008s	sys 0.008s
I20260709 14:54:07.780385  8750 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.825151  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.826658  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:07.837296  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 229873us
I20260709 14:54:07.838652  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.840982  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.844878  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.846150  8797 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:07.847632  8797 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 14:54:07.848948  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:07.828289  8536 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:07.854007  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:07.855782  8764 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
I20260709 14:54:07.855330  8536 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.833236  8667 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:07.833612  8745 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:07.857132  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.828943  8506 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:07.858170  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1553) took 1425 ms. Trace:
I20260709 14:54:07.832527  8754 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:07.862591  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:07.862274  2680 rpcz_store.cc:276] 0709 14:54:06.432325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:07.032209 (+599884us) service_pool.cc:224] Handling call
0709 14:54:07.032667 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:07.033127 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:07.033166 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:07.033835 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 14:54:07.256400 (+222565us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:07.256506 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.256649 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:07.857823 (+601174us) raft_consensus.cc:1685] finished
0709 14:54:07.857851 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:07.858063 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1437,"raft.queue_time_us":101003,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":4392,"thread_start_us":216160,"threads_started":1,"wal-append.queue_time_us":524171}
W20260709 14:54:07.860289  8736 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.001s
I20260709 14:54:07.872025  2685 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:07.855449  8784 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:07.877151  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:07.854324  8797 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:07.861181  8374 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:07.869184  8631 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.879415  8745 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:07.887497  8748 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:07.890960  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:07.892551  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1793) took 1317 ms. Trace:
I20260709 14:54:07.872476  8643 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.896940  2680 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:07.896690  2607 rpcz_store.cc:276] 0709 14:54:06.575479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.724796 (+149317us) service_pool.cc:224] Handling call
0709 14:54:07.311578 (+586782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:07.323326 (+ 11748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.892281 (+568955us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54001d9050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:07.892464 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286262016}
I20260709 14:54:07.872267  8537 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.873338  8763 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:07.885915  8754 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:07.885952  8757 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:07.885100  8557 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:07.909440  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64687.5us
I20260709 14:54:07.911271  2685 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:07.883543  8667 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 14:54:07.914037  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.916633  2755 raft_consensus.cc:2376] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16. Re-sending same reply.
I20260709 14:54:07.914954  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.917876  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.918562  2751 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 14:54:07.919358  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:07.920931  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:07.914604  8631 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 14:54:07.924106  2682 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1563) took 3679 ms (client timeout 1982 ms). Trace:
W20260709 14:54:07.924623  2682 rpcz_store.cc:269] 0709 14:54:04.244358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:04.558856 (+314498us) service_pool.cc:224] Handling call
0709 14:54:07.923816 (+3364960us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:07.923976 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638452096}
I20260709 14:54:07.917295  8745 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.927618  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.927491  8745 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.903573  8536 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.931294  8745 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:07.931690  8794 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 14:54:07.922508  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:07.932919  8792 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 14:54:07.919493  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.933969  8565 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.904160  8506 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:07.935112  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:07.932859  2760 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:07.936268  8753 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.005s
I20260709 14:54:07.956967  2755 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:07.936913  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.938210  2757 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:07.886370  8374 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.892649  8784 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.950233  8763 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.939355  8518 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.917258  8765 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:07.915732  8557 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.919374  8757 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 14:54:07.911445  8769 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:07.938247  8754 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:07.941015  8578 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.964742  8363 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:07.971807  8769 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:07.979907  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1549) took 2890 ms (client timeout 2035 ms). Trace:
I20260709 14:54:07.980798  8784 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.969439  8557 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.981730  8592 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:07.982124  8769 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.947981  8559 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.969465  8753 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:07.985285  8559 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:07.985658  8753 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.969298  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:07.958397  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:07.950716  8772 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.987847  8754 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:07.964193  8504 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.964524  8374 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:07.986742  2682 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 14:54:07.959117  8506 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:07.990799  8374 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:07.949853  8752 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:07.992767  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:07.992928  8506 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:07.993474  8752 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:07.946723  8717 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:07.995685  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1101) took 1564 ms. Trace:
I20260709 14:54:07.996167  2684 rpcz_store.cc:276] 0709 14:54:06.431397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.956131 (+524734us) service_pool.cc:224] Handling call
0709 14:54:06.991604 (+ 35473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:07.791413 (+799809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:07.995334 (+203921us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:07.995570 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1837153664}
W20260709 14:54:07.960672  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43721.6us
I20260709 14:54:07.993523  2755 raft_consensus.cc:2376] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18. Re-sending same reply.
I20260709 14:54:07.997886  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:07.980495  2686 rpcz_store.cc:269] 0709 14:54:05.089571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:05.242968 (+153397us) service_pool.cc:224] Handling call
0709 14:54:07.979589 (+2736621us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:07.979781 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344611072}
I20260709 14:54:08.009742  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:08.011423  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.012936  2760 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:08.013222  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.014420  2755 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.016052  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.016991  2755 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:08.018153  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.019343  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.020296  3536 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:08.021626  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.022387  3538 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:08.028502  2758 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:54:08.018169  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.030642  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.031576  3541 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:08.019405  2755 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:54:08.044392  2760 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:08.046773  8772 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:08.047590  8772 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.050758  8772 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:08.055214  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.013738  8752 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:08.056082  8752 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:08.056085  2606 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 11.
I20260709 14:54:08.056733  8752 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:08.059373  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.062531  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.059538  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.064453  2606 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:08.063913  2603 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:08.066172  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.065644  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.067817  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:08.066996  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
W20260709 14:54:08.069725  8738 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.161s	user 0.002s	sys 0.012s
I20260709 14:54:08.077610  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.077955  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.078631  2607 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:08.078761  3543 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:08.081522  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:08.084293  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.085497  3537 raft_consensus.cc:2368] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:08.086889  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.087136  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:08.088609  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98939.9us
I20260709 14:54:08.090006  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:08.092279  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.093148  3541 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:08.091554  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:08.094348  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:08.083954  8705 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.095791  8705 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.097177  8667 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:08.097892  8667 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:08.098354  8667 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:08.100051  8748 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:08.100771  8748 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:08.101188  8748 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:08.103421  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.111618  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:08.112648  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:08.114609  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:08.118953  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a57a97528f31400dbc0900e76292b021"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:08.120311  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:08.120231  2606 raft_consensus.cc:606] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:08.122344  2606 raft_consensus.cc:614] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:08.117265  8363 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:08.118665  8769 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:08.148067  3543 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:08.118402  8745 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:08.151938  3538 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:08.089691  8775 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.135378  8754 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:08.121560  8651 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:08.157006  8803 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 14:54:08.158277  8739 raft_consensus.cc:993] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: : Instructing follower 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:54:08.139195  8557 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:08.164700  3538 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 19. Ops: []
I20260709 14:54:08.167349  3543 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [16.9-19.10]
I20260709 14:54:08.168730  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.181005  8775 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.183722  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:08.184577  2606 raft_consensus.cc:606] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:08.184983  2606 raft_consensus.cc:614] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:08.162519  8651 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:08.187278  8651 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:08.189127  8778 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:08.189702  8778 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:08.123891  8374 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 14:54:08.153273  8745 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:08.166270  8807 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 14:54:08.162802  8754 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:08.164017  8557 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:08.207382  8769 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:08.194252  8739 raft_consensus.cc:1081] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Signalling peer 4f1347e243fd46a5bb7cee5b2e23b665 to start an election
I20260709 14:54:08.223791  3538 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:08.193737  8803 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:08.196261  8773 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.730s	user 0.000s	sys 0.003s
I20260709 14:54:08.196414  8745 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:08.234091  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 119580us
I20260709 14:54:08.234751  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.235440  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.239213  3538 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:08.241206  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.241971  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.245011  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.224936  8363 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:08.208993  8754 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:08.220706  8557 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:08.208745  8592 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:08.249948  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.248070   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:54:08.253670  3543 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:54:08.260759  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.232934  8803 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:54:08.190727  8778 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.287788  3538 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 16. Current term is 17. Ops: []
W20260709 14:54:08.266175  8781 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.795s	user 0.022s	sys 0.028s
I20260709 14:54:08.254693  8772 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:08.305536  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61820.2us
I20260709 14:54:08.308574  3538 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:08.305616  8769 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:08.311933  8815 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.098s	user 0.000s	sys 0.008s
I20260709 14:54:08.264477  8752 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:08.315806  8788 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.740s	user 0.003s	sys 0.000s
I20260709 14:54:08.317229  8752 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:08.306007  8772 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.335477  8778 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:08.337842  8772 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.288618  8592 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.339715  3547 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:08.348559  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.340579  8591 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.352489  8748 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.357966  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.359063  2603 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:08.365785  8667 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.367873  8772 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:08.375741  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.376960  2603 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:08.379252  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:08.389443  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.390820  2760 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.350646  8591 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.305246  8363 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.373219  8748 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.384310  8816 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:08.374382  8772 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.372854  8651 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:08.418323  3538 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:08.420133  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.424309  8557 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:08.426478  3543 raft_consensus.cc:2368] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:08.432854  8557 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:08.447144  8592 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:08.448498  8772 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.411708  8717 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:08.453163  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.453325  8717 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:08.454480  3536 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:08.454922  8754 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.458287  8667 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:08.460587  8651 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:08.456439  8717 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:08.455929  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.469412  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.421823  8816 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:08.470604  8745 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.498481  8754 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:08.525151  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1733) took 1531 ms. Trace:
I20260709 14:54:08.537624  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.545205  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.526124  2754 rpcz_store.cc:276] 0709 14:54:06.993636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.993899 (+   263us) service_pool.cc:224] Handling call
0709 14:54:08.524739 (+1530840us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400423bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.525010 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182087552}
I20260709 14:54:08.547119  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.548484  2759 raft_consensus.cc:2376] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:08.549026  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.550315  2754 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.551662  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.561619  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.565102  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.557600  8772 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:08.575106  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.579957  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.583837  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.587121  2759 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.589181  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.589593  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.590797  2754 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.596318  2759 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.597373  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.598483  2754 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.562307  8803 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.566021  8816 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.582494  8772 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.626040  8745 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:08.628731  8803 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:08.629213  3540 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:08.632781  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:08.639640  8772 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.643078  8769 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:08.642562  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.652493  8710 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:08.658500  8710 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:08.664319  8785 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:08.664958  8785 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:08.665906  8785 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.634284  8578 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.635663  8708 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:08.683835  8759 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:08.685472  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1100) took 2266 ms (client timeout 2352 ms). Trace:
W20260709 14:54:08.686064  2683 rpcz_store.cc:269] 0709 14:54:06.419421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.429860 (+ 10439us) service_pool.cc:224] Handling call
0709 14:54:08.685211 (+2255351us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540024efd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.685387 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2785785728}
I20260709 14:54:08.690953  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.688207  8785 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:08.690784  8702 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:08.677335  8578 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:08.680621  8708 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:08.697866  8536 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:08.704547  3540 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:08.702298  8703 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:08.699054  8702 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:08.711689  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.712656  2756 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.713835  8375 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:08.706207  8703 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:08.714417  8578 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:08.716069  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1293) took 1543 ms. Trace:
I20260709 14:54:08.716536  3542 rpcz_store.cc:276] 0709 14:54:07.172802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:07.172927 (+   125us) service_pool.cc:224] Handling call
0709 14:54:07.446363 (+273436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:07.449490 (+  3127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:08.715843 (+1266353us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54004b64d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.716001 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537628032}
I20260709 14:54:08.703593  8536 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:08.719038  8772 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 14:54:08.719161  2760 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 14:54:08.714092  2756 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:08.725090  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.729009  8708 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.730762  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.719643  8702 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:08.733454  8591 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:08.720917  8536 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.735509  2576 leader_election.cc:400] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:08.735922  2576 leader_election.cc:403] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 14:54:08.735813  8536 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:08.741942  8750 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:08.751466  2757 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:08.752751  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:08.753274  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.754509  2757 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:08.757566  8749 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:08.756085  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:08.778807  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:08.780344  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44965.2us
I20260709 14:54:08.781629  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:08.767957  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.771814  2756 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
W20260709 14:54:08.772332  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.786804  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:08.774284  8777 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.298s	user 0.002s	sys 0.002s
W20260709 14:54:08.774641  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.759805  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:08.718751  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:08.795771  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:08.774140  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.790055  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.792702  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:08.788404  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.807263  2756 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 18
I20260709 14:54:08.797240  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:08.800586  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81880.1us
I20260709 14:54:08.817687  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:08.818333  2756 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:08.803499  8375 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:54:08.793438  8702 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:08.829097  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:08.831585  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:08.820700  8832 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:08.835145  3545 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:08.806058  8772 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:08.803903  8752 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:08.821481  8708 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:08.821045  8652 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:08.799592  8631 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:08.806473  8749 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:08.831094  8363 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 14:54:08.859644  8776 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.863723  8752 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:08.841095  8183 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:08.873813  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.874734  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.875183  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.936827  2606 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:08.875700  8652 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:08.876456  2607 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:54:08.877972  8412 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:08.877871  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:08.878327  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:08.878983  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1557) took 2445 ms (client timeout 2122 ms). Trace:
I20260709 14:54:08.879302  2605 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:54:08.879945  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.975378  2686 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:08.881306  2683 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:08.822984  8578 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 14:54:08.885851  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:08.887326  8631 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:08.887966  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:08.888875  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:08.889003  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:08.983315  2604 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:08.891688  2609 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:54:08.895170  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:08.986294  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:54:08.899324  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:08.902987  2756 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 11. Current term is 12. Ops: []
I20260709 14:54:08.903270  3537 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
W20260709 14:54:08.906525  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1581) took 2506 ms (client timeout 1664 ms). Trace:
W20260709 14:54:08.989630  2677 rpcz_store.cc:269] 0709 14:54:06.399786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.407033 (+  7247us) service_pool.cc:224] Handling call
0709 14:54:08.906166 (+2499133us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.906389 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2741113728}
W20260709 14:54:08.910568  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1298) took 1673 ms (client timeout 1923 ms). Trace:
W20260709 14:54:08.991312  3546 rpcz_store.cc:269] 0709 14:54:07.237185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:07.269692 (+ 32507us) service_pool.cc:224] Handling call
0709 14:54:08.902642 (+1632950us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540008ff54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.910463 (+  7821us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094596352}
I20260709 14:54:08.912055  2681 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:54:08.916828  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:08.925275  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1570) took 1072 ms. Trace:
I20260709 14:54:08.979920  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:08.993798  2682 rpcz_store.cc:276] 0709 14:54:07.852985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:07.934235 (+ 81250us) service_pool.cc:224] Handling call
0709 14:54:07.934782 (+   547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:07.991262 (+ 56480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:08.905474 (+914212us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:08.925135 (+ 19661us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118704512}
I20260709 14:54:08.934122  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:08.930989  8734 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.971035  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:08.974913  2685 rpcz_store.cc:269] 0709 14:54:06.433556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.674434 (+240878us) service_pool.cc:224] Handling call
0709 14:54:08.878742 (+2204308us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002b3254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:08.878893 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2689073536}
I20260709 14:54:08.978457  2684 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:08.979514  2724 connection.cc:256] RPC call timeout handler was delayed by -1.0021301290000793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 14:54:08.981264  2760 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:08.995344  2679 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:08.942727  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:08.997629  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 98412.8us
I20260709 14:54:08.939062  8750 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:09.003922  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.003605  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.004345  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:09.023303  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.024166  2607 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:09.024340  2685 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:09.004931  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.028564  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:08.939289  8363 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:09.015031  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:09.033916  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.035315  2684 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:09.038156  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.055912  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.020232  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.061553  2686 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:09.056748  2682 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:09.065580  8768 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:09.066354  8768 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:09.067109  8578 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:09.067538  8631 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:09.068251  8578 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 14:54:09.068804  8578 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:09.068115  8834 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:09.070411  8504 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:09.004112  8412 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:08.988783  8652 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:09.076323  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1099) took 1136 ms. Trace:
I20260709 14:54:09.065580  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.077252  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.027037  8803 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 14:54:08.941380  8776 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:09.079340  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 212407us
I20260709 14:54:09.082428  2760 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:09.116915  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.126531  2679 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:09.127307  2682 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:09.080106  2680 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:09.132424  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.132718  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.122969  2759 rpcz_store.cc:276] 0709 14:54:07.940046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.597488 (+657442us) service_pool.cc:224] Handling call
0709 14:54:08.597913 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:08.759630 (+161717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:08.759667 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:08.760613 (+   946us) log.cc:844] Serialized 52 byte log entry
0709 14:54:08.829171 (+ 68558us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:08.844044 (+ 14873us) log.cc:844] Serialized 14 byte log entry
0709 14:54:08.847006 (+  2962us) log.cc:844] Serialized 14 byte log entry
0709 14:54:08.847251 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:08.847341 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:09.075992 (+228651us) raft_consensus.cc:1685] finished
0709 14:54:09.076018 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:09.076236 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162504,"prepare.queue_time_us":39447,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":72,"raft.queue_time_us":168617,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":38122,"spinlock_wait_cycles":108534656,"thread_start_us":61631,"threads_started":1,"wal-append.queue_time_us":167075}
I20260709 14:54:08.996160  8183 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:09.139093  2677 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:09.144960  2607 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:09.055133  8363 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:09.150015  2723 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.631s (SENT)
I20260709 14:54:09.155333  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:09.033919  8832 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:09.158640  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 341017us
I20260709 14:54:09.227834  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:09.083648  7977 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:09.229079  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:08.978857  8820 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.195s
W20260709 14:54:09.231293  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49291.2us
I20260709 14:54:09.304068  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.306097  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:09.307358  2607 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:09.315189  2685 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
W20260709 14:54:09.315340  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 114048us
I20260709 14:54:09.320375  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:09.022351  8734 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:09.069872  8578 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:09.083290  8412 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:09.323310  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.051s	sys 0.054s
I20260709 14:54:09.058849  8750 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:09.351254  2684 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:54:09.351612  8759 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 14:54:09.352872  8750 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:09.149822  8713 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:09.354187  8772 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.015s
I20260709 14:54:09.234581  7977 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:09.158473  8754 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:09.232496  8504 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:09.341854  8768 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:09.345722  8536 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.004s
I20260709 14:54:09.337370  8803 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:09.349079  8412 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:09.227412  8745 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 14:54:09.353863  8713 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:09.340786  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.052s	sys 0.055s
W20260709 14:54:09.375209  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 578061us
I20260709 14:54:09.302033  8767 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
W20260709 14:54:09.359490  8785 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.659s	user 0.018s	sys 0.000s
I20260709 14:54:09.363127  7977 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:09.338083  8820 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:09.373494  2606 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 14:54:09.388549  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 384292us
I20260709 14:54:09.376633  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:09.391058  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:09.373540  8578 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 14:54:09.380256  2755 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 14:54:09.396950  2755 pending_rounds.cc:85] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 7
I20260709 14:54:09.397512  2755 pending_rounds.cc:107] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 14:54:09.397845  2755 raft_consensus.cc:2889] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:09.366521  8754 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:09.375145  8772 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:09.084172  8710 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:09.374514  8703 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.659s	user 0.015s	sys 0.002s
I20260709 14:54:09.394696  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1097) took 3007 ms. Trace:
W20260709 14:54:09.391948  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 241995us
I20260709 14:54:09.404070  2681 rpcz_store.cc:276] 0709 14:54:06.375984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:06.376628 (+   644us) service_pool.cc:224] Handling call
0709 14:54:08.907274 (+2530646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.001691 (+ 94417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:09.383227 (+381536us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b5400376c50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:09.383484 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5704128896}
I20260709 14:54:09.396080  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:09.391923  8769 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.748s	user 0.018s	sys 0.000s
I20260709 14:54:09.361964  8708 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 14:54:09.364785  8845 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 14:54:09.399881  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.405867  2606 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 14:54:09.328142  2607 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:09.388235  8820 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:09.419257  2612 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:09.369832  8504 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:09.420221  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 477623us
I20260709 14:54:09.402148  8754 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:09.417366  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:09.425559  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:09.428380  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.428023  2681 raft_consensus.cc:2376] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:09.434203  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.420881  8769 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:09.372367  8363 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:09.422370  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.442539  2606 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 14:54:09.445940  3536 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:09.420755  8842 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:09.377744  8183 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:09.444149  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1749) took 1132 ms. Trace:
I20260709 14:54:09.453394  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.453680  2751 rpcz_store.cc:276] 0709 14:54:08.311636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.442383 (+1130747us) service_pool.cc:224] Handling call
0709 14:54:09.443547 (+  1164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.443799 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:09.443826 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:09.443872 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:09.443917 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:09.443975 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:09.444087 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:09.395884  8832 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:09.376307  8652 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:09.411139  8703 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:09.486735  2607 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 14:54:09.492501  2607 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:09.461372  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 137996us
W20260709 14:54:09.472733  8839 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 14:54:09.475299  8759 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:09.478554  8734 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:09.432252  8749 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:09.489158  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:09.437361  8841 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:09.506985  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.451542  8776 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:09.505152  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 474800us
I20260709 14:54:09.467828  8834 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:09.495573  8363 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:09.512687  2680 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:09.509943  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:09.504514  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:09.520851  2607 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 14:54:09.531481  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.532836  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.533051  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.537678  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:09.472810  8845 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 14:54:09.543291  2685 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 14:54:09.515491  8472 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:09.512156  8841 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:09.551847  2607 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 14:54:09.516032  8834 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:09.591655  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 132102us
W20260709 14:54:09.540802  8752 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.601s	user 0.013s	sys 0.000s
W20260709 14:54:09.595571  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47380.2us
I20260709 14:54:09.601356  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:09.587848  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:09.597690  2679 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:09.597390  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:09.603127  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 197655us
I20260709 14:54:09.603631  2751 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:54:09.609411  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 161731us
W20260709 14:54:09.610014  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 182619us
I20260709 14:54:09.610725  2607 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:09.544498  8741 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:09.611485  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:09.613754  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:09.614948  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:09.584329  8472 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:09.610050  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:09.590627  8776 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:09.617445  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1738) took 1425 ms. Trace:
I20260709 14:54:09.593434  8834 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:09.540139  8750 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 14:54:09.617293   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:54:09.610663  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:09.618284  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:09.621347  2723 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.076s (SENT)
I20260709 14:54:09.639273  2754 rpcz_store.cc:276] 0709 14:54:08.191680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.599415 (+407735us) service_pool.cc:224] Handling call
0709 14:54:08.608110 (+  8695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:08.761570 (+153460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:08.761623 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:08.762337 (+   714us) log.cc:844] Serialized 26 byte log entry
0709 14:54:09.134728 (+372391us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:09.134782 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:09.134893 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:09.617155 (+482262us) raft_consensus.cc:1685] finished
0709 14:54:09.617193 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:09.617367 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159579,"raft.queue_time_us":154263,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":145,"thread_start_us":371895,"threads_started":1,"wal-append.queue_time_us":592082}
I20260709 14:54:09.619630  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:09.617576  8702 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.778s	user 0.016s	sys 0.000s
I20260709 14:54:09.544006  8668 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 14:54:09.608549  8752 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:09.657234  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:09.586647  8841 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:09.663115  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:09.639062  8776 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:09.680158  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:09.706790  2754 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:09.707545  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 88027.7us
I20260709 14:54:09.680259  2679 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 14:54:09.626693  8472 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:09.616441  8741 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:09.618916  8504 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:09.707718  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86415.7us
W20260709 14:54:09.680516  8850 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:54:09.708451  2679 pending_rounds.cc:85] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 8
I20260709 14:54:09.754796  2679 pending_rounds.cc:107] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 14:54:09.755260  2679 raft_consensus.cc:2889] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:09.642714  8848 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:54:09.654606  8789 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.871s	user 0.000s	sys 0.005s
W20260709 14:54:09.731604  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121051us
W20260709 14:54:09.742406  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 126893us
I20260709 14:54:09.668390  8713 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:09.718986  8741 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:09.627053  8846 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:09.749723  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:09.859364  3543 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:09.681447  8772 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:54:09.868907  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 281144us
I20260709 14:54:09.685616  8734 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:09.701804  8710 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:09.711829  8750 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:54:09.712131  8776 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:09.823603  8823 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.281s	user 0.012s	sys 0.019s
I20260709 14:54:09.796360  8536 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:09.808077  8717 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:09.808898  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:09.748520  8803 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:09.716339  8745 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:09.716624  8704 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 14:54:09.816375  2678 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:54:09.819721  8852 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 14:54:09.820990  2753 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:09.823572  2686 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:09.627090  8853 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.181s
I20260709 14:54:09.832414  8775 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:09.803419  8667 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:09.836911  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:09.684808  8763 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 14:54:09.742419  8702 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:09.700423  8849 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.201s
W20260709 14:54:09.996721  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 159583us
I20260709 14:54:09.936585  8710 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:09.856446  8752 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:09.910877  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:10.009135  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 172380us
I20260709 14:54:09.866586  8772 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:09.948850  2686 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:09.888497  8713 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:09.884093  8864 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:54:09.937554  8703 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.012177  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.029567  3543 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:09.906611  2679 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:09.907604  8848 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:10.016053  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:09.992276  8702 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:10.027781  2753 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:09.933228  8536 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.039717  2753 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 14:54:09.996413  8853 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:09.943590  8717 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.015617  8752 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:10.052289  2753 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:09.981451  8775 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:09.948537  8803 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.022177  8772 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:10.073933  8803 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.039230  8849 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:10.054611  2754 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:54:10.062788  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 226141us
I20260709 14:54:09.888700  8846 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.023566  8745 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.017861  8710 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.045753  8868 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:09.961146  8866 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 14:54:10.039556  8702 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:10.084196  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.085027  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:10.086134  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.021024  8668 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 14:54:10.046872  8853 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:09.995873  8667 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.032940  8713 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.058882  8652 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.067827  8752 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:10.056849  8536 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.067509  8705 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.126071  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:10.076922  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:10.098181  8856 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 14:54:10.101468  8853 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.104148  8710 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:10.104487  8846 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:10.105576  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1106) took 1199 ms. Trace:
W20260709 14:54:10.120672  8823 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 14:54:10.105798  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.111227  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.148260  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.148934  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.077147  8703 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:10.078271  8412 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.126201  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.150627  2686 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 14:54:10.138113  8842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.000s	sys 0.019s
I20260709 14:54:10.140482  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:10.141378  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:10.143009  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 66192.9us
I20260709 14:54:10.155165  2609 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:10.155691  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:10.157986  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.149030  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:54:10.091120  8375 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.254s	user 0.008s	sys 0.008s
I20260709 14:54:10.105883  8868 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:10.179486  8868 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.180416  8710 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.181036  8710 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.182065  8841 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.153891  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.088953  8849 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.087327  8745 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.146701  2682 rpcz_store.cc:276] 0709 14:54:08.906132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.139024 (+232892us) service_pool.cc:224] Handling call
0709 14:54:09.167316 (+ 28292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:09.167915 (+   599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:09.167996 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:09.227111 (+ 59115us) log.cc:844] Serialized 26 byte log entry
0709 14:54:09.594329 (+367218us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:09.594441 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:09.594575 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:10.105301 (+510726us) raft_consensus.cc:1685] finished
0709 14:54:10.105326 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:10.105492 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62368,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":75925,"thread_start_us":231561,"threads_started":1,"wal-append.queue_time_us":651810}
W20260709 14:54:10.175909  8869 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 14:54:10.179999  2651 connection.cc:256] RPC call timeout handler was delayed by -2.0213020989999677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 14:54:10.154089  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.043556  8848 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.193600  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:10.197775  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.198640  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.146324  8705 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:54:10.240793  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.150619  8703 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:10.197027  8536 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:10.197185  8375 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 14:54:10.166345  8412 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:10.156885  8842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.001s	sys 0.019s
W20260709 14:54:10.204706  2651 connection.cc:256] RPC call timeout handler was delayed by -1.7549020989999917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260709 14:54:10.205689  2686 raft_consensus.cc:2376] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15. Re-sending same reply.
I20260709 14:54:10.209335  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.200539  8871 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:10.243152  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.225484  2609 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:10.126806  8652 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.142889  8803 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 14:54:10.247514  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.284371  3543 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:10.249994  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:10.166754  8860 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
I20260709 14:54:10.269004  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.251354  8703 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:10.271437  2682 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 14:54:10.254439  8823 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 14:54:10.316365  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1097) took 2400 ms. Trace:
I20260709 14:54:10.259557  8536 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:10.319350  8536 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:10.289032  8631 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:10.323633  8631 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.328504  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.329695  2605 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:10.331995  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.332993  2610 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:10.341185  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.344094  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1119) took 1206 ms. Trace:
I20260709 14:54:10.344910  2684 rpcz_store.cc:276] 0709 14:54:09.137323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.232465 (+ 95142us) service_pool.cc:224] Handling call
0709 14:54:09.232907 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.351894 (+118987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.343728 (+991834us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:10.344000 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271111040}
I20260709 14:54:10.348532  2756 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 14:54:10.349754  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1759) took 1031 ms. Trace:
I20260709 14:54:10.258570  8886 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.274112  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:10.275797  2651 connection.cc:256] RPC call timeout handler was delayed by -1.223902098999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27480000000000004s for a response
I20260709 14:54:10.276888  8412 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:10.204998  8832 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:10.203500  8565 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 14:54:10.294539  2686 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:10.386236  2686 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 14:54:10.386466  8631 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:10.386683  8881 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 14:54:10.308991  2682 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:10.387864  2686 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:10.388367  2686 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:10.388199  2682 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.389312  2682 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:10.389169  2686 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.391098  8565 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:10.392247  8565 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.393553  8565 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:54:10.395576  2608 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:10.390584  2686 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:10.399879  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:10.400532  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:10.403638  8750 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:10.404574  8750 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:10.405582  8750 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.411723  8667 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.412237  8652 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:10.412421  8750 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:10.271148  8842 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:10.414145  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.353229  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1758) took 1035 ms. Trace:
I20260709 14:54:10.414496  8842 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.386238  8820 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.415041  2760 rpcz_store.cc:276] 0709 14:54:09.317528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.318308 (+   780us) service_pool.cc:224] Handling call
0709 14:54:09.323713 (+  5405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:09.324128 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:09.324172 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:09.332237 (+  8065us) log.cc:844] Serialized 26 byte log entry
0709 14:54:09.545776 (+213539us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:09.545824 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:09.545912 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:10.352947 (+807035us) raft_consensus.cc:1685] finished
0709 14:54:10.352972 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:10.353161 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5397,"spinlock_wait_cycles":24320,"thread_start_us":207654,"threads_started":1,"wal-append.queue_time_us":377769}
I20260709 14:54:10.416405  8820 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:10.416831  8841 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.417873  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.419850  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.423046  8710 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:10.425022  8745 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:10.354676  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.294469  8873 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.426278  3543 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:10.426203  8873 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.428840  8873 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.430213  8713 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.431212  8717 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.433820  8886 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:10.350190  2756 rpcz_store.cc:276] 0709 14:54:09.317841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.318752 (+   911us) service_pool.cc:224] Handling call
0709 14:54:10.347845 (+1029093us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:10.348147 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.349315 (+  1168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.349666 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3528064}
I20260709 14:54:10.362731  2610 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:54:10.437369  2686 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 14:54:10.437952  2686 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 9
I20260709 14:54:10.438244  2686 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 17.10
W20260709 14:54:10.368342  2651 connection.cc:256] RPC call timeout handler was delayed by -1.1065151369999739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 14:54:10.451850  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.185798  8834 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.342684  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1573) took 1112 ms. Trace:
I20260709 14:54:10.373144  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:10.378221  3541 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:10.316988  2758 rpcz_store.cc:276] 0709 14:54:07.916208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.073303 (+157095us) service_pool.cc:224] Handling call
0709 14:54:08.073780 (+   477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:08.076804 (+  3024us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 14:54:08.093653 (+ 16849us) log.cc:844] Serialized 14 byte log entry
0709 14:54:08.391180 (+297527us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:08.406700 (+ 15520us) log.cc:844] Serialized 52 byte log entry
0709 14:54:08.406989 (+   289us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:08.407332 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 14:54:08.407751 (+   419us) log.cc:844] Serialized 14 byte log entry
0709 14:54:08.419520 (+ 11769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:08.419610 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:08.739413 (+319803us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18003109c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:08.748345 (+  8932us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18003109c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:54:09.705855 (+957510us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:10.315989 (+610134us) raft_consensus.cc:1685] finished
0709 14:54:10.316025 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:10.316245 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336260,"prepare.queue_time_us":348822,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":142,"raft.queue_time_us":15409,"raft.run_cpu_time_us":705,"raft.run_wall_time_us":37683,"spinlock_wait_cycles":38752000,"thread_start_us":283701,"threads_started":1,"wal-append.queue_time_us":377131}
I20260709 14:54:10.383533  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.383193  8708 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 14:54:10.269634  8363 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.736s	user 0.015s	sys 0.000s
I20260709 14:54:10.413703  8849 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:10.248538  8578 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.006s
I20260709 14:54:10.457826  2683 rpcz_store.cc:276] 0709 14:54:09.230327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.230499 (+   172us) service_pool.cc:224] Handling call
0709 14:54:10.340916 (+1110417us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:10.342554 (+  1638us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10578816}
I20260709 14:54:10.364099  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.357653  2679 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 14:54:10.413632  7977 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.017s	user 0.013s	sys 0.002s
W20260709 14:54:10.444306  8741 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.001s
I20260709 14:54:10.444979  8886 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:10.291816  8871 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.454128  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.455394  2686 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:10.456045  2682 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 14:54:10.505640  2682 pending_rounds.cc:85] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 10
I20260709 14:54:10.505973  2682 pending_rounds.cc:107] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 14:54:10.506232  2682 raft_consensus.cc:2889] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:10.457625  8873 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:10.470861  2604 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:54:10.475803  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.476081  2679 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:10.440861  3547 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:10.523146  2679 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:10.524461  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:10.508117  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54139.4us
I20260709 14:54:10.457800  8772 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.509003  8882 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:10.511984  8752 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.540382  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.500104  2605 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:10.537674  8882 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:10.542057  8882 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:10.543203  8871 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:10.535413  8849 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:10.468950  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.545454  3541 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:10.546546  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.547596  3541 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:10.526022  3547 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 14:54:10.502802  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 129832us
I20260709 14:54:10.548781  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.549129  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:10.534360  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1809) took 1599 ms (client timeout 1699 ms). Trace:
W20260709 14:54:10.551221  3540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.001s	sys 0.012s
I20260709 14:54:10.551045  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.524504  2679 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:10.552047  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 293234us
W20260709 14:54:10.550448  2612 rpcz_store.cc:269] 0709 14:54:08.935319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.139383 (+204064us) service_pool.cc:224] Handling call
0709 14:54:10.534031 (+1394648us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:10.534252 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437915008}
I20260709 14:54:10.505693  8775 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.558749  2679 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:10.554369  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.555811  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.563598  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.564795  3547 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 14:54:10.551713  3540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.001s	sys 0.012s
I20260709 14:54:10.529970  8652 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.571069  2612 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:10.577147  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.577876  2605 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: []
I20260709 14:54:10.579386  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:10.502792  7977 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.540516  8710 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.582918  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.505177  8741 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.616195  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.618791  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:10.619901  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:10.622748  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:10.623812  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 265229us
I20260709 14:54:10.627300  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.540179  8668 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 14:54:10.632866  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:10.634478  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:10.575932  8873 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:10.557121  8884 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:10.678527  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:10.681201  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1664) took 2489 ms. Trace:
I20260709 14:54:10.681072  8703 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.681686  8752 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.681936  8816 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:10.683040  8565 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 14:54:10.683619  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.684377  2683 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:54:10.623936  8472 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.881s	user 0.016s	sys 0.000s
I20260709 14:54:10.589749  8708 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 14:54:10.681725  3537 rpcz_store.cc:276] 0709 14:54:08.191860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.201861 (+ 10001us) service_pool.cc:224] Handling call
0709 14:54:08.902970 (+701109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:08.989148 (+ 86178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.680952 (+1691804us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b540008ff50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:10.681129 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611335936}
I20260709 14:54:10.570713  8871 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 14:54:10.686731  2685 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:10.688429  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.702098  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:10.625279  8702 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.005s
I20260709 14:54:10.691776  3538 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:10.633806  8814 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:10.694265  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.685967  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:10.706219  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:10.706449  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.686731  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.708614  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.707931  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:10.637089  8775 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:10.581067  8877 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 14:54:10.802819  8772 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.640380  8717 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:10.634910  8820 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:54:10.676282  8578 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:10.583106  8652 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:10.676488  8803 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:10.679885  8873 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:10.737949  3537 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:10.895605  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.896589  3537 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:10.739607  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1118) took 1661 ms. Trace:
I20260709 14:54:10.898078  2680 rpcz_store.cc:276] 0709 14:54:09.078428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.146026 (+ 67598us) service_pool.cc:224] Handling call
0709 14:54:09.146494 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.529924 (+383430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.739148 (+1209224us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:10.739488 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841632512}
I20260709 14:54:10.740445  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:10.901743  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.903571  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.904589  3537 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 14:54:10.905494  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:10.906482  3537 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 14:54:10.909487  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 169141us
I20260709 14:54:10.718423  2604 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:54:10.774699  8821 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:10.914541  3541 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 14:54:10.711395  8814 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.802500  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1760) took 1484 ms. Trace:
I20260709 14:54:10.795228  7977 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.712870  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:10.806063  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:10.715487  8816 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:10.937821  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 131895us
I20260709 14:54:10.938990  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:10.939116  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.788810  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:10.940743  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 227967us
I20260709 14:54:10.752950  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:10.842279  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:10.737223  8713 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:10.801005  2607 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:10.631309  8710 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:10.907573  8873 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:10.797060  8703 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:10.788051  2682 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 14. Current term is 15. Ops: []
W20260709 14:54:10.767685  8890 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 14:54:10.795920  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1321) took 1086 ms. Trace:
I20260709 14:54:10.640774  8768 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
W20260709 14:54:10.941463  8734 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.047s	user 0.011s	sys 0.004s
I20260709 14:54:10.734109  8884 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 14:54:10.710071  8702 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.736965  8565 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:10.779556  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1115) took 1989 ms. Trace:
I20260709 14:54:10.791409  8834 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:10.675952  8537 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 14:54:10.636973  8704 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 14:54:10.704644  8705 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:10.751632  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:10.705029  8472 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:10.706310  8769 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 14:54:10.780153  8741 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:11.003103  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.004119  2759 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:10.789739  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:10.958345  8882 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:11.005460  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:10.802435  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:10.865716  8749 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 14:54:10.877264  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:10.880264  2679 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:10.875499  8776 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.940s	user 0.013s	sys 0.002s
I20260709 14:54:10.886936  8578 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:10.920876  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:10.918282  2752 rpcz_store.cc:276] 0709 14:54:09.318136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.318314 (+   178us) service_pool.cc:224] Handling call
0709 14:54:10.801636 (+1483322us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521f8c 00007fa16b521ef6 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:10.801837 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:10.802115 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.802141 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:10.802188 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:10.802234 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.802299 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:10.802427 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11121792}
I20260709 14:54:10.943893  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:11.006078  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 316121us
I20260709 14:54:10.759371  8775 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:10.914115  8821 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:10.983392  8866 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:10.952195  3546 rpcz_store.cc:276] 0709 14:54:09.709532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.710621 (+  1089us) service_pool.cc:224] Handling call
0709 14:54:09.740072 (+ 29451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:10.104817 (+364745us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:10.149660 (+ 44843us) log.cc:844] Serialized 14 byte log entry
0709 14:54:10.550423 (+400763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:10.550484 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:10.550550 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.795653 (+245103us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54004b1ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:10.795712 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:10.795856 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123704,"prepare.queue_time_us":157854,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":106780,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":8742,"spinlock_wait_cycles":856694528,"thread_start_us":348753,"threads_started":2,"wal-append.queue_time_us":404072}
I20260709 14:54:10.919538  8803 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:10.963220  8734 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:10.967968  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:11.024307  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:11.023478  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:10.950747  8750 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.004s
I20260709 14:54:10.975309  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:11.025832  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50691.5us
W20260709 14:54:10.955372  8536 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.635s	user 0.015s	sys 0.000s
W20260709 14:54:10.977661  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 287200us
I20260709 14:54:10.990271  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:10.996407  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.027242  2757 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:11.028790  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.034063  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:10.997669  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.035480  2756 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:10.949858  8752 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:10.964105  8565 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:11.002460  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:11.009773  2679 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:11.010227  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.010993  2753 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:10.971041  8870 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:11.012295  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.014941  2681 raft_consensus.cc:2368] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:10.956200  8886 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 14:54:11.015725  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 71927.9us
W20260709 14:54:11.017306  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 214985us
I20260709 14:54:11.017334  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.018193  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:10.919299  8504 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:11.021164  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:11.069384  3537 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:10.998804  8717 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:10.947288  8804 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:11.025913  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:10.964515  2755 rpcz_store.cc:276] 0709 14:54:08.790303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.790444 (+   141us) service_pool.cc:224] Handling call
0709 14:54:08.804180 (+ 13736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.398601 (+594421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.779253 (+1380652us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54001cd9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:10.779474 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307714304}
I20260709 14:54:11.036065  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.053196  2759 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:11.115901  2684 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:11.102280  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.117380  2756 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:11.045365  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:11.118803  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73558.5us
I20260709 14:54:11.016754  8578 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:11.119457  2753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1"
dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:42502
I20260709 14:54:11.078504  2680 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:11.122467  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.124567  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:11.038511  2683 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:11.081449  2679 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:11.000597  8705 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:11.070883  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 380437us
I20260709 14:54:11.016049  8834 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:11.022109  8871 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:11.120645  8713 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:11.025091  8741 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:11.022747  7977 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:11.131114  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:11.139753  2755 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 14:54:11.140142  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:11.131867  2679 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:11.104794  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 163083us
I20260709 14:54:11.105398  2685 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:11.106318  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:11.133713  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.107465  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:11.136298  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.079217  2754 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:54:11.143507  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.143913  2753 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:54:11.144594  2753 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:11.148276  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:11.148803  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.055855  8704 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:11.164922  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:11.152439  2684 raft_consensus.cc:2368] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:11.153354  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:11.147317  8183 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:11.171676  8183 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:11.172226  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:11.173048  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.174304  8183 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:11.147616  8871 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:11.020394  8776 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:11.129370  8705 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:11.178857  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 152046us
I20260709 14:54:11.201746  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:11.151052  8579 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 14:54:11.185719  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 159911us
W20260709 14:54:11.185642  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56851.5us
I20260709 14:54:11.188885  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:11.037447  8536 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:11.184594  2757 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:11.194000  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:11.022987  8775 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:11.197870  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1672) took 2442 ms. Trace:
I20260709 14:54:11.228374  3536 rpcz_store.cc:276] 0709 14:54:08.755540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.756385 (+   845us) service_pool.cc:224] Handling call
0709 14:54:08.756696 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:09.446476 (+689780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:11.197538 (+1751062us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54002012d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:11.197791 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584579328}
I20260709 14:54:11.229092  3540 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 14:54:11.031605  8749 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 14:54:11.114769  8734 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:11.149875  8752 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 14:54:11.253644  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.161180  2609 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:11.204232  2679 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:11.104708  8504 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:11.148002  8834 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:11.139572  8741 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:11.190361  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.067968  8363 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 14:54:11.208523  3537 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:11.209721  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:11.101187  8769 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 14:54:11.176429  8704 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:11.147914  7977 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:11.213379  3546 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 14:54:11.292176  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150574us
I20260709 14:54:11.031327  8866 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:11.297420  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.245365  8734 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:11.197995  8836 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 2.369s	user 0.000s	sys 0.018s
I20260709 14:54:11.230275  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:11.196743  8804 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:11.130407  8717 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:11.244531  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:11.101651  8565 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:11.254458  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:11.203233  8768 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:11.256387  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 317522us
I20260709 14:54:11.256760  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:11.211410  8900 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 14:54:11.206432  8579 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:11.213380  8592 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:11.272917  2604 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:11.270339  2753 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:11.293916  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:11.197479  8877 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 14:54:11.237601  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:11.273633  8778 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 14:54:11.263715  8504 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:11.341766  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.310765  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.342612  2754 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:11.264076  8834 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:11.312213  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.000s (SENT) [suppressed 34 similar messages]
I20260709 14:54:11.269003  8749 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 14:54:11.294689  8904 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.339s	user 0.002s	sys 0.000s
I20260709 14:54:11.315096  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.315644  2757 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:11.315927  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:11.492278  3546 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:11.297242  8704 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:11.246685  8750 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 14:54:11.302196  7977 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:11.454259  8880 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:11.268105  8847 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 14:54:11.474985  2753 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:11.313472  8472 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:11.334862  8579 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:11.306742  8734 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:11.308852  8702 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:11.445851  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108521us
W20260709 14:54:11.343977  8905 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.360s	user 0.002s	sys 0.000s
I20260709 14:54:11.477289  2759 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:54:11.478892  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.313177  8871 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:11.276428  8803 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:11.271040  8713 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:11.490545  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.537741  2756 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 14:54:11.314736  8363 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.140s
I20260709 14:54:11.320412  8768 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:11.343509  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 154732us
I20260709 14:54:11.545707  2723 leader_election.cc:400] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:11.546274  2723 leader_election.cc:403] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:11.458223  8710 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:11.477218  8882 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:11.471824  8592 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:11.485235  8914 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:11.476356  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:11.585688  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 109435us
I20260709 14:54:11.541308  3546 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:11.542467  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1680) took 1184 ms. Trace:
I20260709 14:54:11.589138  3543 rpcz_store.cc:276] 0709 14:54:10.357947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.427532 (+ 69585us) service_pool.cc:224] Handling call
0709 14:54:10.427932 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.450318 (+ 22386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.450363 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.494110 (+ 43747us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.713298 (+219188us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:10.713341 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.713416 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:11.542055 (+828639us) raft_consensus.cc:1685] finished
0709 14:54:11.542097 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:11.542361 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61963,"raft.queue_time_us":28951,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":26320,"thread_start_us":198098,"threads_started":1,"wal-append.queue_time_us":378417}
I20260709 14:54:11.511519  8845 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:11.538734  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:11.547930  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.492409  8917 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 14:54:11.552624  8911 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 14:54:11.503670  8804 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 14:54:11.555692  8913 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 14:54:11.555858  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:11.343686  8741 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:11.567584  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:11.567956  8775 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:11.569491  8717 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:11.573894  8578 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:11.575835  8866 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:11.583743  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:11.594632  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55994.4us
I20260709 14:54:11.588030  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:11.521157  8880 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:11.592720  8752 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:11.582402  8768 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:11.608239  2759 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:11.581899  8767 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.404s
W20260709 14:54:11.642725  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 332271us
I20260709 14:54:11.581214  8886 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.405s
W20260709 14:54:11.654722  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.127920  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.128762  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.129213  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:11.602656  2757 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:11.593086  8845 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:12.130456  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.040757  3546 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [16.9-19.10]
I20260709 14:54:12.095261  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:11.983628  8732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.007s	sys 0.017s
W20260709 14:54:12.107404  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 519479us
I20260709 14:54:11.613502  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.157723  3536 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:54:12.106637  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 523000us
I20260709 14:54:12.166788  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1687) took 1476 ms. Trace:
I20260709 14:54:11.596578  8592 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.161725  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.164413  2756 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:12.119339  8752 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:11.604351  8921 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:12.190749  3546 rpcz_store.cc:276] 0709 14:54:10.689314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.604156 (+914842us) service_pool.cc:224] Handling call
0709 14:54:12.040347 (+436191us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:12.165273 (+124926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.165978 (+   705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:12.126194  8776 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:12.198390  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.055s	user 0.006s	sys 0.081s
I20260709 14:54:12.119189  8775 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:12.157256  8842 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:12.157981  8749 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:12.203522  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1681) took 1838 ms. Trace:
I20260709 14:54:12.222303  3547 rpcz_store.cc:276] 0709 14:54:10.374238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.570417 (+196179us) service_pool.cc:224] Handling call
0709 14:54:10.571138 (+   721us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:10.631827 (+ 60689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.631872 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:10.632732 (+   860us) log.cc:844] Serialized 78 byte log entry
0709 14:54:10.994368 (+361636us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:10.994463 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.994592 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.203116 (+1208524us) raft_consensus.cc:1685] finished
0709 14:54:12.203152 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.203429 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34705,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12670,"thread_start_us":274232,"threads_started":1,"wal-append.queue_time_us":357943}
W20260709 14:54:12.209630  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.006s	sys 0.081s
W20260709 14:54:12.214156  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.113s	sys 0.192s
I20260709 14:54:12.175871  8717 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:12.202816  8713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.019s	sys 0.068s
I20260709 14:54:12.220602  2753 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:12.186192  8886 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.035127  8741 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:12.225250  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.113s	sys 0.192s
I20260709 14:54:12.207433  8752 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:12.236284  2756 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:11.943117  8925 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:12.198379  8732 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:11.984004  8536 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:12.213550  8643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.015s	sys 0.059s
I20260709 14:54:12.204321  8921 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:12.197849  8880 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.204658  8363 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:12.246364  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.078s	sys 0.086s
W20260709 14:54:12.275385  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.078s	sys 0.087s
W20260709 14:54:12.234417  8713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.020s	sys 0.068s
I20260709 14:54:12.236848  8717 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:12.234784  8775 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:12.268700  2757 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:12.269599  8643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.016s	sys 0.060s
I20260709 14:54:11.596618  8914 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.242223  8924 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:12.291162  8924 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:12.292338  8923 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 14:54:12.292814  8875 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.109s	user 0.002s	sys 0.003s
W20260709 14:54:12.293684  8873 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.325s	user 0.017s	sys 0.000s
I20260709 14:54:12.293866  8924 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.294819  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1478) took 1619 ms. Trace:
I20260709 14:54:12.295439  3536 rpcz_store.cc:276] 0709 14:54:10.675845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.319730 (+643885us) service_pool.cc:224] Handling call
0709 14:54:11.333887 (+ 14157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.189707 (+855820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.294536 (+104829us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54001a0254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.294743 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894061312}
W20260709 14:54:12.284829  8579 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260709 14:54:12.276866  8921 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:12.281946  8917 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.280045  8717 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:12.287351  8925 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:12.296965  8927 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 14:54:12.294363  8873 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.244253  8886 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.276341  8363 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:12.233635  8845 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.271139  8732 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.326026  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1635) took 1316 ms. Trace:
I20260709 14:54:12.287949  8775 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:12.305233  3543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.011s	sys 0.020s
I20260709 14:54:12.332441  2680 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:12.333745  3543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.012s	sys 0.020s
I20260709 14:54:12.328730  2752 rpcz_store.cc:276] 0709 14:54:11.011679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.013688 (+  2009us) service_pool.cc:224] Handling call
0709 14:54:11.014043 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:11.014295 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:11.188448 (+174153us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.244071 (+1055623us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.325422 (+ 81351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:12.325495 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:12.325564 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.325733 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.325933 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":933366,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":125,"raft.queue_time_us":92892,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":15824,"spinlock_wait_cycles":20731008,"thread_start_us":1284413,"threads_started":3,"wal-append.queue_time_us":1117026}
I20260709 14:54:12.335372  8842 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:12.337440  8948 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.333422  8536 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.338410  8578 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.340497  8916 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:12.341176  8902 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.540s	user 0.000s	sys 0.028s
W20260709 14:54:12.345840  8565 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.000s
I20260709 14:54:12.308658  8579 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:12.324437  8363 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:12.349035  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1767) took 1364 ms. Trace:
I20260709 14:54:12.374919  2758 rpcz_store.cc:276] 0709 14:54:10.984635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.987287 (+  2652us) service_pool.cc:224] Handling call
0709 14:54:10.987732 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:11.291114 (+303382us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:11.503698 (+212584us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.286916 (+783218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:12.286998 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:12.287091 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.348752 (+ 61661us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.348809 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.348967 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13214,"prepare.queue_time_us":742335,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":34358,"raft.queue_time_us":18696,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":666318720,"thread_start_us":964866,"threads_started":2,"wal-append.queue_time_us":793571}
I20260709 14:54:12.378190  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.326997  8915 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:12.347496  8930 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 14:54:12.390229  8935 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 14:54:12.392859  8785 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:54:12.393456  8785 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:12.393985  8938 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 14:54:12.394397  8785 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.395922  8785 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:12.396703  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1128) took 1095 ms. Trace:
I20260709 14:54:12.397498  8775 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:12.399339  8886 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:12.404433  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.33659e+06us
I20260709 14:54:12.405515  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:12.323556  8925 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:12.407203  3546 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 12. Current term is 13. Ops: []
I20260709 14:54:12.408130  8708 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:12.408946  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1646) took 1107 ms. Trace:
I20260709 14:54:12.409178  8925 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.316418  8917 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.409416  2757 rpcz_store.cc:276] 0709 14:54:11.301066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.132175 (+831109us) service_pool.cc:224] Handling call
0709 14:54:12.132743 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.280297 (+147554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.408634 (+128337us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.408869 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312024320}
I20260709 14:54:12.411540  8917 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:12.412153  8933 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 14:54:12.412284  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1814) took 1843 ms. Trace:
I20260709 14:54:12.416657  2612 rpcz_store.cc:276] 0709 14:54:10.568803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.569092 (+   289us) service_pool.cc:224] Handling call
0709 14:54:10.569591 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:10.571524 (+  1933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.411919 (+1840395us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b5400123a20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e649a02 00007fa16e645dd6 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:54:12.412213 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20539136}
I20260709 14:54:10.762800   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:54:12.418119  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.350176  8948 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.413865  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1613) took 1441 ms. Trace:
I20260709 14:54:12.419744  2680 rpcz_store.cc:276] 0709 14:54:10.972718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.121999 (+149281us) service_pool.cc:224] Handling call
0709 14:54:12.413629 (+1291630us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.413793 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2780049152}
I20260709 14:54:12.414389  2759 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:12.413378  8873 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:12.422178  8934 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 14:54:12.422587  8945 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:12.422140  8948 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.423027  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1127) took 1121 ms. Trace:
I20260709 14:54:12.423285  8945 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.423470  2759 rpcz_store.cc:276] 0709 14:54:11.301335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.158558 (+857223us) service_pool.cc:224] Handling call
0709 14:54:12.413982 (+255424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.421221 (+  7239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.422754 (+  1533us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540050e1d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.422959 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513547648}
I20260709 14:54:12.424875  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:12.425094  8899 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.624s	user 0.016s	sys 0.032s
I20260709 14:54:12.425592  2759 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:12.425983  8945 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.410790  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:12.427738  8914 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.374393  8916 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.397220  2756 rpcz_store.cc:276] 0709 14:54:11.301626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.196742 (+895116us) service_pool.cc:224] Handling call
0709 14:54:12.197525 (+   783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.243315 (+ 45790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.396356 (+153041us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.396621 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88184064}
I20260709 14:54:12.426666  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:12.433380  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:12.428526  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1090) took 2531 ms. Trace:
W20260709 14:54:12.428689  7977 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.911s	user 0.013s	sys 0.000s
I20260709 14:54:12.346755  8652 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.350899  8832 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:12.433990  2608 rpcz_store.cc:276] 0709 14:54:09.897125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.396620 (+499495us) service_pool.cc:224] Handling call
0709 14:54:10.396988 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.433689 (+ 36701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.433740 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.434646 (+   906us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.710711 (+276065us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:10.710765 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.710865 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.424342 (+1713477us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:12.428134 (+  3792us) raft_consensus.cc:1685] finished
0709 14:54:12.428159 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.428454 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35973,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":137449216,"thread_start_us":275409,"threads_started":1,"wal-append.queue_time_us":366382}
I20260709 14:54:12.437712  3537 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:12.374099  8565 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.445899  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:12.446954  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.447124  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.452714  2758 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:12.355010  8880 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.430594  8916 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.455354  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:12.452001  2608 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:54:12.457693  2758 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:12.458779  2604 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:12.458032  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1666) took 2589 ms (client timeout 2000 ms). Trace:
I20260709 14:54:12.463757  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:12.463455  2609 rpcz_store.cc:269] 0709 14:54:09.868126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.273739 (+405613us) service_pool.cc:224] Handling call
0709 14:54:12.457791 (+2184052us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b5400280354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.457949 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606499712}
I20260709 14:54:12.466113  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.466719  2609 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16. Re-sending same reply.
I20260709 14:54:12.469301  2752 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:12.358335  8536 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:12.451112  8734 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.914s	user 0.012s	sys 0.000s
I20260709 14:54:12.471335  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.472004  2608 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:12.467664  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.473117  2609 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:12.464722  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60808.9us
I20260709 14:54:12.441797  8732 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.476267  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.452410  8871 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.484444  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.382174  8752 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:12.469820  8840 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.683s	user 0.013s	sys 0.012s
I20260709 14:54:12.491618  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.497887  2609 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:12.350961  8842 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.474805  8947 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.468888  8578 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:12.451781  7977 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.503026  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1686) took 1814 ms. Trace:
I20260709 14:54:12.505412  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.506211  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:12.497148  8840 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.014s	sys 0.012s
I20260709 14:54:12.455196  8832 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.456063  8768 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.509678  2609 raft_consensus.cc:2376] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18. Re-sending same reply.
I20260709 14:54:12.512917  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1614) took 1539 ms. Trace:
I20260709 14:54:12.469151  8880 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.475025  8776 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 14:54:12.498929  8652 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.518517  2685 rpcz_store.cc:276] 0709 14:54:10.973051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.166183 (+193132us) service_pool.cc:224] Handling call
0709 14:54:11.166588 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:11.213896 (+ 47308us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:11.213951 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:11.258574 (+ 44623us) log.cc:844] Serialized 52 byte log entry
0709 14:54:11.513023 (+254449us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:11.513100 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:11.513223 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.510143 (+996920us) raft_consensus.cc:1685] finished
0709 14:54:12.510171 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.512820 (+  2649us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46704,"raft.queue_time_us":62313,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":32972,"thread_start_us":204596,"threads_started":1,"wal-append.queue_time_us":746683}
I20260709 14:54:12.526458  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.523679  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.528071  2608 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:12.530596  8880 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.530798  8708 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:12.531785  8880 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:12.532852  8880 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 14:54:12.535027  8768 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.535882  2751 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:12.537171  8893 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.792s	user 0.011s	sys 0.047s
W20260709 14:54:12.538390  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1637) took 1521 ms (client timeout 1923 ms). Trace:
I20260709 14:54:12.539849  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1477) took 1959 ms. Trace:
I20260709 14:54:12.540324  3540 rpcz_store.cc:276] 0709 14:54:10.580990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.633691 (+ 52701us) service_pool.cc:224] Handling call
0709 14:54:10.691989 (+ 58298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:11.229657 (+537668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.539523 (+1309866us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54004b2dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.539775 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234414976}
I20260709 14:54:12.544750  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.545636  2611 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:12.544799  2751 rpcz_store.cc:269] 0709 14:54:11.017374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.033633 (+ 16259us) service_pool.cc:224] Handling call
0709 14:54:12.538143 (+1504510us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.538324 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3429525248}
I20260709 14:54:12.479080  8734 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.555990  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.515725  8945 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:12.564549  2609 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:12.566779  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111491us
I20260709 14:54:12.562151  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.498308  8871 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.572626  2608 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:12.569939  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.575902  8956 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:12.580442  8956 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:12.580942  8956 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:12.581306  8752 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.581857  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1595) took 1865 ms. Trace:
I20260709 14:54:12.576129  8565 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:12.577877  8703 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:12.584208  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1480) took 1786 ms. Trace:
I20260709 14:54:12.584566  3537 rpcz_store.cc:276] 0709 14:54:10.797964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.293564 (+495600us) service_pool.cc:224] Handling call
0709 14:54:12.583965 (+1290401us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400551254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.584139 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528888704}
I20260709 14:54:12.503083  8921 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:12.579680  8896 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.897s	user 0.009s	sys 0.008s
I20260709 14:54:12.583782  8703 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:12.582291  2679 rpcz_store.cc:276] 0709 14:54:10.716234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.716446 (+   212us) service_pool.cc:224] Handling call
0709 14:54:10.716877 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:11.254523 (+537646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.581553 (+1327030us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400445554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.581779 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375264512}
W20260709 14:54:12.590067  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.561414  8504 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.106s	user 0.008s	sys 0.007s
I20260709 14:54:12.587389  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.584677  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:12.600430  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:12.600934  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:12.606997  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:12.608619  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:12.577947  8704 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.058s	user 0.008s	sys 0.004s
I20260709 14:54:12.612156  2611 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:12.555572  8652 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.612870  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:12.611182  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.609084  8776 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:12.615375  8958 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:54:12.617760  8943 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 14:54:12.618037  8871 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.643644  8871 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:12.644289  8871 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:12.644819  8871 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:12.645169  8936 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.154s	user 0.001s	sys 0.001s
I20260709 14:54:12.630336  8579 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:12.619647  8950 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 14:54:12.620677  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1327) took 2337 ms. Trace:
I20260709 14:54:12.514962  8947 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.607368  8804 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.628345  8776 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:12.623560  8704 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.606555  8504 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.648908  8776 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:12.569509  8945 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:12.715715  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 276506us
W20260709 14:54:12.720952  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.721614  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1591) took 3834 ms (client timeout 2076 ms). Trace:
I20260709 14:54:12.610778  8946 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:12.703795  8853 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:12.705735  8958 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:12.647164  3545 rpcz_store.cc:276] 0709 14:54:10.283336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.286169 (+  2833us) service_pool.cc:224] Handling call
0709 14:54:10.324857 (+ 38688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.325611 (+   754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.325679 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.326526 (+   847us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.567089 (+240563us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:10.567160 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.567278 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.578808 (+2011530us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:12.620264 (+ 41456us) raft_consensus.cc:1685] finished
0709 14:54:12.620294 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.620600 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":104,"spinlock_wait_cycles":5451520,"thread_start_us":211065,"threads_started":1,"wal-append.queue_time_us":355740}
W20260709 14:54:12.721756  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.722190  2678 rpcz_store.cc:269] 0709 14:54:08.887442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.887633 (+   191us) service_pool.cc:224] Handling call
0709 14:54:12.721383 (+3833750us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54002b3c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.721522 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1650463488}
I20260709 14:54:12.635136  8741 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:12.723985  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.712284  8579 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:12.729318  2612 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:12.706565  8914 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:12.732933  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.734586  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110349us
I20260709 14:54:12.735764  2608 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 11.
W20260709 14:54:12.743392  8901 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.948s	user 0.006s	sys 0.010s
W20260709 14:54:12.743674  8955 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 14:54:12.745420  8895 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.992s	user 0.000s	sys 0.018s
I20260709 14:54:12.747222  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1812) took 2602 ms. Trace:
I20260709 14:54:12.747704  2610 rpcz_store.cc:276] 0709 14:54:10.144436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.438723 (+294287us) service_pool.cc:224] Handling call
0709 14:54:10.439233 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.481223 (+ 41990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.481276 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.482263 (+   987us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.690299 (+208036us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:10.690345 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.690425 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.643638 (+1953213us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:12.746778 (+103140us) raft_consensus.cc:1685] finished
0709 14:54:12.746806 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.747136 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39303,"raft.queue_time_us":40749,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":145,"spinlock_wait_cycles":149672320,"thread_start_us":207471,"threads_started":1,"wal-append.queue_time_us":312604}
I20260709 14:54:12.749850  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1325) took 2467 ms. Trace:
I20260709 14:54:12.742882  8914 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.746372  8947 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:12.737764  8894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.956s	user 0.002s	sys 0.010s
I20260709 14:54:12.757992  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1775) took 1273 ms. Trace:
I20260709 14:54:12.707077  8804 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:12.762776  2759 rpcz_store.cc:276] 0709 14:54:11.484346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.433731 (+949385us) service_pool.cc:224] Handling call
0709 14:54:12.439694 (+  5963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.439976 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:12.440864 (+   888us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.757595 (+316731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:12.757660 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:12.757722 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.757808 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.757931 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":289,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":483,"thread_start_us":315804,"threads_started":1}
W20260709 14:54:12.763978  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 268201us
I20260709 14:54:12.750484  3542 rpcz_store.cc:276] 0709 14:54:10.282487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.282668 (+   181us) service_pool.cc:224] Handling call
0709 14:54:10.288378 (+  5710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.288822 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.288856 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.293560 (+  4704us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.593305 (+299745us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:10.689988 (+ 96683us) log.cc:844] Serialized 14 byte log entry
0709 14:54:10.695572 (+  5584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.695693 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:10.799396 (+103703us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b1800389ce0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:12.719256 (+1919860us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:12.749600 (+ 30344us) raft_consensus.cc:1685] finished
0709 14:54:12.749628 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.749787 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5194,"prepare.queue_time_us":73611,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":6061,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":30393,"spinlock_wait_cycles":152921600,"thread_start_us":299128,"threads_started":1,"wal-append.queue_time_us":455441}
I20260709 14:54:12.725062  8717 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.772467  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.767161  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:12.785444  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:12.786721  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.745s (SENT)
I20260709 14:54:12.787539  2759 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 12.
I20260709 14:54:12.787724  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:12.789314  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:12.746515  8956 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.798740  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:12.802387  2759 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:12.765161  8894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.984s	user 0.002s	sys 0.010s
I20260709 14:54:12.767428  8536 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.785140  8741 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.789150  8745 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:12.799036  7977 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:12.815706  8821 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.817271  8821 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:12.823668  8959 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 14:54:12.825824  8816 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:12.826478  8816 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:12.827533  8816 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:12.828649  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1470) took 2327 ms. Trace:
I20260709 14:54:12.829082  8816 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:12.829138  3541 rpcz_store.cc:276] 0709 14:54:10.501003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.632755 (+131752us) service_pool.cc:224] Handling call
0709 14:54:10.692583 (+ 59828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:10.915224 (+222641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.828336 (+1913112us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400225454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:12.828577 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509136128}
I20260709 14:54:12.831715  8969 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 14:54:12.832537  8969 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:12.833000  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1077) took 3940 ms (client timeout 1542 ms). Trace:
W20260709 14:54:12.833534  2603 rpcz_store.cc:269] 0709 14:54:08.892908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.939599 (+ 46691us) service_pool.cc:224] Handling call
0709 14:54:12.832673 (+3893074us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400128254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:12.832896 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651538176}
I20260709 14:54:12.834206  8945 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.823746  8704 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.836529  8504 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:12.786006  8832 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.839222  2606 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:12.844925  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:12.796406  8848 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.847169  8804 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:12.847781  8965 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 14:54:12.800423  8734 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:12.833050  8969 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:54:12.731770  8946 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.863659  2759 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:12.869946  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:12.871098  8963 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.059s	user 0.001s	sys 0.003s
I20260709 14:54:12.859328  8745 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:12.875425  8745 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.878039  8820 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:12.878607  8820 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:12.879009  8820 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:12.879921  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:12.882217  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.65196e+06us
I20260709 14:54:12.852362  7977 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:12.889632  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.894304  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:12.747721  8958 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.897423  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1652) took 1437 ms. Trace:
I20260709 14:54:12.898622  2756 rpcz_store.cc:276] 0709 14:54:11.462426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.430094 (+967668us) service_pool.cc:224] Handling call
0709 14:54:12.430532 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:12.430979 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:12.431025 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:12.431750 (+   725us) log.cc:844] Serialized 52 byte log entry
0709 14:54:12.711513 (+279763us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:12.836292 (+124779us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:12.879692 (+ 43400us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.881054 (+  1362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.881162 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:12.881230 (+    68us) raft_consensus.cc:1685] finished
0709 14:54:12.897172 (+ 15942us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.897360 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192,"raft.queue_time_us":1861,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":12526,"spinlock_wait_cycles":10514688,"thread_start_us":431192,"threads_started":2,"wal-append.queue_time_us":323967}
I20260709 14:54:12.901290  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:12.897998  2759 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 20
I20260709 14:54:12.906301  2756 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:12.908787  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.37417e+06us
I20260709 14:54:12.909780  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:12.878033  8745 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.888159  8908 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.913040  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.855134  8741 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:12.891341  8668 raft_consensus.cc:2764] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 14:54:12.928105  2759 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:12.924808  2756 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
W20260709 14:54:12.932485  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 136117us
I20260709 14:54:12.896274  8717 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:12.936254  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:12.918308  8915 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.009s
I20260709 14:54:12.942274  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:12.941157  2756 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 12: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:12.948666  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.919255  8910 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:12.950850  8960 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 14:54:12.951799  8910 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:12.940060  8363 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.554s	user 0.015s	sys 0.000s
I20260709 14:54:12.909088  7977 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.948551  8915 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:12.934758  8776 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:12.964987  8961 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 14:54:12.963433  8960 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:12.935400  8908 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:12.960901  8717 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 14:54:13.025734  8898 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.232s	user 0.005s	sys 0.008s
I20260709 14:54:12.984910  8914 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:13.035244  8972 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 14:54:13.037070  8910 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.040390  8834 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.051755  8834 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:13.053010  8870 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:13.053561  8870 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:13.050807  8903 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.258s	user 0.017s	sys 0.029s
I20260709 14:54:13.030027  8946 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:13.060109  8921 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.061112  8871 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:12.985992  8958 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 14:54:13.064198  8958 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:13.064774  8958 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:13.073112  8888 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 2.441s	user 0.031s	sys 0.047s
I20260709 14:54:13.074954  8845 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:13.079540  8884 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:13.054456  8870 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.080500  8884 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.081437  8870 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:12.991000  8967 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 14:54:13.086474  8884 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.087549  8908 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.066083  8958 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.089080  8958 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:13.091056  8820 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.094352  8820 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.160981  8847 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:13.161232  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:12.989552  8579 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:13.162528  8847 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.163853  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:13.165103  2759 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:13.164686  8579 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:13.140681  2603 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:13.171144  8820 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:12.965909  8363 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.172406  8820 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:13.176023  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.177445  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:54:13.140451  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:13.142741  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1097) took 2163 ms. Trace:
W20260709 14:54:13.147585  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 262678us
I20260709 14:54:13.149214  2756 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:13.149621  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1584) took 2987 ms. Trace:
I20260709 14:54:13.195240  2677 rpcz_store.cc:276] 0709 14:54:10.162348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.294721 (+132373us) service_pool.cc:224] Handling call
0709 14:54:10.309704 (+ 14983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:10.310089 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:10.310135 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:10.365844 (+ 55709us) log.cc:844] Serialized 26 byte log entry
0709 14:54:10.698004 (+332160us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:10.698050 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:10.698123 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:13.025027 (+2326904us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:13.148613 (+123586us) raft_consensus.cc:1685] finished
0709 14:54:13.148671 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:13.149262 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155077504,"thread_start_us":295167,"threads_started":1,"wal-append.queue_time_us":391465}
I20260709 14:54:13.197081  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1087) took 3674 ms. Trace:
I20260709 14:54:13.197741  2607 rpcz_store.cc:276] 0709 14:54:09.522552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.703985 (+181433us) service_pool.cc:224] Handling call
0709 14:54:09.705249 (+  1264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:10.800791 (+1095542us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b5400122b20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:54:10.946905 (+146114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.196402 (+2249497us) spinlock_profiling.cc:243] Waited 96.4 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:13.196822 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241931776}
I20260709 14:54:13.199308  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:13.182577  2605 rpcz_store.cc:276] 0709 14:54:10.978661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.702681 (+1724020us) service_pool.cc:224] Handling call
0709 14:54:12.703180 (+   499us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:12.703653 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:12.703685 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:12.704407 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 14:54:12.771362 (+ 66955us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:12.798240 (+ 26878us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.798505 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.798591 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:13.026901 (+228310us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b1800082c00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:13.141783 (+114882us) raft_consensus.cc:1685] finished
0709 14:54:13.142017 (+   234us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:13.142468 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26520,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":4750,"raft.queue_time_us":2532,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":284,"spinlock_wait_cycles":3230592,"thread_start_us":66376,"threads_started":1,"wal-append.queue_time_us":176580}
I20260709 14:54:13.200664  2758 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:54:13.201213  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1092) took 2226 ms. Trace:
I20260709 14:54:13.201618  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.201812  2604 rpcz_store.cc:276] 0709 14:54:10.974412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.339085 (+364673us) service_pool.cc:224] Handling call
0709 14:54:12.458583 (+1119498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.469555 (+ 10972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.200700 (+731145us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400280350. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:13.200983 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308623744}
W20260709 14:54:13.156935  2724 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.220s (SENT)
I20260709 14:54:13.159242  8741 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:13.153023  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 243157us
I20260709 14:54:13.206143  2603 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:13.207892  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:13.207999  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:13.192902  8847 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:13.210179  8750 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:13.154649  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.212867  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1803) took 4299 ms. Trace:
I20260709 14:54:13.179927  8363 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.206329  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:13.170630  7977 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:13.226243  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:13.239077  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:13.241001  2758 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:13.231343  2606 rpcz_store.cc:276] 0709 14:54:08.913679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:09.493278 (+579599us) service_pool.cc:224] Handling call
0709 14:54:12.838957 (+3345679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.839804 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.212148 (+372344us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b5400128250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:13.212632 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7693285760}
I20260709 14:54:13.272423  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:13.273649  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1479) took 2475 ms. Trace:
I20260709 14:54:13.242308  8910 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:13.276074  8910 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:13.243901  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.236025  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:13.078306  8969 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:13.247816  8820 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:13.259842  2678 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
W20260709 14:54:13.261816  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105229us
I20260709 14:54:13.267246  2609 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:13.267315  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:13.282030  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142327us
I20260709 14:54:13.274346  3546 rpcz_store.cc:276] 0709 14:54:10.797670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.209488 (+1411818us) service_pool.cc:224] Handling call
0709 14:54:12.233108 (+ 23620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.407759 (+174651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.272781 (+865022us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54002026d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:13.273285 (+   504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399779200}
I20260709 14:54:13.288028  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.289383  8652 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:13.267462  8921 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:13.311673  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:13.313218  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.014s
W20260709 14:54:13.314006  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.001s	sys 0.015s
I20260709 14:54:13.316601  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1096) took 2338 ms. Trace:
I20260709 14:54:13.319286  2681 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:13.319582  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.321183  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.321314  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.322335  2681 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:13.294008  8847 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 14:54:13.324276  8847 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:13.324369  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:13.324780  8847 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:13.325860  8845 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:13.326489  8845 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:13.327032  8845 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:13.329357  8910 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.331089  8969 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:13.336483  8947 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:13.337539  8947 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:13.340215  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.339931  8878 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:13.340626  2753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1131) took 2034 ms (client timeout 2679 ms). Trace:
W20260709 14:54:13.341176  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.343322  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:13.345827  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51194.5us
I20260709 14:54:13.346263  8866 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:13.347445  2603 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:54:13.349622  2754 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:13.355691  8921 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:13.356278  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.341176  8878 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:13.358073  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1593) took 2672 ms (client timeout 2989 ms). Trace:
W20260709 14:54:13.341359  2753 rpcz_store.cc:269] 0709 14:54:11.305919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.605414 (+299495us) service_pool.cc:224] Handling call
0709 14:54:13.340038 (+1734624us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:13.340378 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77467904}
I20260709 14:54:13.360411  8776 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:13.360661  8702 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:54:13.350666  8978 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:13.362408  8947 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.382892  2759 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:13.363090  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.364513  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.317230  2611 rpcz_store.cc:276] 0709 14:54:10.978413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.624172 (+1645759us) service_pool.cc:224] Handling call
0709 14:54:12.626863 (+  2691us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:12.627520 (+   657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:12.627570 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:12.662326 (+ 34756us) log.cc:844] Serialized 52 byte log entry
0709 14:54:13.314919 (+652593us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:13.315517 (+   598us) log.cc:844] Serialized 14 byte log entry
0709 14:54:13.315914 (+   397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.316084 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:13.316161 (+    77us) raft_consensus.cc:1685] finished
0709 14:54:13.316191 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:13.316500 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1639,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":26242,"thread_start_us":581714,"threads_started":1,"wal-append.queue_time_us":521574}
I20260709 14:54:13.293113  8703 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:13.394728  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.363677  8845 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:13.370369  2683 rpcz_store.cc:269] 0709 14:54:10.685561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.686413 (+   852us) service_pool.cc:224] Handling call
0709 14:54:13.357613 (+2671200us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:13.357823 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757793536}
W20260709 14:54:13.406739  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.003s	sys 0.023s
W20260709 14:54:13.407401  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.003s	sys 0.023s
I20260709 14:54:13.412165  2681 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:13.411216  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 644113us
I20260709 14:54:13.418202  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:13.367112  8847 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.427745  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.433656  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.440209  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:13.443028  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62141.3us
W20260709 14:54:13.451810  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.452447  8703 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:13.454624  2603 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
W20260709 14:54:13.455322  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.455226  8885 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:13.456820  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.458880  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.463451  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.464692  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:13.425065  8866 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.468698  8878 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.472820  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.474036  3546 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:13.476389  3538 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:13.481079  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:13.468792  8866 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:13.483116  8652 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 14:54:13.483572  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52454.7us
I20260709 14:54:13.484829  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:13.484992  8845 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.483763  8652 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:13.452353  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:13.503320  2681 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:13.479625  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1649) took 2138 ms. Trace:
I20260709 14:54:13.507612  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.514633  2683 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:13.505395  2752 rpcz_store.cc:276] 0709 14:54:11.341103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.349074 (+1007971us) service_pool.cc:224] Handling call
0709 14:54:12.349488 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:12.469987 (+120499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.478966 (+1008979us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54000e6d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:13.479358 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275047680}
W20260709 14:54:13.528023  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104644us
I20260709 14:54:13.495486  8652 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:13.533043  8847 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:13.534755  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.537147  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1769) took 2551 ms. Trace:
I20260709 14:54:13.537886  8866 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:12.508632  3543 rpcz_store.cc:276] 0709 14:54:10.689055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.598568 (+909513us) service_pool.cc:224] Handling call
0709 14:54:11.599156 (+   588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:11.599524 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:12.334388 (+734864us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.502565 (+168177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:12.502629 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:12.502696 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:12.502773 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:12.502960 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3282,"prepare.queue_time_us":737393,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"thread_start_us":898696,"threads_started":3,"wal-append.queue_time_us":178514}
I20260709 14:54:13.549043  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.482733  8894 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:13.550709  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.559257  2754 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:13.490196  8878 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:13.567927  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.568181  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.447022  8978 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.569161  8908 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:13.599886  8908 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.602558  8866 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:13.603241  8866 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:13.603503  8884 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:13.604458  8884 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:13.607568  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.608793  3540 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:13.607653  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.611831  3547 raft_consensus.cc:2368] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
I20260709 14:54:13.613035  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.613174  8978 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:13.613757  3547 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:13.614635  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162620us
I20260709 14:54:13.617841  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:13.619275  8578 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:13.619515  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:13.620322  8578 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:13.621912  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:13.610599  8956 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:13.624163  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 93250.4us
I20260709 14:54:13.626093  2756 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 14:54:13.626106  8803 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:13.628187  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.630116  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.584241  8894 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:13.632493  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.632853  8894 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:13.635456  2680 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:13.610388  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.526021  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.572868  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.574245  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.490221  7977 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:13.577589  2685 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:13.577538  2760 rpcz_store.cc:276] 0709 14:54:10.985491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:10.989738 (+  4247us) service_pool.cc:224] Handling call
0709 14:54:11.008121 (+ 18383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:11.143559 (+135438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:11.143595 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:11.143643 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:11.143683 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.536563 (+2392880us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:13.536667 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:13.536892 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310875520}
I20260709 14:54:13.595739  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.633555  8921 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:13.633618  2682 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:13.639894  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:13.641232  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.639976  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.638450  2752 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:13.641649  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:13.643110  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64675us
I20260709 14:54:13.645503  3541 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:13.646090  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.646476  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.647054  2685 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:13.647364  3536 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
W20260709 14:54:13.648339  8870 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.013s
I20260709 14:54:13.645604  7977 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:13.651098  7977 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:13.651543  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.651027  8803 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:13.652482  8878 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:13.653155  8878 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:13.653177  8803 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:13.653793  8803 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:13.655015  2682 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:13.657660  2724 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:13.659996  8871 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:13.652503  3543 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
I20260709 14:54:13.661319  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:13.662177  3543 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:13.662467  2760 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:13.605113  8884 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:13.663002  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:13.663757  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:13.664475  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:13.664746  2760 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:13.665438  2677 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:13.667554  3542 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:13.673516  2606 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:13.661458  8713 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 14:54:13.675212  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.679819  2760 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:13.655072  8803 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:13.683239  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.684109  8803 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:13.622081  8578 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.686326  3541 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:13.691771  2606 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 14:54:13.662177  2756 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:13.693663  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67111.6us
I20260709 14:54:13.698736  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:13.699900  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.700202  8921 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:13.702008  8921 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:13.703406  8921 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.706198  2683 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:13.708134  2677 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:13.712311  3538 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:13.605023  8866 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:13.721997  3542 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:13.722205  3547 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 14:54:13.724294  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.721448  8845 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:13.730325  8845 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:13.730969  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.731964  7977 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:13.732486  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.733600  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.693037  8978 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:13.736739  2682 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:13.739245  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.741017  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.711510  8871 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:13.732988  8845 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:13.736799  8978 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:13.751375  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.755928  2682 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 14:54:13.759204  3540 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:13.760226  2683 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 14:54:13.654484  8878 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:13.776726  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65782.1us
I20260709 14:54:13.778079  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:13.787472  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1681) took 1467 ms (client timeout 1509 ms). Trace:
W20260709 14:54:13.788205  2612 rpcz_store.cc:269] 0709 14:54:12.131348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.417788 (+286440us) service_pool.cc:224] Handling call
0709 14:54:13.598300 (+1180512us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54001eb754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:13.598684 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694782336}
I20260709 14:54:13.791893  2682 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:13.813122  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:13.761267  8978 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:13.819087  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:13.821298  3541 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:13.824928  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.233s (SENT)
I20260709 14:54:13.826872  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:13.829402  3546 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:54:13.840777  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:13.855068  2756 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:13.834498  8866 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:13.834255  8803 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:13.773604  8871 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:13.873312  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:13.863130  8994 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 14:54:13.896420  8702 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:13.896782  8965 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:13.901340  8885 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:13.903151  8910 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:13.903774  8910 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:13.905467  8910 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:13.883832  8703 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 14:54:13.907801  8703 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:13.908051  8910 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:13.910308  8978 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:13.894554  3543 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 13.
I20260709 14:54:13.915704  2683 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
W20260709 14:54:13.928350  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.925161  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.914778  3547 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:13.908252  8703 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:13.934383  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:13.932063  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:13.949324  2685 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:13.916163  8978 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:13.954780  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 201082us
I20260709 14:54:13.970773  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:13.973744  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:13.974658  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.985677  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:13.976485  8702 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:13.997205  3540 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:13.977855  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.983778  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:13.989711  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:14.006541  2603 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:14.014598  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1616) took 3040 ms. Trace:
I20260709 14:54:14.015635  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.016510  2610 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:14.044467  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 237256us
I20260709 14:54:14.045881  2685 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 14:54:14.050952  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.021s	sys 0.019s
W20260709 14:54:14.075977  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.022s	sys 0.019s
I20260709 14:54:14.084394  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:14.086845  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:14.015264  2684 rpcz_store.cc:276] 0709 14:54:10.973730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:11.166604 (+192874us) service_pool.cc:224] Handling call
0709 14:54:11.166901 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:11.193423 (+ 26522us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:11.211567 (+ 18144us) log.cc:844] Serialized 14 byte log entry
0709 14:54:12.877012 (+1665445us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18001998a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:54:13.393499 (+516487us) log.cc:844] Serialized 14 byte log entry
0709 14:54:13.518241 (+124742us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:13.567652 (+ 49411us) log.cc:844] Serialized 26 byte log entry
0709 14:54:13.567882 (+   230us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:13.567925 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.568066 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.014041 (+445975us) raft_consensus.cc:1685] finished
0709 14:54:14.014078 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.014311 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1611493,"prepare.queue_time_us":413999,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":6816,"raft.queue_time_us":1597576,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":46031,"spinlock_wait_cycles":13031680,"thread_start_us":612087,"threads_started":4,"wal-append.queue_time_us":530059,"wal-append.run_cpu_time_us":3273,"wal-append.run_wall_time_us":1720899}
I20260709 14:54:14.094975  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:14.048243  8965 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:14.097110  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:13.992835  8894 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.109089  2610 raft_consensus.cc:1275] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 14:54:14.120503  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.110569  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 510384us
I20260709 14:54:14.141408  2677 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 14:54:14.149753  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.011s	sys 0.034s
W20260709 14:54:14.168042  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.012s	sys 0.034s
W20260709 14:54:14.152032  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 269737us
W20260709 14:54:14.159133  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:14.151055  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:14.210925  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71550.7us
I20260709 14:54:14.208294  2612 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 11 FOLLOWER]: Advancing to term 13
I20260709 14:54:14.269076  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:14.248148  8884 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.005s
W20260709 14:54:14.276841  8984 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.863s	user 0.008s	sys 0.000s
I20260709 14:54:14.279002  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1606) took 1009 ms. Trace:
I20260709 14:54:14.279601  2680 rpcz_store.cc:276] 0709 14:54:13.269142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.322624 (+ 53482us) service_pool.cc:224] Handling call
0709 14:54:14.278738 (+956114us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:14.278902 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550508928}
I20260709 14:54:14.143687  8703 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:14.282195  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:14.283097  8703 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:14.283398  3543 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:14.284600  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:14.285189  8537 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:14.285983  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:14.291157  3543 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:14.288122  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:14.298945  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 468408us
I20260709 14:54:14.299371  3538 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:14.301436  3538 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 13. Current term is 14. Ops: [11.5-13.6]
I20260709 14:54:14.289598  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1107) took 1853 ms. Trace:
I20260709 14:54:14.289824  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:14.277027  8989 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.898s	user 0.009s	sys 0.004s
W20260709 14:54:14.268360  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.026s	sys 0.038s
W20260709 14:54:14.304503  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.029s	sys 0.038s
I20260709 14:54:14.295010  8871 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:14.295773  8814 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:14.118340  8965 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:14.286000  8537 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:14.314775  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:14.220082  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.010s	sys 0.049s
W20260709 14:54:14.318008  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59359.9us
I20260709 14:54:14.327915  3547 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:14.325726  3540 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:14.303925  2608 rpcz_store.cc:276] 0709 14:54:12.435920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.813981 (+378061us) service_pool.cc:224] Handling call
0709 14:54:12.862399 (+ 48418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:12.862883 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:12.862916 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:12.868280 (+  5364us) log.cc:844] Serialized 26 byte log entry
0709 14:54:13.446279 (+577999us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:13.446410 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.446557 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.275769 (+829212us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:14.289069 (+ 13300us) raft_consensus.cc:1685] finished
0709 14:54:14.289097 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.289369 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":924,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13336,"spinlock_wait_cycles":9982208,"thread_start_us":499016,"threads_started":1,"wal-append.queue_time_us":522667}
W20260709 14:54:14.325810  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.011s	sys 0.050s
I20260709 14:54:14.321671  8871 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:54:14.350256  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102339us
W20260709 14:54:14.339751  2759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.011s	sys 0.038s
W20260709 14:54:14.333922  8578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.017s	sys 0.044s
W20260709 14:54:14.358072  2759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.012s	sys 0.039s
I20260709 14:54:14.309223  8884 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.368523  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1779) took 1514 ms. Trace:
W20260709 14:54:14.360955  8578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.018s	sys 0.044s
I20260709 14:54:14.369189  2759 rpcz_store.cc:276] 0709 14:54:12.853607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.418230 (+564623us) service_pool.cc:224] Handling call
0709 14:54:13.449091 (+ 30861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:13.465044 (+ 15953us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 14:54:13.465670 (+   626us) log.cc:844] Serialized 14 byte log entry
0709 14:54:14.361444 (+895774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:14.361532 (+    88us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:14.368118 (+  6586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.368239 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.368440 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1215,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"thread_start_us":885810,"threads_started":2,"wal-append.queue_time_us":460956}
I20260709 14:54:14.383662  8894 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:14.389263  8965 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:14.393582  8702 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:14.396298  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1607) took 1126 ms. Trace:
I20260709 14:54:14.396495  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1599) took 1249 ms. Trace:
I20260709 14:54:14.397001  2681 rpcz_store.cc:276] 0709 14:54:13.269629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.358786 (+ 89157us) service_pool.cc:224] Handling call
0709 14:54:14.396034 (+1037248us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b54000c9ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:14.396186 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64182144}
I20260709 14:54:14.339557  8814 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:14.402783  3538 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 14:54:14.403425  7977 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:14.415498  2753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.012s	sys 0.036s
W20260709 14:54:14.416244  2753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.012s	sys 0.036s
I20260709 14:54:14.374130  8537 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:14.426491  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:14.417337  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1778) took 1564 ms. Trace:
W20260709 14:54:14.441454  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 72573.8us
W20260709 14:54:14.444835  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 787375us
I20260709 14:54:14.408118  8871 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:14.453701  3537 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:14.456421  7977 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:14.397953  2679 rpcz_store.cc:276] 0709 14:54:13.147322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.155987 (+  8665us) service_pool.cc:224] Handling call
0709 14:54:13.156807 (+   820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:13.157938 (+  1131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:13.157987 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:13.190073 (+ 32086us) log.cc:844] Serialized 26 byte log entry
0709 14:54:13.553110 (+363037us) spinlock_profiling.cc:243] Waited 943 us on lock 0x7b1800394f60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1
0709 14:54:13.559202 (+  6092us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:13.559734 (+   532us) log.cc:844] Serialized 14 byte log entry
0709 14:54:13.560036 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.560131 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.276144 (+716013us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:14.396064 (+119920us) raft_consensus.cc:1685] finished
0709 14:54:14.396091 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.396260 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36219,"prepare.queue_time_us":487,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":84,"raft.queue_time_us":67144,"raft.run_cpu_time_us":543,"raft.run_wall_time_us":24431,"spinlock_wait_cycles":42430208,"thread_start_us":76095,"threads_started":1,"wal-append.queue_time_us":187579}
W20260709 14:54:14.450452  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.030s	sys 0.018s
W20260709 14:54:14.463997  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.030s	sys 0.018s
I20260709 14:54:14.464192  2603 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:14.464524  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 645438us
W20260709 14:54:14.458313  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.031s	sys 0.033s
W20260709 14:54:14.469208  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.031s	sys 0.034s
I20260709 14:54:14.481127  2612 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:54:14.456301  8713 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:14.465639  8847 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.776s	user 0.016s	sys 0.000s
I20260709 14:54:14.439832  2753 rpcz_store.cc:276] 0709 14:54:12.853345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.416717 (+563372us) service_pool.cc:224] Handling call
0709 14:54:13.471269 (+ 54552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:13.471840 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:13.472324 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 14:54:14.416736 (+944412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:14.416835 (+    99us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:14.416917 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.417030 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.417252 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13901,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":644424,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":926240,"threads_started":2,"wal-append.queue_time_us":503951}
I20260709 14:54:14.455493  8814 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:14.499375  8652 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.005s
I20260709 14:54:14.500018  8652 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:14.501482  8814 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:14.502499  8713 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:14.490506  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.016s	sys 0.038s
W20260709 14:54:14.504382  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.016s	sys 0.038s
W20260709 14:54:14.507393  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.015s	sys 0.042s
W20260709 14:54:14.508080  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.016s	sys 0.042s
I20260709 14:54:14.513559  2611 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:14.504866  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 665531us
W20260709 14:54:14.515645  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.000s	sys 0.065s
W20260709 14:54:14.516389  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.000s	sys 0.065s
W20260709 14:54:14.516808  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 766374us
I20260709 14:54:14.517763  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1137) took 1987 ms. Trace:
I20260709 14:54:14.502701  8652 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:14.517983  3536 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.007s	sys 0.049s
W20260709 14:54:14.528388  3536 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.008s	sys 0.049s
W20260709 14:54:14.568216  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44595.4us
W20260709 14:54:14.529594  8708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.025s	sys 0.045s
W20260709 14:54:14.578106  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.603538  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104658us
I20260709 14:54:14.611436  8965 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:14.565075  8910 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.009s
I20260709 14:54:14.618175  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1143) took 1181 ms. Trace:
I20260709 14:54:14.612795  8908 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:14.663621  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:14.662158  2683 rpcz_store.cc:276] 0709 14:54:13.446896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.761318 (+314422us) service_pool.cc:224] Handling call
0709 14:54:13.780227 (+ 18909us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:13.928567 (+148340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.614714 (+686147us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400445054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:14.618060 (+  3346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310620544}
W20260709 14:54:14.615115  8708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.025s	sys 0.046s
I20260709 14:54:14.522426  2751 rpcz_store.cc:276] 0709 14:54:12.361869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.550199 (+188330us) service_pool.cc:224] Handling call
0709 14:54:12.550642 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:12.551200 (+   558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:12.551233 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:12.551798 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 14:54:13.383539 (+831741us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b1800204760. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:13.667807 (+284268us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:14.321399 (+653592us) spinlock_profiling.cc:243] Waited 86.6 ms on lock 0x7b54003122a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c1733 00007fa16e646dbf 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c
0709 14:54:14.321503 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.335872 (+ 14369us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.335953 (+    81us) raft_consensus.cc:1685] finished
0709 14:54:14.335981 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.349623 (+ 13642us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2269,"raft.queue_time_us":4248,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":10790,"spinlock_wait_cycles":202745472,"thread_start_us":47784,"threads_started":1,"wal-append.queue_time_us":1004927}
I20260709 14:54:14.678767  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:14.681744  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 155540us
I20260709 14:54:14.686333  8965 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:14.687877  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1336) took 1114 ms. Trace:
I20260709 14:54:14.688331  3537 rpcz_store.cc:276] 0709 14:54:13.573447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.606176 (+ 32729us) service_pool.cc:224] Handling call
0709 14:54:13.606717 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:14.465616 (+858899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.687527 (+221911us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54004b23d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:14.687784 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872720128}
I20260709 14:54:14.689534  8965 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:14.690786  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1692) took 1237 ms. Trace:
I20260709 14:54:14.691318  2611 rpcz_store.cc:276] 0709 14:54:13.453084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.463022 (+  9938us) service_pool.cc:224] Handling call
0709 14:54:14.690538 (+1227516us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400371c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:14.690713 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389685120}
I20260709 14:54:14.665995  8652 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 14:54:14.694335  8894 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:14.677187  8910 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:14.721091  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:14.724175  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 378501us
I20260709 14:54:14.700719  8894 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:14.727833  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.020s	sys 0.056s
W20260709 14:54:14.736027  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.020s	sys 0.057s
I20260709 14:54:14.705905  8884 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:14.749794  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51631.1us
W20260709 14:54:14.724973  8921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.023s	sys 0.043s
W20260709 14:54:14.758420  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.012s	sys 0.036s
W20260709 14:54:14.759836  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.013s	sys 0.036s
I20260709 14:54:14.762020  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:14.722038  8910 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:14.787074  9010 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:14.787875  9010 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:14.788403  9010 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:14.788775  8910 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:14.779379  9000 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 14:54:14.792043  8910 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:14.793061  8910 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:14.794504  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:14.796267  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:14.796533  8910 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:14.798291  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:14.768482  8894 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:14.772763  9003 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:14.683483  8871 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:14.773442  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:14.773936  9004 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:14.775611  9006 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:14.776624  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:14.847856  3537 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:14.777031  8713 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:14.777395  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1138) took 1484 ms. Trace:
I20260709 14:54:14.778177  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1650) took 3432 ms. Trace:
I20260709 14:54:14.779336  8803 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:14.781792  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:14.781090  2431 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 10 to 14, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:14.750414  8578 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 14:54:14.809167  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.858698  7977 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:54:14.802533  3543 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 14:54:14.858662  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.017s
W20260709 14:54:14.859915  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.008s	sys 0.017s
I20260709 14:54:14.860015  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1689) took 1103 ms. Trace:
I20260709 14:54:14.860517  3543 rpcz_store.cc:276] 0709 14:54:13.756803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:14.285615 (+528812us) service_pool.cc:224] Handling call
0709 14:54:14.859923 (+574308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:14.811028  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.807566  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.813371  2760 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 14:54:14.861478  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.812206  2754 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:14.810141  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.803609  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:14.819213  9008 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:14.800796  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 233491us
W20260709 14:54:14.807739  2435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 167) took 34218 ms (client timeout 15000 ms). Trace:
W20260709 14:54:14.764247  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.016s	sys 0.041s
W20260709 14:54:14.865329  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.017s	sys 0.042s
W20260709 14:54:14.760715  8921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.023s	sys 0.044s
I20260709 14:54:14.866652  2612 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:14.816216  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:14.845309  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.851009  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52872.7us
W20260709 14:54:14.855971  2755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.004s	sys 0.020s
W20260709 14:54:14.847745  8866 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.976s	user 0.013s	sys 0.005s
I20260709 14:54:14.859248  7977 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:14.876644  7977 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:14.861941  2603 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:14.876775  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:14.851967  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.011s	sys 0.014s
I20260709 14:54:14.859127  2759 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:14.878463  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.015s
I20260709 14:54:14.848735  9006 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:14.868736  2755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.004s	sys 0.020s
W20260709 14:54:14.873814  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.879432  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.862785  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.011s	sys 0.014s
I20260709 14:54:14.877496  2610 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:14.880107  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.011s	sys 0.014s
I20260709 14:54:14.870787  2752 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:14.849938  2686 rpcz_store.cc:276] 0709 14:54:13.292844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.705011 (+412167us) service_pool.cc:224] Handling call
0709 14:54:13.705472 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:13.705823 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:13.706226 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 14:54:14.765613 (+1059387us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:14.765715 (+   102us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:14.765806 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.765935 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.777307 (+ 11372us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":923,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":5262,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":1053634,"threads_started":1,"wal-append.queue_time_us":1067477}
W20260709 14:54:14.880605  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 621360us
I20260709 14:54:14.881778  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1776) took 2029 ms. Trace:
W20260709 14:54:14.879170  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 577149us
I20260709 14:54:14.886277  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:14.865072  8878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.037s	sys 0.045s
I20260709 14:54:14.878736  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.889091  2611 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:14.852420  8803 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:14.862460  8965 raft_consensus.cc:2764] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 14:54:14.890980  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.883731  2610 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:14.899622  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:14.890193  8878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.037s	sys 0.045s
W20260709 14:54:14.904506  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 131225us
I20260709 14:54:14.902352  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:14.851526  2757 rpcz_store.cc:276] 0709 14:54:11.346130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:12.427853 (+1081723us) service_pool.cc:224] Handling call
0709 14:54:13.359859 (+932006us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:13.390871 (+ 31012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:13.390928 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:13.441759 (+ 50831us) log.cc:844] Serialized 52 byte log entry
0709 14:54:13.883101 (+441342us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:13.883623 (+   522us) log.cc:844] Serialized 14 byte log entry
0709 14:54:13.884045 (+   422us) log.cc:844] Serialized 14 byte log entry
0709 14:54:13.884220 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:13.892729 (+  8509us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.346112 (+453383us) raft_consensus.cc:1685] finished
0709 14:54:14.346150 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.777815 (+431665us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b5400184cd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:14.777986 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18566,"prepare.queue_time_us":7267,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":134,"raft.queue_time_us":1108861,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":26844,"spinlock_wait_cycles":2132244352,"thread_start_us":428605,"threads_started":1,"wal-append.queue_time_us":795611}
I20260709 14:54:14.907313  2606 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:54:14.910542  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:14.907899  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 91846.9us
I20260709 14:54:14.912979  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.914142  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:14.886009  9006 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:14.885900  2755 rpcz_store.cc:276] 0709 14:54:12.852770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.353335 (+500565us) service_pool.cc:224] Handling call
0709 14:54:13.354143 (+   808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:13.367374 (+ 13231us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:13.437413 (+ 70039us) log.cc:844] Serialized 14 byte log entry
0709 14:54:14.881244 (+1443831us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:14.881331 (+    87us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:14.881427 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.881526 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.881697 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50015,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":287378,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":1327974,"threads_started":2,"wal-append.queue_time_us":1073502}
I20260709 14:54:14.913863  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.887226  9008 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:14.921053  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.014s	sys 0.014s
W20260709 14:54:14.981928  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.014s	sys 0.014s
I20260709 14:54:14.917599  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.920185  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:14.926074  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.864619  9003 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.927820  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:14.904506  8885 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.021s	sys 0.035s
I20260709 14:54:14.884941  9004 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.937803  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:14.986146  2679 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:14.986300  2760 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:14.987709  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:14.938969  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.004s	sys 0.012s
I20260709 14:54:14.988685  2756 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:14.989071  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.005s	sys 0.012s
I20260709 14:54:14.939273  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:14.939437  8565 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:14.940366  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:14.941426  8884 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:14.942201  8999 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.327s	user 0.001s	sys 0.002s
I20260709 14:54:14.994910  8702 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:14.941506  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:14.995559  9003 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:14.995488  8702 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.996112  2683 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:14.996233  9003 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:14.996431  9010 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:14.996856  9003 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.988502  8878 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 14:54:14.998234  8702 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:14.942710  9011 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:14.943463  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:14.943357  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:14.939697  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:14.944362  8803 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:14.945387  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:15.001665  2685 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:14.945421  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1147) took 1474 ms. Trace:
W20260709 14:54:14.984822  8885 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.021s	sys 0.035s
W20260709 14:54:14.951261  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61092.6us
I20260709 14:54:14.953109  9027 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:14.954342  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:14.955278  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:15.015995  2752 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:14.908783  8894 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:14.957185  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:15.017359  2677 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:14.922737  8910 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:14.964980  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:14.929955  2431 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 6 to 14, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:54:14.959247  8921 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:14.967856  8713 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.026098  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:15.027051  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:15.020231  2605 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:14.979984  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:14.964105  9016 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:14.980120  2611 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:14.990356  2680 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:14.982266  2757 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:14.993993  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:14.980031  8578 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:14.939289  2681 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 14:54:14.973968  8957 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.004s	sys 0.025s
I20260709 14:54:15.004853  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1140) took 2544 ms. Trace:
I20260709 14:54:14.964265  8845 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 14:54:15.006247  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.019s	sys 0.024s
W20260709 14:54:15.092612  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.020s	sys 0.024s
I20260709 14:54:15.007563  2751 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:15.010890  8847 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.011667  8908 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.012235  8878 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:15.093375  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52500.4us
I20260709 14:54:15.002200  2682 rpcz_store.cc:276] 0709 14:54:13.471346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.793078 (+321732us) service_pool.cc:224] Handling call
0709 14:54:13.793518 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:13.802157 (+  8639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:13.802198 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:13.806883 (+  4685us) log.cc:844] Serialized 26 byte log entry
0709 14:54:14.508820 (+701937us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:14.508870 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.508951 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.945104 (+436153us) raft_consensus.cc:1685] finished
0709 14:54:14.945130 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:14.945345 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7739,"raft.queue_time_us":7903,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":690332,"threads_started":1,"wal-append.queue_time_us":1031398}
I20260709 14:54:15.018980  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:15.094485  3545 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.007s	sys 0.058s
I20260709 14:54:15.004419  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:15.016815  2678 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:15.025806  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:15.033572  2679 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:15.095391  2679 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:15.033730  8870 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.022s	sys 0.018s
W20260709 14:54:15.037189  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 82948.1us
I20260709 14:54:15.082370  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:15.083567  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:15.086370  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:15.091571  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:15.094870  3545 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.007s	sys 0.059s
I20260709 14:54:15.099107  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:15.078168  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1347) took 1468 ms. Trace:
W20260709 14:54:15.088209  8957 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.004s	sys 0.026s
I20260709 14:54:15.110508  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1111) took 1151 ms. Trace:
I20260709 14:54:15.119925  2677 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:15.025775  8921 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:15.083247  8702 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:15.088354  2758 rpcz_store.cc:276] 0709 14:54:12.460938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.274014 (+813076us) service_pool.cc:224] Handling call
0709 14:54:13.274922 (+   908us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:13.276001 (+  1079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:13.278156 (+  2155us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:13.386480 (+108324us) log.cc:844] Serialized 52 byte log entry
0709 14:54:14.076637 (+690157us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:14.076697 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:14.076780 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:14.807990 (+731210us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:14.998865 (+190875us) raft_consensus.cc:1685] finished
0709 14:54:14.998893 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:15.004760 (+  5867us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37483,"raft.queue_time_us":2999,"raft.run_cpu_time_us":638,"raft.run_wall_time_us":50529,"spinlock_wait_cycles":8038144,"thread_start_us":643144,"threads_started":1,"wal-append.queue_time_us":1070342}
I20260709 14:54:15.128510  2576 leader_election.cc:400] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
W20260709 14:54:15.105095  8870 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.023s	sys 0.018s
W20260709 14:54:14.865029  2435 rpcz_store.cc:269] 0709 14:53:40.589049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:40.591167 (+  2118us) service_pool.cc:224] Handling call
0709 14:54:14.806723 (+34215556us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:53:59.356222 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:53:59.357130 (+   908us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:53:59.357200 (+    70us) write_op.cc:435] Acquired schema lock
0709 14:53:59.361058 (+  3858us) tablet.cc:662] Decoding operations
0709 14:53:59.462649 (+101591us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:53:59.462804 (+   155us) write_op.cc:641] Partition lock acquired for write op
0709 14:53:59.462873 (+    69us) tablet.cc:685] Acquiring locks for 26 operations
0709 14:53:59.712970 (+250097us) tablet.cc:701] Row locks acquired
0709 14:53:59.737215 (+ 24245us) write_op.cc:260] PREPARE: finished
0709 14:53:59.770524 (+ 33309us) write_op.cc:270] Start()
0709 14:53:59.822026 (+ 51502us) write_op.cc:276] Timestamp: P: 1783608839753608 usec, L: 0
0709 14:53:59.842050 (+ 20024us) op_driver.cc:348] REPLICATION: starting
0709 14:54:00.097026 (+254976us) log.cc:844] Serialized 10233 byte log entry
0709 14:54:02.351992 (+2254966us) op_driver.cc:464] REPLICATION: finished
0709 14:54:02.492446 (+140454us) write_op.cc:301] APPLY: starting
0709 14:54:02.492608 (+   162us) tablet.cc:1367] starting BulkCheckPresence
0709 14:54:02.875789 (+383181us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b18003632e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e3850ec 00007fa16e3d1c0a 00007fa16e3d14f7 00007fa16e3d1490 00007fa16e388f48 00007fa17209d1bb 00007fa17209d532 00007fa1720e4b93 00007fa171f91de3
0709 14:54:03.362999 (+487210us) tablet.cc:1370] finished BulkCheckPresence
0709 14:54:03.363050 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:54:04.401616 (+1038566us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:54:04.402152 (+   536us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 14:54:04.402227 (+    75us) write_op.cc:312] APPLY: finished
0709 14:54:04.623214 (+220987us) log.cc:844] Serialized 227 byte log entry
0709 14:54:05.619419 (+996205us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:05.619988 (+   569us) write_op.cc:454] Released schema lock
0709 14:54:05.620106 (+   118us) write_op.cc:341] FINISH: Updating metrics
0709 14:54:05.634879 (+ 14773us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:05.652601 (+ 17722us) write_op.cc:454] Released schema lock
0709 14:54:06.391019 (+738418us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
Metrics: {"mutex_wait_us":2109721,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":135422,"apply.run_cpu_time_us":79094,"apply.run_wall_time_us":3128343,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112755,"num_ops":26,"prepare.queue_time_us":324648,"prepare.run_cpu_time_us":118712,"prepare.run_wall_time_us":1255403,"raft.queue_time_us":771482,"raft.run_cpu_time_us":9877,"raft.run_wall_time_us":459208,"replication_time_us":2476877,"spinlock_wait_cycles":32747264,"thread_start_us":3824578,"threads_started":5,"wal-append.queue_time_us":1847663,"wal-append.run_cpu_time_us":67191,"wal-append.run_wall_time_us":5161893}]]}
I20260709 14:54:15.022172  8884 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:15.131042  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44774.9us
W20260709 14:54:15.154729  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:15.129106  2576 leader_election.cc:403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:15.132189  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:15.133301  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:15.095870  9003 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:15.140583  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 222613us
I20260709 14:54:15.148404  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:15.156072  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:15.161899  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 140972us
I20260709 14:54:15.038309  8713 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:15.163161  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 203152us
I20260709 14:54:15.164719  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:14.994474  9008 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:15.118991  2607 rpcz_store.cc:276] 0709 14:54:13.959257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.965212 (+  5955us) service_pool.cc:224] Handling call
0709 14:54:13.996861 (+ 31649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:14.009724 (+ 12863us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:14.010127 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 14:54:15.110038 (+1099911us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:15.110098 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:15.110163 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:15.110250 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:15.110429 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16854,"prepare.queue_time_us":5161,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":17869,"thread_start_us":1082557,"threads_started":1}
I20260709 14:54:15.151780  2679 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:15.183882  2681 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:15.183423  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 203937us
W20260709 14:54:15.152511  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 175650us
I20260709 14:54:15.041257  8910 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:15.185840  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:15.190073  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1335) took 1617 ms. Trace:
I20260709 14:54:15.182220  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:15.129024  9010 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:15.108101  8847 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:15.106786  3536 rpcz_store.cc:276] 0709 14:54:13.609943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.649856 (+ 39913us) service_pool.cc:224] Handling call
0709 14:54:13.650344 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:13.691097 (+ 40753us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:14.537374 (+846277us) log.cc:844] Serialized 14 byte log entry
0709 14:54:15.037592 (+500218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:15.037681 (+    89us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:15.037748 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:15.037848 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:15.078046 (+ 40198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1107766,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"thread_start_us":1185424,"threads_started":2}
I20260709 14:54:15.151780  8866 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:15.202512  2679 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:15.208657  2686 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:54:15.137247  8921 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:15.201406  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71930.7us
I20260709 14:54:15.197983  2431 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 3 to 12. New cstate: current_term: 12 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:54:15.122741  8908 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:15.223299  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67390us
I20260709 14:54:15.223765  2755 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:15.228663  7977 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:15.192454  8814 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:15.122864  8878 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:15.196163  9003 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:15.200371  8871 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 14:54:15.241608  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:15.111950  9016 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 14:54:15.205684  8947 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 12. Current term is 13.
W20260709 14:54:15.256358  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 173406us
I20260709 14:54:15.257278  2680 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:15.258992  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76872.8us
I20260709 14:54:15.209602  8866 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:15.210424  8803 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:15.238068  8702 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:15.282506  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:15.282905  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:15.250046  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85478.9us
I20260709 14:54:15.254813  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:15.248080  7977 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:15.249571  2678 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 14:54:15.260528  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46710.2us
I20260709 14:54:15.284195  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:15.287958  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:15.232395  9025 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 14:54:15.293442  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90079.5us
I20260709 14:54:15.193943  3545 rpcz_store.cc:276] 0709 14:54:13.572966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.575121 (+  2155us) service_pool.cc:224] Handling call
0709 14:54:13.596721 (+ 21600us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:13.654083 (+ 57362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:13.654135 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:13.655349 (+  1214us) log.cc:844] Serialized 78 byte log entry
0709 14:54:14.469916 (+814567us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:15.101687 (+631771us) log.cc:844] Serialized 14 byte log entry
0709 14:54:15.189559 (+ 87872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:15.189671 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:15.189742 (+    71us) raft_consensus.cc:1685] finished
0709 14:54:15.189759 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:15.189991 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45019,"prepare.queue_time_us":329680,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":57590,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":192,"thread_start_us":1515280,"threads_started":3,"wal-append.queue_time_us":1101075}
I20260709 14:54:15.209580  9019 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.242628  9027 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:15.310655  3536 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:15.286870  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:15.269052  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56983.7us
I20260709 14:54:15.295418  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:15.295634  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:15.296264  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:15.258438  8708 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:15.297268  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55116.1us
I20260709 14:54:15.280170  8866 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:15.313655  3542 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:15.313655  3546 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:54:15.317744  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63043.4us
I20260709 14:54:15.314702  3538 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:15.255996  8578 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:15.319190  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:15.328557  3540 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 14:54:15.319329  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 362308us
I20260709 14:54:15.279438  8652 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:15.289852  8702 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:15.321631  2431 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 11 to 14, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 14 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:54:15.373627  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56896.6us
W20260709 14:54:15.375174  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 122823us
W20260709 14:54:15.395895  9001 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 14:54:15.571192  2726 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:15.571859  2726 leader_election.cc:403] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:54:15.422890  9015 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 14:54:15.422969  9017 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
W20260709 14:54:15.429522  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:15.403151  9004 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:15.332063  8965 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:15.397277  9010 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 14:54:15.554201  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 200150us
I20260709 14:54:15.335476  8708 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:15.327479  9019 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:15.357936  7977 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:15.327450  9024 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:15.344841  8702 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:15.427913  9042 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 14:54:15.317999  9003 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:15.337829  8814 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.420261  9021 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:15.338371  8652 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:15.420498  9043 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:15.393414  9012 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:15.328259  9016 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:54:15.561708  3540 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:54:15.339151  8565 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:15.569187  9027 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 14:54:15.574348  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:15.594190  2683 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.597s	user 0.013s	sys 0.001s
W20260709 14:54:15.571852  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 236874us
W20260709 14:54:15.572517  9006 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.001s
W20260709 14:54:15.573660  2752 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.000s
I20260709 14:54:15.579035  2679 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
W20260709 14:54:15.600551  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:15.610616  7977 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:15.582127  8894 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.565s	user 0.016s	sys 0.000s
W20260709 14:54:15.627090  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 302020us
I20260709 14:54:15.620069  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:15.629998  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:15.572888  8871 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:15.597375  8965 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:15.602485  9004 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.738291  2752 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 14:54:15.733466  2683 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:15.625944  8537 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:15.732890  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:15.793255  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 509121us
I20260709 14:54:15.572331  9031 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 14:54:15.811496  7977 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:15.794364  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:15.800038  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:15.807730  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:15.644507  9035 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.311s	user 0.000s	sys 0.010s
I20260709 14:54:15.814754  2611 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:54:15.644232  8884 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:15.801198  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 484782us
I20260709 14:54:15.786492  8965 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:15.731957  8565 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:15.734721  9012 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:15.867215  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 247230us
W20260709 14:54:15.790789  8703 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 14:54:15.859423  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 266821us
I20260709 14:54:15.792985  8537 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:15.791805  9025 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:15.880913  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86642.5us
I20260709 14:54:15.638064  8814 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:15.816926  9031 consensus_queue.cc:1243] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:15.791394  9016 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.955646  9016 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:15.893674  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 617277us
I20260709 14:54:15.955678  8652 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:15.819135  8910 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:15.880196  8565 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:15.802644  9004 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:15.959331  9053 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 14:54:15.960150  9060 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:15.960709  9004 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:15.786564  9043 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:15.961004  9060 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:15.962235  9003 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:15.815605  8578 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:15.945405  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60344.9us
I20260709 14:54:15.785553  9006 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:15.791805  8866 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:15.942909  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:15.866110  8713 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:15.952610  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 350820us
I20260709 14:54:15.965987  2684 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:15.948913  9047 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:15.807638  8894 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:15.892331  8537 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:15.971392  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 178151us
W20260709 14:54:15.956542  9040 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 14:54:15.977773  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:15.878218  9012 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:15.928654  9010 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:15.740494  9021 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:15.943439  8978 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:15.963685  9032 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260709 14:54:16.019309  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:15.967339  8921 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.726s	user 0.016s	sys 0.000s
I20260709 14:54:15.950974  8878 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:16.170853  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103108us
I20260709 14:54:15.962742  9056 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:15.978472  8908 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:16.075657  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 342861us
I20260709 14:54:15.967988  9022 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:54:15.999539  9047 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:15.964814  9019 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:16.068907  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 498277us
W20260709 14:54:16.173679  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 97002.4us
W20260709 14:54:16.179075  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 250529us
W20260709 14:54:16.180703  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:16.182410  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:16.013190  8870 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:15.992518  8565 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:16.010257  9004 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20.
I20260709 14:54:16.211213  3540 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:16.223906  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:16.225908  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47882.2us
I20260709 14:54:16.042217  8978 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:16.249044  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 214568us
I20260709 14:54:16.224862  3537 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:16.263016  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:16.179000  8713 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:16.006615  8894 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:15.965943  9049 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.252s
I20260709 14:54:16.267371  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:16.203696  8910 raft_consensus.cc:2764] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 14:54:16.218730  8803 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.927s	user 0.016s	sys 0.000s
I20260709 14:54:16.270056  2605 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:54:16.217054  9047 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:16.289700  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 270450us
W20260709 14:54:16.291776  3543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.001s	sys 0.047s
I20260709 14:54:16.246384  8878 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:16.257366  9019 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:16.175401  8921 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:16.277027  2723 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.983s (SENT)
W20260709 14:54:16.399461  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 129734us
W20260709 14:54:16.294335  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 355591us
W20260709 14:54:16.297394  3543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.002s	sys 0.047s
W20260709 14:54:16.398480  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123911us
I20260709 14:54:16.297571  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:16.386466  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.136s (SENT) [suppressed 25 similar messages]
W20260709 14:54:16.405189  3412 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:39575, user_credentials={real_user=slave}} blocked reactor thread for 72342.9us
I20260709 14:54:16.418926  8978 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:16.454384  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 156914us
I20260709 14:54:16.473086  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:16.473574  9050 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:16.474113  9050 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:16.474738  9050 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:16.474279  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52603.9us
I20260709 14:54:16.476941  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:16.308097  9019 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:16.475992  8910 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:16.457139  9054 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.176582  8871 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:16.505340  9073 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:16.512183  9045 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:16.512681  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 256701us
I20260709 14:54:16.198014  9056 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:16.497835  8978 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:16.524289  3581 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:39575 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:39575 timed out after 15.000s (SENT)
I20260709 14:54:16.525017  9067 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:16.503952  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:16.296155  8713 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:16.532903  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56069.9us
W20260709 14:54:16.533857  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 113119us
I20260709 14:54:16.526623  9019 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:16.560132  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:16.591831  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:16.576825  2723 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.509s (SENT)
I20260709 14:54:16.587348  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:16.553624  9045 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:16.479909  8708 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:16.556054  9054 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:16.640252  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.014s	sys 0.009s
W20260709 14:54:16.674803  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.017s	sys 0.017s
W20260709 14:54:16.675310  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 874287us
I20260709 14:54:16.676105  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:16.676905  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.011s	sys 0.024s
W20260709 14:54:16.593647  9033 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.730s	user 0.001s	sys 0.001s
I20260709 14:54:16.620110  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:16.677655  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.011s	sys 0.024s
I20260709 14:54:16.687263  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:16.687662  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:16.612738  9052 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.637s	user 0.000s	sys 0.001s
W20260709 14:54:16.595176  9008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.000s	sys 0.008s
W20260709 14:54:16.670418  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93656.9us
I20260709 14:54:16.688889  2679 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:16.688736  2683 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:16.689908  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:16.691452  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 187640us
I20260709 14:54:16.627768  8565 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:16.706421  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:16.706756  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 119505us
I20260709 14:54:16.691741  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:16.588955  9067 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:16.629245  8814 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:16.620815  9028 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:16.710968  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:16.675604  8910 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.727370  9006 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:16.731220  8910 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:16.733047  9064 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 14:54:16.734197  8910 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:16.735339  9061 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.676230  8866 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:16.742955  9046 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 14:54:16.743813  8921 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:16.719712  9067 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:16.746925  9043 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:16.750990  8803 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:54:16.752117  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:16.753994  8921 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:16.755237  8921 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:16.606760  9045 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:16.711376  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:16.756661  2751 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:16.764936  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1665) took 1921 ms. Trace:
W20260709 14:54:16.768939  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77243.4us
I20260709 14:54:16.611438  9071 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:16.715556  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:16.716004  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:16.715636  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:16.787096  2680 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:16.629959  9016 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:16.738657  8713 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:16.791388  9064 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:16.792831  9064 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:16.793407  9064 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:16.794201  9021 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:16.718878  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:16.795229  2755 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:16.719000  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:16.785046  9028 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:16.796825  2677 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:16.677112  9056 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:16.704394  9008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.000s	sys 0.009s
I20260709 14:54:16.720650  2678 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:54:16.721051  9074 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.798442  9074 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:16.799728  2752 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:16.801292  9074 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:16.727331  8894 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:16.736033  9061 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:16.777992  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:16.804215  9061 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:16.783576  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 826080us
I20260709 14:54:16.805308  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:16.807181  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 187285us
I20260709 14:54:16.808763  9061 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:16.808604  9073 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:16.765801  2759 rpcz_store.cc:276] 0709 14:54:14.843557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:14.990644 (+147087us) service_pool.cc:224] Handling call
0709 14:54:15.031287 (+ 40643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:15.736843 (+705556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:15.736887 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:15.736956 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:15.737012 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:16.764508 (+1027496us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:16.764592 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:16.764829 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":361469,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":49810,"spinlock_wait_cycles":1410872704}
W20260709 14:54:16.812075  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:16.769973  9054 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:16.810542  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103200us
I20260709 14:54:16.817649  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1129) took 1064 ms. Trace:
I20260709 14:54:16.819912  2607 rpcz_store.cc:276] 0709 14:54:15.745673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:15.745814 (+   141us) service_pool.cc:224] Handling call
0709 14:54:15.807984 (+ 62170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:16.078265 (+270281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:16.078319 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:16.189628 (+111309us) log.cc:844] Serialized 26 byte log entry
0709 14:54:16.690052 (+500424us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:16.690129 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:16.690225 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:16.809912 (+119687us) raft_consensus.cc:1685] finished
0709 14:54:16.809936 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:16.810102 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120472,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":36286,"thread_start_us":600751,"threads_started":2,"wal-append.queue_time_us":555652}
I20260709 14:54:16.797025  9028 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:16.826267  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:16.827059  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:16.824316  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:16.829666  2755 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:16.827466  2753 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:16.809953  8578 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:16.835250  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:16.819876  9055 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:16.839489  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:16.836500  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:16.840443  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62599.4us
I20260709 14:54:16.840535  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:16.825686  8845 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:16.830977  8921 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:16.851266  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:16.831416  9054 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:16.847422  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:16.854993  3537 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:16.855208  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:16.837188  9008 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:16.819192  8814 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 14:54:16.845211  8537 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.005s
I20260709 14:54:16.794445  9064 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:16.861743  9012 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.001s
I20260709 14:54:16.862980  8578 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:16.864483  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:16.855185  9043 consensus_queue.cc:1048] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 14:54:16.855430  9078 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:16.851994  8978 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:16.886986  8978 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:16.887516  9064 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:16.844245  8708 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:16.867152  9044 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:16.841990  9055 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:16.838923  9073 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 14:54:16.879406  8894 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.885649  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:16.885262  9012 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:16.885382  9025 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:16.887667  8978 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:16.866664  8965 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.003s
I20260709 14:54:16.912992  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:16.891484  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55973.7us
W20260709 14:54:16.897743  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50608.6us
I20260709 14:54:16.901144  3540 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:16.901911  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:16.902813  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:16.908962  3537 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:16.917078  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:16.917418  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:16.924811  2759 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:16.934105  2756 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:16.911453  8702 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.272s	user 0.010s	sys 0.006s
W20260709 14:54:17.004351  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 142639us
I20260709 14:54:16.871178  9054 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:16.907990  9078 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:16.880131  8537 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:17.029276  3542 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:16.914921  9050 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:17.050053  8537 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:17.050297  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:17.033123  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:17.051429  9050 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:17.051944  3537 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:17.052551  9045 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:17.053282  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:17.052591  9072 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.561s	user 0.000s	sys 0.003s
I20260709 14:54:17.035035  3540 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:17.055236  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1504) took 1767 ms. Trace:
I20260709 14:54:17.036171  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:17.056257  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 80582.1us
I20260709 14:54:17.038007  9080 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:17.038296  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:16.910876  9025 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:17.040958  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:17.042040  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:17.043181  9081 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 14:54:16.913553  9069 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:16.969607  8708 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:16.905042  9047 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:17.047134  9078 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:17.048409  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 156844us
I20260709 14:54:17.057651  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:17.055790  3543 rpcz_store.cc:276] 0709 14:54:15.287809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:15.294981 (+  7172us) service_pool.cc:224] Handling call
0709 14:54:15.308264 (+ 13283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:15.351475 (+ 43211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:15.351518 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:15.591300 (+239782us) log.cc:844] Serialized 26 byte log entry
0709 14:54:16.475728 (+884428us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:16.475790 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:16.475884 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:17.051705 (+575821us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:17.054906 (+  3201us) raft_consensus.cc:1685] finished
0709 14:54:17.054936 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:17.055175 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71288,"raft.queue_time_us":482327,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":39683,"spinlock_wait_cycles":16546176,"thread_start_us":808818,"threads_started":1,"wal-append.queue_time_us":827698}
I20260709 14:54:17.039494  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:17.076859  3545 raft_consensus.cc:2410] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 14:54:17.061216  3541 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:17.064134  3542 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:17.066087  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 163386us
I20260709 14:54:17.068532  2757 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 14:54:17.063791  2756 pending_rounds.cc:85] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 8
I20260709 14:54:17.028522  9031 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:17.081885  8708 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:17.078504  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:17.080560  2759 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 14:54:17.062501  9071 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:17.057813  8814 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:16.926728  8965 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:17.092250  8965 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:17.064079  9025 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:17.093273  8921 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:17.081962  9056 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:17.112862  9056 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:17.113030  9054 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:17.113456  9056 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:17.113903  9060 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:17.073302  9019 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:17.118467  8871 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:17.119153  8871 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:17.120868  9019 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:17.100809  9047 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:17.101848  9078 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:17.101509  9071 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:17.030948  8702 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:17.105953  9085 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:17.141036  9071 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:17.093636  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:18.269343  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.17583e+06us
I20260709 14:54:18.270522  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.270735  9027 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.271380  9027 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:17.072700  9055 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.273543  9055 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:18.273634  9096 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:18.274722  9096 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:18.275601  9096 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:18.277362  9096 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 14:54:17.074856  9069 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:17.081239  9045 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:17.078125  2756 pending_rounds.cc:107] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:54:17.143374  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:17.123534  8871 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:17.141057  8702 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.290530  8702 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.291186  8702 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:17.087798  9023 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 14:54:16.914062  8894 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:17.152740  9067 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.493077  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:17.089946  8814 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:17.225926  9090 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 14:54:17.137216  9012 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:17.226944  9056 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.503590  9012 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:18.504115  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.504220  9012 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.505312  9056 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:17.230695  9039 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.896s	user 0.012s	sys 0.025s
I20260709 14:54:17.318069  2431 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 11 to 14, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:18.494657  9067 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:17.345077  9078 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:18.508044  2431 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:54:18.508078  9067 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:17.346901  8652 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:17.345971  9064 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.509933  9064 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.510008  9067 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.511226  2606 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
W20260709 14:54:17.151208  2726 connection.cc:256] RPC call timeout handler was delayed by -2.2025589479999326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260709 14:54:17.088842  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:18.549342  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.46051e+06us
I20260709 14:54:18.549628  9019 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:18.550163  9019 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:18.550557  9019 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:17.990187  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:18.544298  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:18.222254  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba240 after lost signal to thread 434
I20260709 14:54:18.551298  2606 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:18.259395  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.002s	sys 0.007s
W20260709 14:54:18.258913  8978 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.227s	user 0.025s	sys 0.004s
W20260709 14:54:18.266446  9025 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.132s	user 0.021s	sys 0.000s
W20260709 14:54:18.552152  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.003s	sys 0.007s
W20260709 14:54:18.552733  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08001e65a0 after lost signal to thread 2469
I20260709 14:54:18.552424  8978 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.281816  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.282760  9095 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:18.283560  2756 raft_consensus.cc:2889] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:18.274545  9027 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.286583  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1811) took 1209 ms. Trace:
I20260709 14:54:18.288828  8871 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:17.065642  9008 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 14:54:18.295017  8702 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:18.300747  9058 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 2.076s	user 0.009s	sys 0.020s
I20260709 14:54:18.328598  9044 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 14:54:18.344476  2724 connection.cc:256] RPC call timeout handler was delayed by -1.5171990970000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 14:54:18.606578  2724 connection.cc:256] RPC call timeout handler was delayed by -1.486599096999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 14:54:18.346419  8866 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.349872  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.359786  8803 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:18.369912  2752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1669) took 1682 ms (client timeout 1836 ms). Trace:
W20260709 14:54:18.614364  2752 rpcz_store.cc:269] 0709 14:54:16.687363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:16.699665 (+ 12302us) service_pool.cc:224] Handling call
0709 14:54:18.369588 (+1669923us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b540013db54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:18.369814 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202909952}
I20260709 14:54:18.374049  8947 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:18.392611  8537 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:54:18.395510  9045 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 14:54:18.398268  8708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.287s	user 0.026s	sys 0.034s
I20260709 14:54:18.400797  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1717) took 1364 ms. Trace:
I20260709 14:54:18.405031  9038 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:18.412815  8847 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:18.606102  9056 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:18.428520  9075 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.901s	user 0.016s	sys 0.007s
I20260709 14:54:18.440375  9098 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.439496  9097 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.447216  9043 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.454499  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1804) took 1545 ms. Trace:
I20260709 14:54:18.456194  9041 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.463611  8921 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:18.622357  9038 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:18.463785  9105 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:18.464536  9023 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:18.465957  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:18.466058  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.467015  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.645135  9112 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.647046  9112 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.647095  9023 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.635097  9056 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.649727  9023 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.649874  9112 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.632441  9111 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:18.469343  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:18.473603  8965 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:18.468339  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.650875  9111 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.651077  8965 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.652324  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.619081  8947 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:18.485124  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.653211  8965 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.488492  9020 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.654117  8947 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.637957  2760 rpcz_store.cc:276] 0709 14:54:16.909547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:16.909767 (+   220us) service_pool.cc:224] Handling call
0709 14:54:16.970506 (+ 60739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:17.026029 (+ 55523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:17.026073 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:17.062486 (+ 36413us) log.cc:844] Serialized 26 byte log entry
0709 14:54:17.130068 (+ 67582us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:17.130122 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:17.142119 (+ 11997us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:18.454114 (+1311995us) raft_consensus.cc:1685] finished
0709 14:54:18.454147 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:18.454400 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2382,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":25824,"thread_start_us":41938,"threads_started":1,"wal-append.queue_time_us":94574,"wal-append.run_cpu_time_us":2930,"wal-append.run_wall_time_us":1021489}
I20260709 14:54:18.655100  9020 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.655846  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.656141  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.620983  8803 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.656800  2760 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 12 FOLLOWER]: Advancing to term 13
I20260709 14:54:18.657123  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.657513  9020 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.490051  8703 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.658419  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:18.496186  9003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.973s	user 0.000s	sys 0.019s
W20260709 14:54:18.681205  9003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.158s	user 0.000s	sys 0.019s
I20260709 14:54:18.508584  9078 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.509356  8652 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:18.683409  8652 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.684331  9078 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.685230  8652 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:18.511420  2431 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 8 to 12, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:18.687597  2431 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 4 to 6, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:54:18.689278  2431 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:18.506613  9012 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.503396  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.691586  2431 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 4 to 11, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:18.692586  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.693802  2431 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 9 to 12, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:18.521353  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.699527  3541 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:18.525760  2758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.371s	user 0.021s	sys 0.052s
I20260709 14:54:18.526454  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.527108  8894 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:18.700600  3536 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:18.528301  8814 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.530937  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.536947  9064 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.703116  9112 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.534322  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.707803  9112 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.708282  2759 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:18.710209  9112 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.539446  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.712105  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.540774  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.713771  9112 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.713925  3546 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:18.714494  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:18.714926  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.541601  9067 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:18.542547  9104 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:18.542057  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.715986  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.716543  3547 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:18.716737  2677 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:18.716382  9104 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:18.717685  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.717833  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.543465  3540 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:18.547627  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.552462  9025 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.725643  9025 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.555943  2757 rpcz_store.cc:276] 0709 14:54:17.077003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:17.089514 (+ 12511us) service_pool.cc:224] Handling call
0709 14:54:18.286493 (+1196979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:18.555668  9095 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:18.560992  9027 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.727540  9027 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.727766  9095 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.729499  9095 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:18.730264  9027 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.562346  8885 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:18.731168  8885 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:18.731783  8885 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:18.564272  8978 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.565737  9096 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:18.559456  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:18.582050  2606 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:18.581923  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:18.593220  2466 debug-util.cc:398] Leaking SignalData structure 0x7b0800137a40 after lost signal to thread 2636
I20260709 14:54:18.737010  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:18.737852  2466 debug-util.cc:398] Leaking SignalData structure 0x7b0800056ea0 after lost signal to thread 2710
W20260709 14:54:18.738271  2466 debug-util.cc:398] Leaking SignalData structure 0x7b0800056e60 after lost signal to thread 2711
I20260709 14:54:18.738977  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:18.599326  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 317615us
I20260709 14:54:18.740554  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.741771  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.742537  2757 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:54:18.611104  2724 connection.cc:256] RPC call timeout handler was delayed by -1.1283990969999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 14:54:18.743086  2724 connection.cc:256] RPC call timeout handler was delayed by -1.0464339359999713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 14:54:18.743621  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.743944  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.744455  2757 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:18.745368  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:18.417189  2684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1625) took 3475 ms (client timeout 1985 ms). Trace:
I20260709 14:54:18.746876  9064 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:18.747896  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:18.746078  2684 rpcz_store.cc:269] 0709 14:54:14.942273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:14.943136 (+   863us) service_pool.cc:224] Handling call
0709 14:54:18.416893 (+3473757us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b5400445a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:18.417079 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827899008}
I20260709 14:54:18.748904  2753 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:18.750098  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.751116  2684 raft_consensus.cc:2410] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 14:54:18.750485  9008 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.752658  9008 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.739678  2686 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:18.739665  2466 debug-util.cc:398] Leaking SignalData structure 0x7b080036fac0 after lost signal to thread 2784
I20260709 14:54:18.620776  3542 rpcz_store.cc:276] 0709 14:54:17.035849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:17.049908 (+ 14059us) service_pool.cc:224] Handling call
0709 14:54:18.400403 (+1350495us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b5400224cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:18.400652 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13570816}
W20260709 14:54:18.629794  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 639718us
I20260709 14:54:18.612363  8702 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.631536  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.634730  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.613619  9004 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.608970  9044 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:18.622329  8708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.511s	user 0.027s	sys 0.036s
I20260709 14:54:18.467343  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:18.762769  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 203425us
W20260709 14:54:18.764424  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 129791us
I20260709 14:54:18.653002  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:18.765664  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.481395  9107 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 14:54:18.489588  9062 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.291s	user 0.010s	sys 0.019s
W20260709 14:54:18.767979   436 debug-util.cc:398] Leaking SignalData structure 0x7b080000fe20 after lost signal to thread 2723
I20260709 14:54:18.655449  8947 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:18.658228  8703 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.490545  8910 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:18.495738  9028 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.664s	user 0.028s	sys 0.000s
I20260709 14:54:18.659257  2751 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:18.659204  8921 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.643116  9019 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.644399  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:18.661909  9102 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 14:54:18.663069  9105 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.664322  2605 raft_consensus.cc:2410] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:18.796977  2684 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:18.653221  9111 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.644239  9043 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.806512  9078 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.807133  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.807289  9078 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:18.807940  9043 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.658667  8803 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.666049  9038 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:18.809334  9038 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:18.809485  2612 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:18.810281  2684 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:18.617295  8866 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.811118  9078 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.811391  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.812592  8866 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:18.643702  9097 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.813716  3546 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 14.
I20260709 14:54:18.815210  9097 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.667181  9045 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.816395  2686 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:18.667708  8965 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:18.817173  8965 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:18.643955  9041 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.654420  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.654495  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:18.681485  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 13.
I20260709 14:54:18.819459  2754 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:18.819823  2604 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:18.820581  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:17.643761   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:54:18.821338  8703 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.637929  9098 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.706357  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.823464  3541 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:18.703114  2758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.548s	user 0.022s	sys 0.052s
I20260709 14:54:18.824913  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.825244  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1677) took 1758 ms. Trace:
I20260709 14:54:18.825385  9041 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.706655  8814 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:18.538308  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.713375  3538 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:18.714336  9112 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.715206  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.827400  2677 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:18.827723  2678 raft_consensus.cc:2376] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20. Re-sending same reply.
I20260709 14:54:18.279898  9069 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.828681  9112 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:18.830442  9105 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.715880  2681 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:18.718811  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.713377  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:18.819510  2610 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:18.833431  2683 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:18.715231  2680 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:54:18.721338  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52800us
W20260709 14:54:18.834683  3411 connection.cc:256] RPC call timeout handler was delayed by -1.9911363539999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 14:54:18.756647  2466 debug-util.cc:398] Leaking SignalData structure 0x7b080027cbe0 after lost signal to thread 2785
W20260709 14:54:18.836544  2466 debug-util.cc:398] Leaking SignalData structure 0x7b080035dc00 after lost signal to thread 3580
I20260709 14:54:18.750649  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.760099  2603 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:18.760304  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:18.825734  2758 rpcz_store.cc:276] 0709 14:54:17.067301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:17.067489 (+   188us) service_pool.cc:224] Handling call
0709 14:54:17.084940 (+ 17451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:17.087234 (+  2294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:17.087264 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:17.127235 (+ 39971us) log.cc:844] Serialized 26 byte log entry
0709 14:54:18.824836 (+1697601us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:18.824893 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:18.824969 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:18.825021 (+    52us) raft_consensus.cc:1685] finished
0709 14:54:18.825036 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:18.825178 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1669924,"threads_started":1,"wal-append.queue_time_us":84032,"wal-append.run_cpu_time_us":1903,"wal-append.run_wall_time_us":1001942}
I20260709 14:54:18.650309  9056 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.768854  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:18.879870  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 111185us
I20260709 14:54:18.881068  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.883200  8866 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:18.884428  8866 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:18.773485  3536 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 14: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:18.889074  9025 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:18.890750  8866 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:18.891482  8866 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:18.892794  8866 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.895284  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:18.901691  9112 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 14:54:18.903254  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:18.903882  8703 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:18.904441  8703 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:18.905393  8703 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.906541  8703 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:18.909274  8866 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:18.875670  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:18.917268  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.767683  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:18.918179  2610 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
W20260709 14:54:18.781524  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:18.777053  2759 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:18.779670  9004 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:18.768712  8702 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.780520  9027 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:18.800210  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:18.801198  9012 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:18.782612  9096 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.801903  9008 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.809324  8921 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:18.816141  9045 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.667032  9103 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 14:54:18.875620  9003 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:18.987641  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.706560  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:18.825852  2752 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:18.839035  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78853.4us
I20260709 14:54:18.746409  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:19.000162  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 253798us
I20260709 14:54:19.001034  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:18.942011  9004 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:18.821486  2760 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:18.843736  2684 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:19.002012  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.837551  2753 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:18.874758  3541 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:18.999019  2677 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
W20260709 14:54:18.835117  3411 connection.cc:256] RPC call timeout handler was delayed by -1.9254363539999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 14:54:19.005293  3411 connection.cc:256] RPC call timeout handler was delayed by -1.5671408800000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
W20260709 14:54:19.005748  3411 connection.cc:256] RPC call timeout handler was delayed by -1.2981363540000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2557s for a response
W20260709 14:54:19.006023  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1883363539999436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26789999999999997s for a response
W20260709 14:54:18.875279  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:18.878356  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:18.917520  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:18.999082  2609 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.008219  2683 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:18.918509  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 154915us
I20260709 14:54:18.791407  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.927582  2751 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:19.008666  2679 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 14:54:18.929488  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:18.922086  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:19.010519  2575 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.600s (SENT)
I20260709 14:54:19.011065  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.011404  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:19.012112  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:18.932886  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.933773  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:18.802029  8708 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:18.945786  2604 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:19.015632  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:19.016405  2683 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:18.935365  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:18.951540  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.874562  9097 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:18.820816  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1671) took 2132 ms (client timeout 1594 ms). Trace:
I20260709 14:54:18.987654  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.788337  8910 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:18.980293  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.020005  2758 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:19.020743  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.006304  9025 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:18.979665  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.006758  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:19.023664  3538 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.022717  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.026672  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:18.795233  9028 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:18.822863  9019 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:18.962554  2680 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:19.046392  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:18.932960  8978 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:18.997095  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.934345  9027 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:18.845453  3545 raft_consensus.cc:2368] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:18.782497  8847 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:19.013293  2610 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:19.019198  9008 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:54:19.070657  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.018510  2754 rpcz_store.cc:269] 0709 14:54:16.688180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:16.699547 (+ 11367us) service_pool.cc:224] Handling call
0709 14:54:18.820540 (+2120993us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:18.820700 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4676241280}
W20260709 14:54:19.035602  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113640us
I20260709 14:54:19.071646  2603 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:18.925948  9021 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:18.928038  9112 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:19.042673  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:19.043551  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.039067  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:18.994547  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.052553  2650 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:19.060088  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.051275  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:19.064287  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52346.2us
W20260709 14:54:19.065977  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55496.8us
I20260709 14:54:19.066632  2684 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 14:54:19.067746  2681 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:19.067628  2683 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:18.983120  9104 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:19.066862  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:19.035094  9004 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:19.045722  8921 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:19.106442  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.147027  3542 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:19.072691  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:19.148224  2684 pending_rounds.cc:85] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 11
I20260709 14:54:19.035387  9060 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.150024  9097 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:19.148583  2684 pending_rounds.cc:107] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 14:54:19.151341  2684 raft_consensus.cc:2889] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:19.105772  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54586.4us
I20260709 14:54:19.147539  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.147946  3543 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.030956  9098 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.076719  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.069988  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.152346  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.161201  3541 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:19.153266  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80644.1us
I20260709 14:54:19.161743  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.113917  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.162544  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:19.048252  9067 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:19.146917  2686 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:19.158783  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.159245  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:18.958416  9096 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.050648  8965 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:19.163396  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 49615.1us
I20260709 14:54:19.164395  2678 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:19.068665  9027 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.057888  8978 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.155728  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.037091  8708 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.059177  9056 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:19.056391  8652 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:19.167738  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:19.168952  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116518us
I20260709 14:54:19.184716  2603 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:19.172591  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.043984  8803 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:19.076406  9055 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 14:54:19.068194  9011 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:19.000245  9003 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:19.185330  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.193574  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.026914  9114 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.142s
I20260709 14:54:19.205761  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:19.170441  9020 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:19.161226  9041 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:54:19.200596  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 193812us
W20260709 14:54:19.200030  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44413us
I20260709 14:54:19.173904  9112 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:19.183449  8978 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:19.206956  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.199946  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.204555  2754 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:19.176213  8894 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:19.210919  3540 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:19.167119  9116 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.160061  9008 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:19.200222  8803 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:19.207042  9011 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.266158  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:18.934974  9044 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:19.183840  9056 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.205602  9004 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:19.209503  9020 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:19.236508  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64035.5us
I20260709 14:54:19.255097  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.255512  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.255867  3543 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:19.168632  9060 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:19.242901  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.301594  3545 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:19.267292  2609 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:19.200507  8652 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.192438  8965 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:19.277828  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.309407  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.285356  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 122883us
I20260709 14:54:19.151631  9097 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:19.212781  9133 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:19.222664  9112 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:19.220296  9041 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:19.300632  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 145106us
I20260709 14:54:19.268060  2751 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
W20260709 14:54:19.322629  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60519.7us
I20260709 14:54:19.222190  2752 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 13.
I20260709 14:54:19.229477  8847 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:19.223140  8894 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.301493  9020 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.224491  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.272501  8803 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:19.279109  9011 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.279572  9056 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.314464  8921 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 14:54:19.312556  9133 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:19.308203  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53235.8us
I20260709 14:54:19.328577  9041 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.276674  9019 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 14:54:19.235505  9131 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 14:54:19.348479  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.363996  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:19.367880  2609 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:19.350508  2603 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:19.350418  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.326900  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.275309  9116 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:19.355075  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.302930  9135 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:19.280084  9004 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.376101  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.357645  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.378142  2686 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.346306  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.364671  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:19.349967  8894 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:19.348362  9132 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.349416  9109 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.308085  9067 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:19.370910  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 128175us
I20260709 14:54:19.348850  9016 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:19.381783  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.383618  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.384958  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:19.407487  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.401072  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.417080  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.557564  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.558419  2760 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.385305  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.452487  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.560057  2752 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.560642  2753 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:19.394565  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.465689  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:19.561892  2681 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:19.562383  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.408440  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.563460  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 187436us
I20260709 14:54:19.564002  2610 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 13: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:19.564671  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.411175  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.565021  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.412122  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.409556  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.597203  2605 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:19.597828  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.514213  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:19.598619  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.599923  2605 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:19.470023  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:19.599949  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85909.4us
I20260709 14:54:19.383719  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.601531  2607 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:19.557651  2759 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:19.539368  3538 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.512s	user 0.006s	sys 0.007s
I20260709 14:54:19.602751  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.603627  3538 raft_consensus.cc:2410] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 14:54:19.570851  2678 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:19.605358  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:19.606066  2678 raft_consensus.cc:2376] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20. Re-sending same reply.
I20260709 14:54:19.557030  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:19.361572  9133 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.477936  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:19.573428  3542 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
W20260709 14:54:19.615558  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 137752us
W20260709 14:54:19.616266  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.573896  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:19.580832  8885 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.008s
I20260709 14:54:19.613413  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.623026  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 437776us
I20260709 14:54:19.416002  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:19.624614  9153 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:19.625059  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:19.625317  9153 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.625721  9153 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.626075  2604 raft_consensus.cc:2410] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:54:19.370030  8702 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:19.608753  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.627305  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.627781  2753 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:19.628178  2604 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
W20260709 14:54:19.609874  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44917.6us
I20260709 14:54:19.611581  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1148) took 1097 ms. Trace:
W20260709 14:54:19.497217  8703 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
I20260709 14:54:19.629381  2751 rpcz_store.cc:276] 0709 14:54:18.514144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.374565 (+860421us) service_pool.cc:224] Handling call
0709 14:54:19.611496 (+236931us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:19.504649  8814 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.675s	user 0.007s	sys 0.011s
I20260709 14:54:19.382433  9004 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:19.623049  9152 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:19.625862  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:19.628283  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1158) took 1093 ms. Trace:
I20260709 14:54:19.645648  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.646281  2753 rpcz_store.cc:276] 0709 14:54:18.534609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.608413 (+1073804us) service_pool.cc:224] Handling call
0709 14:54:19.628204 (+ 19791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.448580  9003 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.652498  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:19.620210  9134 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.263s	user 0.000s	sys 0.004s
W20260709 14:54:19.449725  9069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.014s	sys 0.020s
W20260709 14:54:19.612679  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50335.9us
I20260709 14:54:19.631331  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.607280  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50369.1us
I20260709 14:54:19.613078  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:19.633006  2607 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:19.668140  2610 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:19.630908  8871 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:19.633652  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.681504  2603 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:19.638897  9044 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.629534  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.382329  9019 consensus_queue.cc:1243] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 14:54:19.403806  9095 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:19.640002  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.405045  9103 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:54:19.641691  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 67924.4us
W20260709 14:54:19.667692  9069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.015s	sys 0.028s
I20260709 14:54:19.701601  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:19.639921  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:19.606103  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 88821.2us
I20260709 14:54:19.383090  8921 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:19.443349  9096 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.435927  8708 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.720420  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:19.443003  9060 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:19.647197  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.408777  8947 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.626s	user 0.003s	sys 0.008s
I20260709 14:54:19.441541  8910 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:19.630558  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.733597  2609 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:19.647764  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.687942  9111 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:19.556393  9025 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.558785  9016 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.699611  9103 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:19.446651  9028 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:19.653003  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:19.744853  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:19.620085  9128 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.244s	user 0.001s	sys 0.003s
I20260709 14:54:19.653474  3542 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:19.615334  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.649466  3537 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:19.654675  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.747057  3538 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:19.655471  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:19.656098  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 291511us
I20260709 14:54:19.459468  8652 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.571592  9078 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.609784  9115 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:19.571789  8965 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:19.469844  9116 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:19.595129  2758 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.006s
I20260709 14:54:19.669423  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.629648  2759 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:19.725896  8871 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:19.729231  8921 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:19.768810  2758 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:19.670734  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1171) took 1024 ms. Trace:
I20260709 14:54:19.674062  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.626041  8894 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:19.636873  9012 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 14:54:19.676278  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.676937  2752 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:19.677165  2724 leader_election.cc:400] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
W20260709 14:54:19.677964  2726 leader_election.cc:336] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.213s (SENT)
W20260709 14:54:19.675976  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46899.4us
I20260709 14:54:19.686668  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:19.396195  9098 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.256s
I20260709 14:54:19.371898  9008 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:19.662973  9118 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.852s	user 0.009s	sys 0.006s
I20260709 14:54:19.654407  9152 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.763793  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.657814  8703 raft_consensus.cc:2764] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 14:54:19.710103  2576 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.711498  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.716821  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.852473  2677 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:19.717293  2681 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:19.467187  9132 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.717206  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.721598  2685 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:19.608685  9109 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.670136  9045 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:19.669122  8702 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.731737  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1850) took 1169 ms. Trace:
I20260709 14:54:19.384491  8847 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:19.735190  2753 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:19.908598  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.737167  2608 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:19.637868  8885 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:19.738188  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 374282us
I20260709 14:54:19.436579  9138 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:19.723455  9049 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:19.732640  9130 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.367s	user 0.000s	sys 0.004s
I20260709 14:54:19.752408  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.723876  9114 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:19.769146  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1148) took 1076 ms. Trace:
I20260709 14:54:19.756780  3536 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:19.769483  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1175) took 1104 ms. Trace:
I20260709 14:54:19.939028  2758 rpcz_store.cc:276] 0709 14:54:18.665055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.923741 (+258686us) service_pool.cc:224] Handling call
0709 14:54:19.769418 (+845677us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.938731  2606 rpcz_store.cc:276] 0709 14:54:18.674121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.736476 (+1062355us) service_pool.cc:224] Handling call
0709 14:54:19.749337 (+ 12861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:19.749651 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:19.749692 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:19.749784 (+    92us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:19.749835 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:19.749919 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:19.750095 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.650641  9064 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:19.853247  2724 leader_election.cc:403] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 14:54:19.943058  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:19.943934  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 275805us
I20260709 14:54:19.944722  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.945501  2606 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:19.765802  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1620) took 1053 ms. Trace:
I20260709 14:54:19.947212  2603 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 14:54:19.947527  9003 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16.
I20260709 14:54:19.741951  9025 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:19.949429  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1808) took 2916 ms. Trace:
W20260709 14:54:19.769829  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 197882us
I20260709 14:54:19.762636  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:19.835572  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89848.8us
W20260709 14:54:19.819650  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.951702  2678 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.952827  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.953881  2679 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.953979  2678 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.858495  9020 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:19.955395  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.956734  9020 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:19.859849  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 148506us
I20260709 14:54:19.957793  2678 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.865314  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.958765  2681 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:19.862036  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.959435  2759 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:19.960096  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.960605  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.961402  2759 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:19.864365  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.858739  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1854) took 1054 ms. Trace:
W20260709 14:54:19.963523  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 285588us
I20260709 14:54:19.963721  2607 rpcz_store.cc:276] 0709 14:54:18.803952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.686504 (+882552us) service_pool.cc:224] Handling call
0709 14:54:19.858654 (+172150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.965258  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:19.965790  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.965962  2607 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.732731  9112 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.902482  2604 rpcz_store.cc:276] 0709 14:54:18.562036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.698638 (+1136602us) service_pool.cc:224] Handling call
0709 14:54:19.730666 (+ 32028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:19.731012 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:19.731052 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:19.731136 (+    84us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:19.731198 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:19.731454 (+   256us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:19.731638 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.748610  9103 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:19.968092  2604 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 14:54:19.968781  9112 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:19.869630  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 159666us
I20260709 14:54:19.968822  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1700) took 1023 ms. Trace:
I20260709 14:54:19.969834  2604 rpcz_store.cc:276] 0709 14:54:18.945379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.967403 (+1022024us) service_pool.cc:224] Handling call
0709 14:54:19.967832 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:19.968529 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:19.968758 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.970453  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:19.758347  9145 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:19.972064  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.744088  9016 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.973066  9069 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:19.974020  9069 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:19.974558  9069 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:19.754117  9019 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:19.975685  9019 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:19.975606  9069 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:19.761795  9123 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.233s	user 0.001s	sys 0.006s
I20260709 14:54:19.977502  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.978963  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.979036  9114 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.979839  2759 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:19.980878  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:19.981460  2759 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:19.947096  9049 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:19.983340  9055 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:19.984077  9049 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:19.963523  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:19.985932  2678 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:19.947012  2682 rpcz_store.cc:276] 0709 14:54:18.712703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.722760 (+ 10057us) service_pool.cc:224] Handling call
0709 14:54:18.723267 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:18.767078 (+ 43811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:18.767123 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:18.785504 (+ 18381us) log.cc:844] Serialized 26 byte log entry
0709 14:54:19.172383 (+386879us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:19.172582 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:19.172687 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:19.765524 (+592837us) raft_consensus.cc:1685] finished
0709 14:54:19.765548 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:19.765729 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31029,"raft.queue_time_us":31234,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":23287,"thread_start_us":236401,"threads_started":1,"wal-append.queue_time_us":440235}
I20260709 14:54:19.854207  2760 rpcz_store.cc:276] 0709 14:54:18.646237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.417626 (+771389us) service_pool.cc:224] Handling call
0709 14:54:19.668659 (+251033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.909456  2753 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 15.
I20260709 14:54:19.989529  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.728225  9067 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:19.917716  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.007500  2677 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:20.008719  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:20.009124  9137 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.492s	user 0.000s	sys 0.003s
I20260709 14:54:20.010253  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:19.858134  8978 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.002s
W20260709 14:54:19.900242  8866 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.990s	user 0.016s	sys 0.003s
I20260709 14:54:19.854463  9012 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:19.923452  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:20.017480  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 94137.7us
I20260709 14:54:19.994572  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:19.868556  9104 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:19.858347  9152 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:20.028738  2683 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:19.940510  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.030085  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:20.030282  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 89878.1us
I20260709 14:54:20.032141  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:19.940603  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.941926  2685 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 15.
I20260709 14:54:19.758473  9155 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:19.942416  9011 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.003s
I20260709 14:54:19.948979  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:19.949874  2756 rpcz_store.cc:276] 0709 14:54:17.033040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:17.063152 (+ 30112us) service_pool.cc:224] Handling call
0709 14:54:17.063591 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:18.556253 (+1492662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:18.556290 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:18.581767 (+ 25477us) log.cc:844] Serialized 52 byte log entry
0709 14:54:18.778571 (+196804us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:18.780956 (+  2385us) log.cc:844] Serialized 14 byte log entry
0709 14:54:18.784138 (+  3182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:18.784232 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:19.238320 (+454088us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b1800399f40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db
0709 14:54:19.428047 (+189727us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:19.948981 (+520934us) raft_consensus.cc:1685] finished
0709 14:54:19.949015 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:19.949217 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":300,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":8960,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":8426,"spinlock_wait_cycles":81500928,"thread_start_us":169517,"threads_started":1,"wal-append.queue_time_us":174920}
I20260709 14:54:19.739975  8910 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.957494  2684 raft_consensus.cc:2368] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:19.957350  8871 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:19.961848  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:19.992763  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 240425us
I20260709 14:54:20.017326  9064 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:19.901523  9107 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:19.913084  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:19.995656  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:20.060448  2678 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:19.998293  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:19.953392  8885 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.028409  9045 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:20.004256  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1816) took 1534 ms. Trace:
I20260709 14:54:19.758646  9149 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.010638  2726 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.076973  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1165) took 1068 ms. Trace:
I20260709 14:54:20.077363  2679 rpcz_store.cc:276] 0709 14:54:19.008217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.026241 (+ 18024us) service_pool.cc:224] Handling call
0709 14:54:20.076759 (+1050518us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b5400376c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.076912 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125745536}
I20260709 14:54:20.078580  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:20.082477  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71959.3us
W20260709 14:54:20.050586  9120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.031s	sys 0.049s
I20260709 14:54:20.021428  2759 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:19.859555  9116 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.025107  2607 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:20.021157  8978 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.034533  2758 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:20.019184  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:20.075866  9041 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.001s
I20260709 14:54:20.054752  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:20.115639  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96572.2us
I20260709 14:54:19.951702  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:20.111124  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:20.028330  9038 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.218s	user 0.010s	sys 0.006s
I20260709 14:54:20.055441  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1169) took 1032 ms. Trace:
I20260709 14:54:20.003698  2753 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:20.025846  9055 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:20.021471  9012 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:19.910624  8702 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.060490  2752 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:20.072758  2760 rpcz_store.cc:276] 0709 14:54:18.470274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.003017 (+1532743us) service_pool.cc:224] Handling call
0709 14:54:20.003479 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.003755 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.003793 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:20.003872 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:20.003906 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.003977 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.004159 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:19.764422  9078 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.088303  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:20.021230  9028 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 14:54:20.103844  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1682) took 1060 ms. Trace:
I20260709 14:54:20.105568  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.115446  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1684) took 1046 ms. Trace:
I20260709 14:54:20.040289  9095 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:20.118780  9104 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:20.129398  9095 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:20.129961  9104 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.130250  9095 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:20.130645  9104 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:20.103420  9120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.031s	sys 0.050s
I20260709 14:54:20.134866  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:20.135757  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 184161us
I20260709 14:54:20.136965  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.065389  9107 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:20.079631  9045 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.123317  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1172) took 1476 ms. Trace:
I20260709 14:54:20.156767  9045 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:20.157446  3546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.012s	sys 0.106s
W20260709 14:54:20.158035  3546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.012s	sys 0.106s
I20260709 14:54:20.128931  2677 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:20.117326  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:20.127334  2679 raft_consensus.cc:2376] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15. Re-sending same reply.
I20260709 14:54:20.128747  2759 rpcz_store.cc:276] 0709 14:54:19.043213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.980559 (+937346us) service_pool.cc:224] Handling call
0709 14:54:20.103712 (+123153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.058739  8910 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.140682  9154 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.129244  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1852) took 1331 ms. Trace:
I20260709 14:54:20.054316  9155 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.055562  9011 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.101271  9019 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:20.145808  9038 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:20.147444  9078 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:20.160320  2606 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:20.119819  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1815) took 1649 ms. Trace:
W20260709 14:54:20.185814  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 131209us
I20260709 14:54:20.187726  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.188793  2606 raft_consensus.cc:2393] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.187810  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.185376  2753 rpcz_store.cc:276] 0709 14:54:18.469984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.003255 (+1533271us) service_pool.cc:224] Handling call
0709 14:54:20.003498 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.119338 (+115840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.119696 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.111358  8652 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:20.165957  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48741.8us
I20260709 14:54:20.166090  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.157269  2752 rpcz_store.cc:276] 0709 14:54:18.646572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.866869 (+1220297us) service_pool.cc:224] Handling call
0709 14:54:20.123208 (+256339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.168367  2612 rpcz_store.cc:276] 0709 14:54:18.797921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.105129 (+1307208us) service_pool.cc:224] Handling call
0709 14:54:20.129158 (+ 24029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.127574  2611 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:20.168789  2759 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:20.080796  8847 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.170461  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.037976  8921 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.128427  9028 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:20.072187  9064 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.138306  9116 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.206881  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.176239  9060 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:20.141530  9041 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.143976  9107 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:20.155155  9027 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.954s	user 0.010s	sys 0.004s
I20260709 14:54:20.119084  2685 rpcz_store.cc:276] 0709 14:54:19.009789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.040633 (+1030844us) service_pool.cc:224] Handling call
0709 14:54:20.041175 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.041555 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.041602 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:20.041670 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:20.041735 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.041818 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.042023 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.105032  9003 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 14:54:20.123515  2757 raft_consensus.cc:2410] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:54:20.148586  8885 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:20.231686  8885 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:20.232372  9139 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.733s	user 0.000s	sys 0.004s
I20260709 14:54:20.233207  8885 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.234853  9021 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:20.235349  9021 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:20.235605  2685 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:20.128994  2758 rpcz_store.cc:276] 0709 14:54:19.068753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.940170 (+871417us) service_pool.cc:224] Handling call
0709 14:54:20.115356 (+175186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.237802  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1818) took 1617 ms. Trace:
I20260709 14:54:20.083479  9103 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.239437  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.240567  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1173) took 1130 ms. Trace:
I20260709 14:54:20.240618  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1636) took 1108 ms. Trace:
I20260709 14:54:20.241293  9103 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.241032  2683 rpcz_store.cc:276] 0709 14:54:19.131954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.158473 (+ 26519us) service_pool.cc:224] Handling call
0709 14:54:20.240466 (+1081993us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.240564 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986219392}
I20260709 14:54:20.163026  2680 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:20.243161  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1845) took 1776 ms. Trace:
I20260709 14:54:20.243709  2608 rpcz_store.cc:276] 0709 14:54:18.483948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.484711 (+   763us) service_pool.cc:224] Handling call
0709 14:54:18.581765 (+ 97054us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:20.242958 (+1661193us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540016d5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.243090 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305608320}
I20260709 14:54:20.244356  9155 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.245061  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.192513  2753 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:20.236410  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1633) took 1105 ms. Trace:
I20260709 14:54:20.238219  2758 rpcz_store.cc:276] 0709 14:54:18.620891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.236784 (+1615893us) service_pool.cc:224] Handling call
0709 14:54:20.237161 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.237393 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.237428 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:20.237481 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:20.237532 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.237592 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.237709 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.247296  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.247727  2758 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:54:20.248386  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1163) took 1618 ms. Trace:
I20260709 14:54:20.248651  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.248725  2758 rpcz_store.cc:276] 0709 14:54:18.629638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.246967 (+1617329us) service_pool.cc:224] Handling call
0709 14:54:20.247358 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.248119 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.248320 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.193374  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.120139  9055 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.250087  9055 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:20.250203  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.169005  9132 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:20.251195  3543 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:20.251989  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.253232  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.198202  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1149) took 1683 ms. Trace:
I20260709 14:54:20.172273  9150 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:20.165722  9109 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:20.254714  2759 rpcz_store.cc:276] 0709 14:54:18.514500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.168053 (+1653553us) service_pool.cc:224] Handling call
0709 14:54:20.168539 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.197845 (+ 29306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.198119 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.250241  2606 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 14: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:20.255473  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1644) took 1106 ms. Trace:
I20260709 14:54:20.256052  2684 rpcz_store.cc:276] 0709 14:54:19.149394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.469476 (+320082us) service_pool.cc:224] Handling call
0709 14:54:20.255258 (+785782us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400445a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.255397 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819740032}
I20260709 14:54:20.256458  2759 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 14:54:20.257045  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.257228  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1178) took 1591 ms. Trace:
I20260709 14:54:20.257390  2684 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:20.201933  2604 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:20.258090  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1637) took 1110 ms. Trace:
I20260709 14:54:20.258762  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.259687  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1702) took 1253 ms. Trace:
I20260709 14:54:20.202592  2754 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.260192  9011 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.260432  9116 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:20.261324  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1164) took 1618 ms. Trace:
I20260709 14:54:20.235353  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1147) took 1589 ms. Trace:
I20260709 14:54:20.261835  2754 rpcz_store.cc:276] 0709 14:54:18.643173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.344722 (+701549us) service_pool.cc:224] Handling call
0709 14:54:20.261112 (+916390us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54002e22d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.261260 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1883303808}
I20260709 14:54:20.262351  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.178251  9114 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.256601  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.207342  2752 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:20.270159  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1156) took 1736 ms. Trace:
I20260709 14:54:20.270344  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.074291  8965 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.101891  9149 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.211807  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:20.220885  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.279127  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.204973  9028 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:20.199714  8708 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:20.229404  9160 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 14:54:20.239822  9159 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 14:54:20.246237  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1153) took 1713 ms. Trace:
I20260709 14:54:20.301898  2753 rpcz_store.cc:276] 0709 14:54:18.532908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.191731 (+1658823us) service_pool.cc:224] Handling call
0709 14:54:20.192207 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.245922 (+ 53715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.246163 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.249563  2612 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:20.199425  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.199826  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.203640  2679 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:20.306173  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.307580  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.308346  3541 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.240983  2682 rpcz_store.cc:276] 0709 14:54:19.110098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.239084 (+1128986us) service_pool.cc:224] Handling call
0709 14:54:20.240501 (+  1417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.257650  2759 rpcz_store.cc:276] 0709 14:54:18.666084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.255717 (+1589633us) service_pool.cc:224] Handling call
0709 14:54:20.256104 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.256968 (+   864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.257168 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.254562  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:20.179759  9144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.015s	sys 0.004s
I20260709 14:54:20.311223  2681 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:20.312099  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112394us
W20260709 14:54:20.315646  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1522) took 1768 ms (client timeout 2170 ms). Trace:
W20260709 14:54:20.316444  3537 rpcz_store.cc:269] 0709 14:54:18.547121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.547267 (+   146us) service_pool.cc:224] Handling call
0709 14:54:20.315338 (+1768071us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54004b0fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.315538 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138509568}
I20260709 14:54:20.258514  2684 rpcz_store.cc:276] 0709 14:54:19.148048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.256926 (+1108878us) service_pool.cc:224] Handling call
0709 14:54:20.257213 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.257862 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.258038 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:20.280112  9153 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.654s	user 0.000s	sys 0.015s
I20260709 14:54:20.260092  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.246069  2685 rpcz_store.cc:276] 0709 14:54:19.130964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.234925 (+1103961us) service_pool.cc:224] Handling call
0709 14:54:20.235392 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.236111 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.236345 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.260082  2604 rpcz_store.cc:276] 0709 14:54:19.006716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.970699 (+963983us) service_pool.cc:224] Handling call
0709 14:54:19.971164 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.258327 (+287163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.259422 (+  1095us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.259610 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530168192}
I20260709 14:54:20.262341  2609 rpcz_store.cc:276] 0709 14:54:18.645933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.734529 (+1088596us) service_pool.cc:224] Handling call
0709 14:54:19.742015 (+  7486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:19.742535 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:19.742584 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:19.846395 (+103811us) log.cc:844] Serialized 26 byte log entry
0709 14:54:20.063645 (+217250us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:20.065308 (+  1663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.065454 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:20.235064 (+169610us) raft_consensus.cc:1685] finished
0709 14:54:20.235091 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.235284 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57952,"raft.queue_time_us":2381,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":19673,"thread_start_us":198717,"threads_started":1,"wal-append.queue_time_us":334051}
I20260709 14:54:20.271401  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:20.286688  9004 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.626s	user 0.007s	sys 0.002s
I20260709 14:54:20.270634  2752 rpcz_store.cc:276] 0709 14:54:18.533888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.196623 (+1662735us) service_pool.cc:224] Handling call
0709 14:54:20.206959 (+ 10336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.269849 (+ 62890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.270098 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.272046  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:20.274134  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1360) took 1257 ms. Trace:
I20260709 14:54:20.274613  2683 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 14:54:20.274708  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 108725us
I20260709 14:54:20.204922  9019 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:20.347698  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:20.275296  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.277456  2751 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.178498  9154 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:20.220432  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.212684  8894 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.283839  8978 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.210774  9060 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.203987  9145 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:20.222457  9038 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:20.294474  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.295445  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.296236  3543 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.309403  2611 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:54:20.320025  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48718.5us
I20260709 14:54:20.271610  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:20.263717  9027 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:20.344169  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1166) took 1700 ms (client timeout 1753 ms). Trace:
I20260709 14:54:20.321795  3541 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:20.322625  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.377422  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1635) took 1245 ms. Trace:
I20260709 14:54:20.331594  3540 raft_consensus.cc:1217] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Deduplicated request from leader. Original: 13.7->[15.8-17.9]   Dedup: 15.8->[17.9-17.9]
I20260709 14:54:20.475652  3540 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 14:54:20.321398  2759 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: []
I20260709 14:54:20.328743  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1658) took 1123 ms. Trace:
W20260709 14:54:20.341409  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 321922us
I20260709 14:54:20.477027  2677 rpcz_store.cc:276] 0709 14:54:19.205263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.009898 (+804635us) service_pool.cc:224] Handling call
0709 14:54:20.328514 (+318616us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.328664 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269981056}
I20260709 14:54:20.384819  2758 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:20.293959  8803 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.933s	user 0.011s	sys 0.002s
I20260709 14:54:20.201786  8702 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:20.387056  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 115089us
I20260709 14:54:20.320147  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:20.389734  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.288130  8910 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.377866  3547 rpcz_store.cc:276] 0709 14:54:19.016428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.016637 (+   209us) service_pool.cc:224] Handling call
0709 14:54:19.055993 (+ 39356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:19.622066 (+566073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:19.622104 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:19.659770 (+ 37666us) log.cc:844] Serialized 52 byte log entry
0709 14:54:19.864839 (+205069us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:19.923950 (+ 59111us) log.cc:844] Serialized 14 byte log entry
0709 14:54:19.924324 (+   374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.262713 (+338389us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54004b37d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.262780 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:20.262856 (+    76us) raft_consensus.cc:1685] finished
0709 14:54:20.273833 (+ 10977us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.274065 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30022,"prepare.queue_time_us":58855,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":57645,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":23419,"spinlock_wait_cycles":665255296,"thread_start_us":402871,"threads_started":2,"wal-append.queue_time_us":255794}
I20260709 14:54:20.394786  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.395999  9004 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.403429  2605 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:20.297112  9114 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:20.294842  9116 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:20.492408  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 172363us
I20260709 14:54:20.408806  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:20.333416  9144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.016s	sys 0.004s
I20260709 14:54:20.413475  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.321918  9115 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.002s
I20260709 14:54:20.299731  8708 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:20.322029  9155 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:20.524794  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.525043  9155 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:20.527881  9044 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.831s	user 0.009s	sys 0.005s
I20260709 14:54:20.484172  9028 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.529107  9115 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:20.529440  9028 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:20.420136  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:20.530993  9154 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.531147  2752 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:20.531265  2610 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:20.497521  9114 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.533798  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1177) took 1868 ms. Trace:
I20260709 14:54:20.534164  9114 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:20.535132  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.535390  9064 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:20.535559  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1154) took 2002 ms (client timeout 2136 ms). Trace:
I20260709 14:54:20.535923  2685 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:20.327988  9041 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:20.536031  2755 rpcz_store.cc:269] 0709 14:54:18.533268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.038585 (+505317us) service_pool.cc:224] Handling call
0709 14:54:20.535331 (+1496746us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54000e6354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.535494 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2919822720}
I20260709 14:54:20.403756  9049 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:20.537038  9049 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.236399  9021 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.537397  2685 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 14:54:20.537303  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:20.454576  2757 rpcz_store.cc:269] 0709 14:54:18.644050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.745055 (+101005us) service_pool.cc:224] Handling call
0709 14:54:20.343911 (+1598856us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001ff9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.344082 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3167007488}
I20260709 14:54:20.538138  2755 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:20.538662  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:20.538954  9027 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.344750  9149 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.540390  8708 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:20.387307  9019 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:20.478083  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 183704us
I20260709 14:54:20.544306  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1651) took 1389 ms. Trace:
I20260709 14:54:20.484009  9038 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:20.526650  3537 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.475409  2683 rpcz_store.cc:276] 0709 14:54:19.131639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.242300 (+1110661us) service_pool.cc:224] Handling call
0709 14:54:20.242697 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.376975 (+134278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.377342 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194432}
I20260709 14:54:20.534320  2758 rpcz_store.cc:276] 0709 14:54:18.665717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.249745 (+1584028us) service_pool.cc:224] Handling call
0709 14:54:20.384257 (+134512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.478490 (+ 94233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.533479 (+ 54989us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001ff9d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.533676 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266431872}
W20260709 14:54:20.477274  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 212587us
I20260709 14:54:20.546281  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:20.476855  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1821) took 1675 ms. Trace:
I20260709 14:54:20.547180  2758 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:20.547482  2759 rpcz_store.cc:276] 0709 14:54:18.801871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.309297 (+1507426us) service_pool.cc:224] Handling call
0709 14:54:20.321185 (+ 11888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.476532 (+155347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.476787 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.548650  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:20.476683  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.478794  2682 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:20.481922  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.418728  9131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.007s	sys 0.032s
I20260709 14:54:20.484699  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:20.310153  9016 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:20.557293  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80714.7us
I20260709 14:54:20.558393  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.558698  2752 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:20.490962  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.483364  8803 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.491633  9141 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:54:20.562191  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77542.6us
I20260709 14:54:20.385645  2678 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:20.492439  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 97748.7us
I20260709 14:54:20.563827  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1656) took 1361 ms. Trace:
I20260709 14:54:20.564482  2755 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 16: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.564793  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.544166  9019 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:20.497381  2604 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:20.480446  9112 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:20.564217  2678 rpcz_store.cc:276] 0709 14:54:19.202490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.995243 (+792753us) service_pool.cc:224] Handling call
0709 14:54:20.563746 (+568503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.387786  9153 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.476790  2612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:20.626363  2612 raft_consensus.cc:606] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:20.627060  2612 raft_consensus.cc:614] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:20.419894  9012 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.484100  8978 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:20.522509  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1698) took 1577 ms. Trace:
I20260709 14:54:20.639511  2605 rpcz_store.cc:276] 0709 14:54:18.944822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.899414 (+954592us) service_pool.cc:224] Handling call
0709 14:54:19.907084 (+  7670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.488083 (+580999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.522151 (+ 34068us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400346054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.522420 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140815232}
I20260709 14:54:20.422511  9145 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.538251  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1660) took 1271 ms. Trace:
I20260709 14:54:20.640782  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:20.551820  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1657) took 1347 ms. Trace:
I20260709 14:54:20.544827  2680 rpcz_store.cc:276] 0709 14:54:19.154443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.685920 (+531477us) service_pool.cc:224] Handling call
0709 14:54:20.543927 (+858007us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.544165 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712232832}
W20260709 14:54:20.553158  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139816us
I20260709 14:54:20.323100  9150 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 14:54:20.555507  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.403841  9163 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:20.547034  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:20.420344  9060 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:20.559702  3547 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:20.559691  2677 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:20.559566  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.651971  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:20.652602  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 93766.7us
I20260709 14:54:20.653069  2759 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:20.655009  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.568967  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1155) took 2035 ms (client timeout 2679 ms). Trace:
I20260709 14:54:20.569522  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:20.522789  9167 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 14:54:20.497151  3411 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.679s (SENT)
I20260709 14:54:20.576819  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1817) took 2103 ms. Trace:
I20260709 14:54:20.612614  9011 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:20.526109  9155 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.581362  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:20.581848  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.578254  9038 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.488159  9103 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 14:54:20.585805  9021 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:20.646778  9149 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:20.587076  9044 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:20.601514  8708 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:20.589545  9025 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 14:54:20.669409  8978 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:20.591074  2753 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.591795  9016 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.592005  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1652) took 1433 ms. Trace:
I20260709 14:54:20.574784  9104 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.593684  9055 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:20.576124  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.598634  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1851) took 1801 ms. Trace:
I20260709 14:54:20.537484  9049 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:20.599939  9133 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.000s
I20260709 14:54:20.600378  9004 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:20.557554  9131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.008s	sys 0.035s
I20260709 14:54:20.603941  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1146) took 2064 ms. Trace:
I20260709 14:54:20.604480  2607 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 14:54:20.611775  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.496961  9008 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:20.622627  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1715) took 1385 ms. Trace:
I20260709 14:54:20.735224  2604 rpcz_store.cc:276] 0709 14:54:19.236909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.477288 (+1240379us) service_pool.cc:224] Handling call
0709 14:54:20.495682 (+ 18394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.622193 (+126511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.622530 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.735737  3536 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.518749  9132 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 14:54:20.626402  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 144552us
I20260709 14:54:20.624115  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:20.641480  2678 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:20.644486  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89096.3us
I20260709 14:54:20.742563  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.642738  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:20.571548  9056 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.201s	user 0.014s	sys 0.001s
I20260709 14:54:20.641243  2685 rpcz_store.cc:276] 0709 14:54:19.266974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.536709 (+1269735us) service_pool.cc:224] Handling call
0709 14:54:20.537140 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.537965 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.538191 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.742569  3546 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.745008  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.745967  2685 raft_consensus.cc:2376] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17. Re-sending same reply.
W20260709 14:54:20.722646  9071 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.286s	user 0.004s	sys 0.031s
I20260709 14:54:20.647183  2682 rpcz_store.cc:276] 0709 14:54:19.204845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.416481 (+1211636us) service_pool.cc:224] Handling call
0709 14:54:20.475481 (+ 59000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.551362 (+ 75881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.551745 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.748355  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:20.749369  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.567829  9116 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:20.647294  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100368us
I20260709 14:54:20.751329  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.568380  3543 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:20.656394  2751 rpcz_store.cc:269] 0709 14:54:18.533602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.639478 (+1105876us) service_pool.cc:224] Handling call
0709 14:54:20.568642 (+929164us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.568862 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315384576}
I20260709 14:54:20.650323  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.581411  8803 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:20.667814  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.764496  2680 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:20.668433  2756 rpcz_store.cc:276] 0709 14:54:18.472964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.044162 (+1571198us) service_pool.cc:224] Handling call
0709 14:54:20.116120 (+ 71958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.116433 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.116474 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:20.116546 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:20.116607 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.116689 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.576597 (+459908us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b54003363d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.576748 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164162816}
W20260709 14:54:20.577142  9171 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 14:54:20.669121  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.669610  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:20.774168  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105150us
I20260709 14:54:20.617180  9112 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:20.645505  9095 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.514s	user 0.014s	sys 0.000s
I20260709 14:54:20.782581  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.783181  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.783795  3547 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 14:54:20.675812  3540 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:20.784083  2685 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:20.565939  9141 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:20.680397  2608 raft_consensus.cc:2368] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14.
W20260709 14:54:20.680671  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.677703  8652 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 14:54:20.639343  9003 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 14:54:20.671893  9027 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:20.698601  9153 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:20.587627  9164 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 14:54:20.733018  9133 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.705307  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:20.787865  8910 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.703840  2758 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:20.590847  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1824) took 1677 ms. Trace:
I20260709 14:54:20.651222  9163 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:20.711309  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:20.713812  2681 rpcz_store.cc:276] 0709 14:54:19.158800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.961525 (+802725us) service_pool.cc:224] Handling call
0709 14:54:20.591759 (+630234us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54002b3c54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.591930 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591044992}
I20260709 14:54:20.672173  9098 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.639832  9041 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 14:54:20.717281  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 147828us
I20260709 14:54:20.762972  9069 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:20.717412  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:20.803802  3538 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:20.773046  9109 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:20.727103  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1820) took 2023 ms. Trace:
I20260709 14:54:20.713357  8978 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.732290  2603 rpcz_store.cc:276] 0709 14:54:18.539250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.696975 (+1157725us) service_pool.cc:224] Handling call
0709 14:54:19.698052 (+  1077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:19.947841 (+249789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.603452 (+655611us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54002ff3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.603828 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571564416}
I20260709 14:54:20.749482  2682 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:20.570907  9115 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:20.755612  2751 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:20.833460  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1827) took 1272 ms. Trace:
I20260709 14:54:20.833889  2751 rpcz_store.cc:276] 0709 14:54:19.560847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.754692 (+1193845us) service_pool.cc:224] Handling call
0709 14:54:20.755388 (+   696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.833164 (+ 77776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.833394 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.756448  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.760746  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:20.836660  3542 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 14:54:20.726460  9016 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:20.763103  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.763046  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 120460us
I20260709 14:54:20.775674  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.710763  9025 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.780390  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.782684  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.801463  9104 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:20.677920  9096 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:20.755234  9071 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.005s	sys 0.031s
W20260709 14:54:20.862445  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86855.4us
I20260709 14:54:20.739074  9008 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:20.788766  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.787974  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.760061  9149 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 14:54:20.799430  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 302720us
I20260709 14:54:20.839354  9115 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:20.800578  9045 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.643s	user 0.010s	sys 0.003s
I20260709 14:54:20.717738  2759 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:20.800992  2685 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
W20260709 14:54:20.802470  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 97324.6us
I20260709 14:54:20.875943  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:20.877152  2680 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.804774  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:20.806577  2677 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:20.808151  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.814123  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1711) took 1578 ms. Trace:
I20260709 14:54:20.879364  2607 rpcz_store.cc:276] 0709 14:54:19.235559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.109505 (+873946us) service_pool.cc:224] Handling call
0709 14:54:20.604004 (+494499us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:20.732536 (+128532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.813897 (+ 81361us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54002ff3d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.814054 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040306048}
I20260709 14:54:20.804948  2760 rpcz_store.cc:276] 0709 14:54:18.913085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.330933 (+1417848us) service_pool.cc:224] Handling call
0709 14:54:20.331517 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.332002 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.332066 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:20.332745 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 14:54:20.396171 (+ 63426us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:20.396223 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.396336 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:20.590554 (+194218us) raft_consensus.cc:1685] finished
0709 14:54:20.590580 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.590746 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9828,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":135,"thread_start_us":62736,"threads_started":1,"wal-append.queue_time_us":143402}
I20260709 14:54:20.807140  9098 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.786414  9112 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:20.819599  2753 rpcz_store.cc:276] 0709 14:54:18.703467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.303344 (+1599877us) service_pool.cc:224] Handling call
0709 14:54:20.303844 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.714915 (+411071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.720720 (+  5805us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.726999 (+  6279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660022016}
I20260709 14:54:20.861933  9153 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:20.825258  9069 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.787936  8702 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:54:20.924662  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1634) took 1793 ms (client timeout 1909 ms). Trace:
W20260709 14:54:20.925283  2686 rpcz_store.cc:269] 0709 14:54:19.131338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.407134 (+275796us) service_pool.cc:224] Handling call
0709 14:54:20.924419 (+1517285us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:20.924584 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3073080576}
I20260709 14:54:20.831607  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:20.927179  2682 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:20.776674  9116 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:20.928630  9116 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.751165  9056 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.932370  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:20.932780  9056 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.838989  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.933422  2757 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.792683  9028 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:20.934387  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.935099  2754 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.935738  2603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:20.936436  2603 raft_consensus.cc:606] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:20.937067  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1819) took 2237 ms. Trace:
I20260709 14:54:20.937460  2754 rpcz_store.cc:276] 0709 14:54:18.699631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.266221 (+1566590us) service_pool.cc:224] Handling call
0709 14:54:20.266683 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.935590 (+668907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.936798 (+  1208us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.936998 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536721792}
I20260709 14:54:20.797096  9027 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.849633  3547 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:20.844504  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:20.849452  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86452.6us
I20260709 14:54:20.772856  9020 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 14:54:20.792857  9132 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:20.801558  8652 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:20.803403  9133 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:20.881117  8871 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:20.964702  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111230us
I20260709 14:54:20.965488  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.854550  3537 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.851126  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:20.965999  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:20.830199  8978 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:20.802657  9169 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:20.787108  2678 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:20.966923  3546 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:20.863632  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:20.863998  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83943.2us
I20260709 14:54:20.978132  2726 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.799899  9168 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:20.872300  3540 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:20.875823  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1826) took 1315 ms. Trace:
I20260709 14:54:20.984728  2759 rpcz_store.cc:276] 0709 14:54:19.560534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.704406 (+1143872us) service_pool.cc:224] Handling call
0709 14:54:20.717513 (+ 13107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.875455 (+157942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.875743 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:20.803200  8910 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.717674  2610 rpcz_store.cc:276] 0709 14:54:18.797514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:19.846330 (+1048816us) service_pool.cc:224] Handling call
0709 14:54:19.902852 (+ 56522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.531871 (+629019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.598351 (+ 66480us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54003c8554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:20.598549 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444565760}
I20260709 14:54:20.894048  9153 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:20.888636  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:20.814297  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:20.883867  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.006932  2758 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:21.008078  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1692) took 1335 ms. Trace:
I20260709 14:54:21.008630  2758 rpcz_store.cc:276] 0709 14:54:19.672696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.006038 (+1333342us) service_pool.cc:224] Handling call
0709 14:54:21.006606 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.007647 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.007983 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:20.884600  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76530.2us
I20260709 14:54:21.010594  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.886632  2681 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:20.824960  9182 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 14:54:20.791229  8921 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:21.012239  8921 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:20.895416  2609 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: []
I20260709 14:54:20.833753  9109 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.013947  9168 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:21.014541  9168 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.014920  9168 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:21.015102  9109 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.015780  9109 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 14:54:21.016721  8921 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:21.017532  2610 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:21.018366  9098 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:20.953975  8652 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:21.023079  9055 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:21.023886  2759 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:21.024693  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1694) took 1312 ms. Trace:
I20260709 14:54:21.024926  9025 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:21.025055  2759 rpcz_store.cc:276] 0709 14:54:19.712085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.023166 (+1311081us) service_pool.cc:224] Handling call
0709 14:54:21.023636 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.024405 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.024624 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:21.026450  9069 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.026945  9153 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:20.898025  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.029183  8921 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.868351  9008 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:21.030741  2612 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:20.985116  8871 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:21.032661  9152 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.005s	user 0.015s	sys 0.000s
I20260709 14:54:21.033561  8871 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:20.904383  9184 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:20.893402  9064 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 14:54:21.034193  9184 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:20.880254  9154 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 14:54:20.911402  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:21.035434  9154 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:21.035645  2608 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:21.035849  9154 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:21.037448  8708 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:20.913992  9019 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:20.992512  9056 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:20.856061  9078 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 14:54:21.039152  9078 consensus_queue.cc:1243] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 14:54:21.040123  9019 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:20.849798  9003 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 14:54:20.883059  9045 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.044281  9045 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:20.919939  9177 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 14:54:21.044927  8652 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.045960  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1828) took 1484 ms. Trace:
I20260709 14:54:20.854060  8578 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 14:54:20.926496  2679 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:21.046437  2756 rpcz_store.cc:276] 0709 14:54:19.561146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.766254 (+1205108us) service_pool.cc:224] Handling call
0709 14:54:20.766816 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:20.861624 (+ 94808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.861659 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:20.861707 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:20.861751 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.045649 (+183898us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:21.045711 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:21.045889 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214498560}
I20260709 14:54:21.048189  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1655) took 1863 ms. Trace:
I20260709 14:54:21.048359  3545 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:21.048677  2756 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:54:21.048600  2679 rpcz_store.cc:276] 0709 14:54:19.184713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.305591 (+1120878us) service_pool.cc:224] Handling call
0709 14:54:20.926262 (+620671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.047279 (+121017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.047975 (+   696us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400444b50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:21.048120 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426561792}
I20260709 14:54:21.050379  2756 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:20.840202  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:20.883165  9185 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:21.051064  2605 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:21.051371  2755 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.904820  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:20.947875  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:20.949741  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:21.052472  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 238280us
I20260709 14:54:20.881732  9115 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:20.939853  2751 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:21.056125  2758 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:21.058024  2759 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:20.951807  3536 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:54:21.058804  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1829) took 1105 ms. Trace:
I20260709 14:54:20.934387  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:20.938740  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:20.948011  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 106645us
I20260709 14:54:21.063328  2757 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:21.063529  2754 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:20.806022  9163 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:20.893842  9038 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:20.948547  9027 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:20.866616  9096 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:20.971200  9132 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:20.974314  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1156) took 1046 ms. Trace:
I20260709 14:54:21.083789  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:21.083024  2611 rpcz_store.cc:276] 0709 14:54:19.927707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.570046 (+642339us) service_pool.cc:224] Handling call
0709 14:54:20.643786 (+ 73740us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:20.674103 (+ 30317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.674149 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:20.701961 (+ 27812us) log.cc:844] Serialized 52 byte log entry
0709 14:54:20.836735 (+134774us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:20.836784 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.836859 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:20.974010 (+137151us) raft_consensus.cc:1685] finished
0709 14:54:20.974035 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:20.974235 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25421,"raft.queue_time_us":32498,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":22631,"thread_start_us":121220,"threads_started":1,"wal-append.queue_time_us":186879}
I20260709 14:54:20.794625  9095 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:20.888754  9012 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.084563  3546 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:20.986500  2682 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:21.086212  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 75794.3us
I20260709 14:54:20.988886  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:21.087180  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:21.087514  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 98795.5us
I20260709 14:54:20.986862  2724 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:21.088383  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:20.871289  3411 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.753s (SENT)
I20260709 14:54:21.068553  8885 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:54:21.089638  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 218605us
I20260709 14:54:21.092717  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.094285  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1155) took 1167 ms. Trace:
I20260709 14:54:20.995672  8702 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:21.096228  9168 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:20.964020  9107 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.690s	user 0.013s	sys 0.001s
I20260709 14:54:21.097343  9107 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:21.098775  9056 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:21.099381  9056 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:21.099787  9056 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:21.100031  9107 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:21.000025  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 136179us
I20260709 14:54:21.105257  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.105468  9190 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:21.036909  9154 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.033316  9152 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.107147  9154 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:21.107297  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 120584us
I20260709 14:54:21.050096  9145 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:21.108743  9152 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.109047  9145 consensus_queue.cc:1243] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:21.109354  9016 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:21.109519  9152 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:21.110029  9152 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:21.110097  9016 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:21.110522  9152 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:20.872066  8894 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:20.983484  9114 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:21.113114  9186 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:21.053498  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.058434  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1689) took 1441 ms. Trace:
I20260709 14:54:21.114462  2751 rpcz_store.cc:276] 0709 14:54:19.616671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.939292 (+1322621us) service_pool.cc:224] Handling call
0709 14:54:20.939639 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.058117 (+118478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.058369 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:21.111296  2683 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:21.005962  9135 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:21.116247  9135 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:21.117354  9135 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:21.057426  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 109639us
I20260709 14:54:21.060883  8652 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:21.062940  2759 rpcz_store.cc:276] 0709 14:54:19.953560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.026182 (+1072622us) service_pool.cc:224] Handling call
0709 14:54:21.026591 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.058521 (+ 31930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.058746 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:21.094794  2609 rpcz_store.cc:276] 0709 14:54:19.927395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.539761 (+612366us) service_pool.cc:224] Handling call
0709 14:54:20.895128 (+355367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.012841 (+117713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.094019 (+ 81178us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540016dad0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:21.094192 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816379776}
I20260709 14:54:20.983192  9169 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.056362  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:21.121879  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:21.124104  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.096064  9141 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.106526  9190 consensus_queue.cc:1243] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:20.835444  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:20.912465  9179 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 14:54:21.111130  9004 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:21.111793  2606 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:21.111260  9016 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.112291  9041 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:21.112218  8702 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:21.113617  9095 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:21.128731  2726 leader_election.cc:400] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14.
I20260709 14:54:21.135337  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:21.137084  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:21.188028  3540 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:21.186614  3546 pending_rounds.cc:85] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 4
I20260709 14:54:21.188875  3546 pending_rounds.cc:107] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 14:54:21.189540  3546 raft_consensus.cc:2889] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:21.190099  3546 pending_rounds.cc:107] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 14:54:21.190433  3546 raft_consensus.cc:2889] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:21.143846  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90512.8us
I20260709 14:54:21.144707  2605 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:21.060693  9098 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:21.195439  3542 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:21.152644  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.118657  9135 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:21.154726  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 158419us
W20260709 14:54:21.202138  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49653.1us
I20260709 14:54:21.159623  2610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "963a4155717b4274b0121341056b7cb8"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:21.044349  9112 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.119293  8910 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:21.188949  2726 leader_election.cc:403] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:54:21.143134  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54835.8us
I20260709 14:54:21.154093  8871 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:21.144094  9169 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:21.075222  9185 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:21.203068  2610 raft_consensus.cc:606] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.245707  2610 raft_consensus.cc:614] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:21.203646  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:21.247452  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43863.9us
I20260709 14:54:21.155541  9045 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:21.139489  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46856.6us
I20260709 14:54:21.228363  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:21.230755  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:21.231925  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:21.159354  9021 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 14:54:21.236029  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107558us
I20260709 14:54:21.259630  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:21.244364  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 245993us
W20260709 14:54:21.254369  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51897us
I20260709 14:54:21.195603  9098 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.213217  9141 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:21.037472  8921 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:21.161018  9038 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:21.153896  9104 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
W20260709 14:54:21.274070  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 61357us
I20260709 14:54:21.213492  8885 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:21.283244  8885 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:21.283070  8537 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 14:54:21.243986  8894 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:21.284333  8885 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.285264  2723 leader_election.cc:400] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:21.285707  2723 leader_election.cc:403] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 14:54:21.185894  9041 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.286948  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.287341  9150 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:21.288432  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.289224  9041 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.289903  9097 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:21.290417  9168 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.280577  9185 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:21.290740  9097 consensus_queue.cc:1243] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:21.290851  9041 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18.
I20260709 14:54:21.269486  9020 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 14:54:21.326756  9020 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:21.327212  9186 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:21.327247  9020 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:21.327919  9186 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:21.329211  9186 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:21.330405  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47771.1us
I20260709 14:54:21.147183  9184 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:21.331297  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:21.331856  9184 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:21.209554  8708 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 14:54:21.293800  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1825) took 1733 ms. Trace:
I20260709 14:54:21.159300  9004 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:21.307562  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1638) took 1261 ms. Trace:
I20260709 14:54:21.362236  2684 rpcz_store.cc:276] 0709 14:54:20.046171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.542166 (+495995us) service_pool.cc:224] Handling call
0709 14:54:20.542617 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.542980 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.543013 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:20.543924 (+   911us) log.cc:844] Serialized 26 byte log entry
0709 14:54:20.680533 (+136609us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:20.680592 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:20.680695 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:21.307065 (+626370us) raft_consensus.cc:1685] finished
0709 14:54:21.307108 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:21.307446 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":135735,"threads_started":1,"wal-append.queue_time_us":232203}
I20260709 14:54:21.256073  9045 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.325644  9041 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:21.346095  8702 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.258534  8910 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 14:54:21.243836  9193 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 14:54:21.351575  2752 rpcz_store.cc:276] 0709 14:54:19.560210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.639669 (+1079459us) service_pool.cc:224] Handling call
0709 14:54:20.640123 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.711426 (+ 71303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.711488 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:20.731230 (+ 19742us) log.cc:844] Serialized 26 byte log entry
0709 14:54:21.157632 (+426402us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:21.157681 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.157772 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:21.293518 (+135746us) raft_consensus.cc:1685] finished
0709 14:54:21.293545 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:21.293725 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73925,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":11143,"thread_start_us":445859,"threads_started":2,"wal-append.queue_time_us":514771}
W20260709 14:54:21.358814  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99292.6us
I20260709 14:54:21.213866  9028 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 14:54:21.274973  9098 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:21.375332  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:21.374581  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:21.384591  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96302.9us
I20260709 14:54:21.239893  9049 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:21.393464  2684 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:21.361500  9095 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:21.402572  9049 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:21.275584  9199 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
I20260709 14:54:21.286978  9149 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 14:54:21.407742  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.373369  9045 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:21.394902  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.411242  2604 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:21.397886  9071 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:21.371922  9041 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.416363  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:21.417486  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:21.420588  2684 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 16: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:21.418259  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.421440  9041 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:21.412036  9096 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.425839  9199 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:21.426401  9199 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:21.427399  9199 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:21.428462  9004 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:21.428990  9199 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:21.428802  9020 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.430505  9020 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:21.456842  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:21.485549  2757 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 14.
W20260709 14:54:21.433907  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:21.505650  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:21.507213  2608 raft_consensus.cc:1217] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Deduplicated request from leader. Original: 12.8->[13.9-15.10]   Dedup: 13.9->[15.10-15.10]
I20260709 14:54:21.508045  2608 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 14:54:21.435119  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.436709  8578 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:21.438145  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:21.520922  3545 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 14:54:21.521412  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83388.7us
W20260709 14:54:21.457580  9196 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 14:54:21.342024  9192 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 14:54:21.456956  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.447829  8803 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:21.472954  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.463157  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.392328  9069 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
W20260709 14:54:21.452392  2575 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.909s (SENT) [suppressed 21 similar messages]
I20260709 14:54:21.459542  2605 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 14:54:21.464745  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.547227  2753 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:21.550357  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:21.450102  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1644) took 1038 ms. Trace:
I20260709 14:54:21.456475  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:21.455379  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:21.559151  2683 rpcz_store.cc:276] 0709 14:54:20.411648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.545983 (+134335us) service_pool.cc:224] Handling call
0709 14:54:20.546346 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.115922 (+569576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.449798 (+333876us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400269154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:21.450022 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298789120}
I20260709 14:54:21.560379  2607 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:21.487525  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.561384  2751 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:21.432833  9152 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.487004  2759 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:21.511938  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:21.511839  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:21.515439  2603 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 14:54:21.564013  3538 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:21.516923  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:21.378086  9064 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 14:54:21.407378  9095 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:21.522373  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:21.568301  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46047.3us
I20260709 14:54:21.569182  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.443594  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1866) took 1188 ms. Trace:
I20260709 14:54:21.571025  2606 rpcz_store.cc:276] 0709 14:54:20.255441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.275004 (+ 19563us) service_pool.cc:224] Handling call
0709 14:54:21.443304 (+1168300us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:21.443502 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663043456}
I20260709 14:54:21.524120  2756 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 14:54:21.485491  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:21.456103  9185 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:21.574815  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:21.419705  9071 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:21.393057  9133 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 14:54:21.595381  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 177174us
W20260709 14:54:21.595587  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 143227us
I20260709 14:54:21.533651  2682 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:21.457468  9205 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:21.534435  2681 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:54:21.450028  8978 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.527566  9097 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:21.488787  9004 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:21.486089  9011 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 14:54:21.532827  8803 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:21.698565  8910 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:21.699832  8910 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.701715  9152 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.704624  8910 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.711437  9004 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:21.460407  9012 consensus_queue.cc:1175] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 18 status { last_received { term: 17 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:54:21.455826  9206 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:21.561216  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:21.454098  2760 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:21.565339  2609 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:21.573194  2684 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:21.578557  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.577049  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:21.589879  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.446636  9041 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.456305  9115 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:21.573776  9153 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
W20260709 14:54:21.672101  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71150.2us
I20260709 14:54:21.449026  9008 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:21.610225  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:21.599615  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49158.3us
I20260709 14:54:21.676795  2752 pending_rounds.cc:85] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 8
I20260709 14:54:21.571586  9168 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:54:21.695138  9203 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 14:54:21.695442  8978 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.696175  9011 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:21.576609  9095 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:21.576333  9104 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:21.779667  2751 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:21.780462  2752 pending_rounds.cc:107] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:54:21.780942  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.792109  2752 raft_consensus.cc:2889] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:21.591226  9150 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:21.783457  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.809320  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:21.784789  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 195032us
I20260709 14:54:21.811074  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.781987  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.782194  2760 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:21.782804  2682 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:21.595283  9133 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:21.813854  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.576864  9190 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 13. Current term is 14.
I20260709 14:54:21.814757  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:21.825378  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43491.5us
I20260709 14:54:21.579492  9207 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:21.818862  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.827625  2758 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 14:54:21.828147  2751 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:21.828791  3538 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:21.813925  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.820567  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.830060  2682 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:21.817154  2760 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:21.822561  2684 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:21.823496  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:21.823911  2678 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:21.824474  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.835618  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1181) took 1259 ms. Trace:
I20260709 14:54:21.781188  9168 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:21.606585  9055 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.213s
I20260709 14:54:21.836273  2685 rpcz_store.cc:276] 0709 14:54:20.575936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.894748 (+318812us) service_pool.cc:224] Handling call
0709 14:54:20.901761 (+  7013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:20.902433 (+   672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:20.902488 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:20.903398 (+   910us) log.cc:844] Serialized 26 byte log entry
0709 14:54:21.007994 (+104596us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:21.442403 (+434409us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b18003941e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf
0709 14:54:21.442604 (+   201us) log.cc:844] Serialized 14 byte log entry
0709 14:54:21.456782 (+ 14178us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:21.477932 (+ 21150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.478078 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:21.835345 (+357267us) raft_consensus.cc:1685] finished
0709 14:54:21.835365 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:21.835543 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":333388,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4654,"raft.queue_time_us":22008,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":252,"spinlock_wait_cycles":55557632,"thread_start_us":292349,"threads_started":2,"wal-append.queue_time_us":283596}
I20260709 14:54:21.714658  9004 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:21.607398  9131 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.214s
I20260709 14:54:21.811647  9011 consensus_queue.cc:1243] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:21.835819  9150 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:21.842665  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 296427us
I20260709 14:54:21.854156  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1868) took 1100 ms. Trace:
I20260709 14:54:21.809847  8578 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:21.844406  2756 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:21.824357  9021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.034s	sys 0.082s
I20260709 14:54:21.671231  9049 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.153s
I20260709 14:54:21.848075  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:21.879676  2753 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:21.880580  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.881209  2753 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:21.882294  2753 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:21.821580  9078 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:21.825882  9141 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
W20260709 14:54:21.833122  9169 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.005s
I20260709 14:54:21.854666  2604 rpcz_store.cc:276] 0709 14:54:20.754200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.757183 (+  2983us) service_pool.cc:224] Handling call
0709 14:54:21.853907 (+1096724us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:21.854080 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250153728}
I20260709 14:54:21.857064  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:21.859586  2682 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:21.888975  2682 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:21.894132  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.831622  9205 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.850419  9112 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:21.860397  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.861485  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1830) took 1524 ms. Trace:
I20260709 14:54:21.718284  9185 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 14:54:21.869503  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:21.870579  2685 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:21.875149  2677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:54:21.842797  9192 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 14:54:21.794039  9153 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:21.871616  9060 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.163s	user 0.014s	sys 0.003s
I20260709 14:54:21.860539  2751 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:21.918530  2682 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 14:54:21.936987  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:21.990350  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1183) took 1381 ms. Trace:
I20260709 14:54:21.990986  2678 rpcz_store.cc:276] 0709 14:54:20.608609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.976386 (+367777us) service_pool.cc:224] Handling call
0709 14:54:20.976860 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.834373 (+857513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:21.990068 (+155695us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400377154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:21.990280 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947572992}
W20260709 14:54:21.991539  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54722.8us
I20260709 14:54:21.992641  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:21.990195  2677 raft_consensus.cc:606] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:21.993898  2677 raft_consensus.cc:614] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:21.989517  2751 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:54:21.994570  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:21.902282  2684 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:21.996019  2751 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:21.997191  2684 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:21.997663  2605 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:21.999460  2759 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:21.999557  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:22.000304  2755 rpcz_store.cc:276] 0709 14:54:20.337039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.623669 (+286630us) service_pool.cc:224] Handling call
0709 14:54:21.861287 (+1237618us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:21.861423 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711195392}
I20260709 14:54:21.990365  2682 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 14:54:21.897261  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:22.003280  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:21.878616  9056 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.778s	user 0.012s	sys 0.004s
I20260709 14:54:22.014431  2756 raft_consensus.cc:2410] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 14:54:21.884037  9027 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.732s	user 0.013s	sys 0.000s
W20260709 14:54:22.029052  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 432737us
W20260709 14:54:21.969012  9098 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.004s
I20260709 14:54:22.037894  9152 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.055603  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.041203  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:22.032483  2603 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:22.056449  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:22.056603  2754 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:22.061241  2686 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:21.874505  9021 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:21.964031  9196 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 14:54:22.055137  2608 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:22.052400  2685 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
W20260709 14:54:22.068112  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 170932us
I20260709 14:54:21.905596  9169 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.046237  9056 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:22.088653  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47554.7us
I20260709 14:54:22.039742  9060 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.014544  9055 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:22.097061  9206 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:22.029222  8978 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.029173  9131 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:22.120232  8871 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.848s	user 0.009s	sys 0.004s
I20260709 14:54:21.921651  9116 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.151145  8871 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:22.151312  9116 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.152650  3547 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:22.154284  9218 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 14:54:22.156311  9205 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:22.082319  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:22.086236  8708 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 14:54:22.086090  9152 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:21.883849  9044 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.180s
I20260709 14:54:22.047089  9221 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:21.904846  9038 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.187s
I20260709 14:54:22.073530  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:22.164394  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 91024.8us
I20260709 14:54:22.123283  8894 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:22.119000  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:22.085515  9185 consensus_queue.cc:1243] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:22.125200  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:22.098836  9194 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:22.093381  9149 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.091990  9011 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:22.014731  8537 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.130530  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 194902us
I20260709 14:54:22.149276  9008 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:22.166041  9044 consensus_queue.cc:1243] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:22.045830  9027 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:22.029719  8578 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:22.149242  3543 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:22.126674  9209 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 14:54:22.065003  9098 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:21.960145  9199 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:21.926512  9168 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.161315  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:22.245913  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84773.3us
W20260709 14:54:22.241420  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 169521us
I20260709 14:54:21.896252  9078 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:21.904908  9132 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.190s
I20260709 14:54:22.167793  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:22.136428  9115 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:22.172308  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53477.4us
W20260709 14:54:22.173447  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48372.9us
I20260709 14:54:22.263000  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:22.175822  9112 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:22.141366  9041 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:22.150573  9207 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.183645  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 180403us
I20260709 14:54:22.274052  2679 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:22.183844  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 184393us
I20260709 14:54:22.148518  8978 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.190692  2608 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 14:54:22.193079  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:22.117400  9219 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 14:54:22.195367  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.196658  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:21.887562  9133 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.197172  2611 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:22.171657  8910 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:22.170372  8894 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:22.240365  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 133881us
I20260709 14:54:22.221067  8708 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 14:54:22.191609  9011 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:22.174139  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.265151  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:22.223489  9021 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:22.198661  9194 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:22.257076  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:22.271173  3542 raft_consensus.cc:2368] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17.
W20260709 14:54:22.319821  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45455.7us
I20260709 14:54:22.240098  8578 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:22.283066  9230 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:22.291054  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:22.291802  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 95266.4us
I20260709 14:54:22.331947  9069 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:22.332473  9069 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:22.332846  9069 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:22.333577  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:22.292392  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:22.338326  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:22.282465  9228 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:22.187539  9185 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:22.290006  9168 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:22.217322  8537 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.257529  9131 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:22.265851  9141 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:22.306061  2759 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 14:54:22.306803  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:22.308084  2751 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:22.195711  9149 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.303923  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:22.273674  9155 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:22.364069  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.273640  9064 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.320883  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63908.3us
I20260709 14:54:22.283057  9207 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.335088  9115 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:22.281908  9214 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:22.331607  3545 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:22.442299  3536 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 14:54:22.341178  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:22.342000  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 77010.4us
I20260709 14:54:22.294265  9019 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:22.443747  9019 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:22.444062  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:22.442489  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:22.446817  3545 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:22.448040  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:22.450165  3545 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:22.254197  9199 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.453132  9199 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:22.309082  9011 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:22.300850  9170 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:22.357110  2610 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:22.366860  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:22.367105  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:22.367480  9141 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:22.455431  9141 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:22.368880  3540 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:22.462018  3540 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:22.279673  2604 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:22.463896  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.464918  2604 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:22.374271  9098 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:22.377485  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 209785us
I20260709 14:54:22.466229  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:22.290303  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:22.466784  2755 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:22.467464  3538 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:22.434396  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:22.330962  9230 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:22.350608  9112 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:22.293893  9153 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:22.469753  2759 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:22.488471  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54194.9us
I20260709 14:54:22.445113  9019 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:22.476485  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:22.369541  9206 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.479065  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:22.434271  9016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.026s	sys 0.014s
I20260709 14:54:22.382015  9225 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.503677  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:22.504356  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60426.3us
I20260709 14:54:22.504535  3545 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:54:22.501596  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 205525us
W20260709 14:54:22.475226  9154 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.367s	user 0.012s	sys 0.005s
I20260709 14:54:22.516115  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:22.502763  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 180865us
I20260709 14:54:22.519685  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:22.522810  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.527014  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:22.503446  9016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.026s	sys 0.014s
I20260709 14:54:22.533877  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:22.479060  9056 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:22.491183  9199 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:22.536864  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:22.479346  9169 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:22.499728  9060 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:22.542806  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.543148  9235 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:22.543277  9149 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:22.553315  9153 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:22.543668  9211 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.411s	user 0.000s	sys 0.012s
I20260709 14:54:22.547953  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:22.543788  9207 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:22.521559  9008 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:22.364148  9228 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.465469  9064 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:22.552129  9227 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 14:54:22.552285  8978 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:22.569643  9232 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.570254  9224 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.363s	user 0.001s	sys 0.001s
W20260709 14:54:22.573251  8652 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.426s	user 0.012s	sys 0.003s
I20260709 14:54:22.575955  9153 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:22.569648  9216 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.322s	user 0.000s	sys 0.002s
I20260709 14:54:22.581669  2759 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:22.469738  9214 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.507624  9170 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 14:54:22.515476  9055 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 14:54:22.604907  9008 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:22.578678  9184 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.246s	user 0.017s	sys 0.000s
W20260709 14:54:22.578675  9095 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.002s
I20260709 14:54:22.575421  9149 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:22.545262  9152 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:22.590389  9222 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.398s	user 0.000s	sys 0.002s
I20260709 14:54:22.611850  9027 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:22.525612  8578 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.648347  2759 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:22.588210  9199 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:22.619304  2604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:22.651505  2604 raft_consensus.cc:606] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:22.651954  2604 raft_consensus.cc:614] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 14:54:22.625721  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 157267us
I20260709 14:54:22.653885  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:22.536165  9229 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.617929  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 84205.7us
W20260709 14:54:22.625866  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 71316.1us
I20260709 14:54:22.521224  9225 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:22.630992  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 154413us
W20260709 14:54:22.617029  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 275242us
I20260709 14:54:22.598428  2608 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
W20260709 14:54:22.640126  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 173976us
I20260709 14:54:22.662930  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:22.516331  9206 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:22.643955  2680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.004s	sys 0.014s
W20260709 14:54:22.664611  2680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.004s	sys 0.014s
I20260709 14:54:22.545038  9131 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.627089  8652 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.707316  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:22.707798  9198 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.382s	user 0.003s	sys 0.001s
I20260709 14:54:22.708295  2685 raft_consensus.cc:2368] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:54:22.707885  8652 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:22.709561  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:22.710351  8652 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:22.710577  2677 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 14:54:22.529903  9194 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.609191  9169 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 14:54:22.575845  8708 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 14:54:22.632130  9008 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:22.660965  9149 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:22.650362  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:22.655334  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:22.639109  9219 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:22.727831  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65062.4us
I20260709 14:54:22.666368  2608 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 14:54:22.643492  9112 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:22.588809  9235 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:22.744267  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1197) took 1252 ms. Trace:
I20260709 14:54:22.754245  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:22.742444  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1196) took 1274 ms. Trace:
W20260709 14:54:22.746975  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83731us
W20260709 14:54:22.751416  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96167.3us
I20260709 14:54:22.754393  2752 rpcz_store.cc:276] 0709 14:54:21.492134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.567813 (+ 75679us) service_pool.cc:224] Handling call
0709 14:54:21.676527 (+108714us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:21.812422 (+135895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:21.812459 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:21.813116 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 14:54:22.030552 (+217436us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:22.030606 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:22.031304 (+   698us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:22.743974 (+712670us) raft_consensus.cc:1685] finished
0709 14:54:22.744016 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:22.744199 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1243,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6267,"thread_start_us":187971,"threads_started":1,"wal-append.queue_time_us":319435}
W20260709 14:54:22.644022  2678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.010s	sys 0.008s
I20260709 14:54:22.689819  9250 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:22.690457  9237 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:22.691102  9249 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:22.691542  9226 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:22.691890  9199 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:22.685411  9170 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 14:54:22.695201  8537 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:22.696357  9228 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:22.696928  9244 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:22.697690  9245 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:22.623657  9232 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.699864  9064 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:22.700726  9135 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.487s	user 0.008s	sys 0.006s
I20260709 14:54:22.639219  9095 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.647114  9078 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:22.703694  9020 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.179s	user 0.010s	sys 0.004s
I20260709 14:54:22.703416  9261 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:22.665830  9229 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:22.711611  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:22.706660  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:22.713820  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:22.641873  8910 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:22.726572  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1385) took 1570 ms. Trace:
W20260709 14:54:22.698709  9045 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.214s	user 0.011s	sys 0.001s
W20260709 14:54:22.694643  9097 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.972s	user 0.006s	sys 0.005s
I20260709 14:54:22.667091  9152 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 14:54:22.757551  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:22.688858  9246 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.108s
W20260709 14:54:22.758237  2678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.011s	sys 0.008s
W20260709 14:54:22.802238  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146885us
I20260709 14:54:22.901094  2677 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:22.901139  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1653) took 1618 ms. Trace:
I20260709 14:54:22.746536  2575 leader_election.cc:400] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:54:22.756107  2757 rpcz_store.cc:276] 0709 14:54:21.468448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.533022 (+ 64574us) service_pool.cc:224] Handling call
0709 14:54:21.533415 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:21.718550 (+185135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:21.718609 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:21.719426 (+   817us) log.cc:844] Serialized 26 byte log entry
0709 14:54:21.968845 (+249419us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:22.225696 (+256851us) log.cc:844] Serialized 14 byte log entry
0709 14:54:22.243285 (+ 17589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:22.741822 (+498537us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400571054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:22.741893 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:22.741965 (+    72us) raft_consensus.cc:1685] finished
0709 14:54:22.742102 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:22.742319 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":313488,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":8729,"raft.queue_time_us":19175,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":14024,"spinlock_wait_cycles":424342784,"thread_start_us":419897,"threads_started":2,"wal-append.queue_time_us":420037}
W20260709 14:54:22.752701  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:22.902673  2575 leader_election.cc:403] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 14:54:22.782464  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1649) took 1501 ms. Trace:
I20260709 14:54:22.901726  2678 rpcz_store.cc:276] 0709 14:54:21.282612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.992057 (+709445us) service_pool.cc:224] Handling call
0709 14:54:21.992492 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:21.992979 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:21.993008 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:22.029259 (+ 36251us) log.cc:844] Serialized 26 byte log entry
0709 14:54:22.900708 (+871449us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:22.900760 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:22.900848 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:22.900900 (+    52us) raft_consensus.cc:1685] finished
0709 14:54:22.900911 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:22.901073 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8775,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":25008,"thread_start_us":791154,"threads_started":1,"wal-append.queue_time_us":629667}
I20260709 14:54:22.710357  9214 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:22.728147  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74336.5us
I20260709 14:54:22.791271  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:22.908066  2751 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:22.905740  2752 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:22.728508  9252 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.292s
W20260709 14:54:22.730711  9163 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.611s	user 0.000s	sys 0.015s
I20260709 14:54:22.956502  3537 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 14:54:22.957113  3537 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 8
I20260709 14:54:22.957254  2609 raft_consensus.cc:2368] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:22.957418  3537 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:54:22.957865  3537 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:22.953388  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 240383us
W20260709 14:54:22.794615  9150 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.923s	user 0.013s	sys 0.002s
I20260709 14:54:22.922982  2757 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:22.983263  2757 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 14:54:22.987246  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:22.903254  2680 rpcz_store.cc:276] 0709 14:54:21.281344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.930926 (+649582us) service_pool.cc:224] Handling call
0709 14:54:21.985075 (+ 54149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.985429 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:22.064512 (+ 79083us) log.cc:844] Serialized 14 byte log entry
0709 14:54:22.782019 (+717507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:22.782086 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:22.782162 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:22.782245 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:22.782396 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116605,"prepare.queue_time_us":71908,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5419,"raft.queue_time_us":121208,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":572312,"threads_started":1,"wal-append.queue_time_us":721086}
I20260709 14:54:22.742969  9132 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:22.903438  3546 rpcz_store.cc:276] 0709 14:54:21.159077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.185764 (+ 26687us) service_pool.cc:224] Handling call
0709 14:54:21.186309 (+   545us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:21.242187 (+ 55878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:21.242237 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:21.257997 (+ 15760us) log.cc:844] Serialized 78 byte log entry
0709 14:54:21.802151 (+544154us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:22.027561 (+225410us) log.cc:844] Serialized 14 byte log entry
0709 14:54:22.029545 (+  1984us) log.cc:844] Serialized 14 byte log entry
0709 14:54:22.225332 (+195787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:22.225425 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:22.718233 (+492808us) raft_consensus.cc:1685] finished
0709 14:54:22.718261 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:22.726484 (+  8223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6036,"prepare.queue_time_us":432589,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":4024,"raft.queue_time_us":124179,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":26023,"thread_start_us":800441,"threads_started":3,"wal-append.queue_time_us":652967}
I20260709 14:54:22.960959  2608 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:22.731904  9169 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:22.961323  2678 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:22.983917  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:22.925091  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:22.784461  9250 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:22.962425  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 486001us
I20260709 14:54:22.955080  9228 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:22.911516  9155 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:22.784505  9237 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:22.953370  9229 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:23.033023  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:22.955014  9095 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:22.794690  9242 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.059187  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:23.009820  9153 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:23.003973  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 79036.8us
W20260709 14:54:22.956758  8894 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.003s
I20260709 14:54:23.062819  2752 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:22.909701  9112 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.007673  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:22.740723  9027 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:22.732100  9238 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.269s
I20260709 14:54:22.954133  9232 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:22.751641  9219 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.794863  8978 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:22.795358  9256 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:22.979813  2683 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:22.982790  8537 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:22.905827  9244 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:22.942577  9097 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:23.061151  9078 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:23.005987  9132 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.759353  9257 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:23.103982  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:22.811352  9241 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.323s
I20260709 14:54:22.729135  9011 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:22.797730  9141 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:22.738109  9069 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.015849  9020 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:23.021533  9016 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 14:54:23.029202  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:23.001324  9150 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:22.909202  9045 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.005600  9169 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.055536  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:23.003273  3546 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:23.021453  9135 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:23.067478  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 59914.5us
I20260709 14:54:22.736984  9149 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.054013  9237 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:23.084043  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:22.954159  8910 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.013422  9249 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:23.011487  9250 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:22.793051  9253 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.282s
I20260709 14:54:23.022778  9240 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 14:54:23.088228  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:23.090065  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 332604us
I20260709 14:54:23.018448  2751 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:23.046007  9056 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 14:54:23.097880  9262 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.075266  9152 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.069468  9027 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:22.985558  9163 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.072201  9261 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:22.900303  9226 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.089473  9244 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:23.072289  8894 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:23.109136  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 201210us
I20260709 14:54:23.105985  9078 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.102257  8537 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:23.126768  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.124598  2680 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:23.120792  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65388.9us
W20260709 14:54:23.116736  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 87689.1us
I20260709 14:54:23.131079  9225 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.117894  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.079586  9242 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.185595  2685 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:23.087859  8978 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.188606  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:23.150622  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59100.5us
I20260709 14:54:23.072189  9112 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.129696  9249 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:23.125728  9064 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:23.142127  9194 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 14:54:23.175825  2759 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:23.145596  9027 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.136642  2608 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:23.182644  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:23.184130  2604 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:23.128571  9135 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.122447  9263 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:23.149549  9241 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 14:54:23.146862  9152 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.178791  8885 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:23.207062  2752 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.158995  9008 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.207334  2682 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:23.182489  9021 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:23.187645  9238 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:23.184928  9235 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:23.212900  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61027.2us
W20260709 14:54:23.208688  9266 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 14:54:23.199970  9020 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:23.178951  9095 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 14:54:23.203293  9246 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.201938  9249 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:23.184175  9261 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 14:54:23.198588  9251 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.242656  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:23.198546  9258 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 14:54:23.209280  9263 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:23.150934  9262 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:23.254236  9265 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 14:54:23.254539  9135 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.255146  8978 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:23.108650  9256 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:23.258320  9268 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:54:23.259090  9045 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.258016  8537 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.261860  9152 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.263703  9141 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.214300  8885 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:23.165845  9169 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.217693  9064 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.239544  9011 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.237273  9225 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:23.188032  9271 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 14:54:23.238111  9238 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:23.284821  9230 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.264511  9112 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.269609  9078 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.297817  2608 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:23.311252  9069 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:23.290570  9235 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:23.287580  9263 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:23.324272  9163 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.308349  9132 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.267789  9246 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.243462  9021 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.332896  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.217270  9275 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 14:54:23.318038  2752 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:23.273386  9251 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.329449  9169 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.342293  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1540) took 1050 ms. Trace:
I20260709 14:54:23.331423  9202 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:23.346320  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:23.342276  9230 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:23.349844  3541 rpcz_store.cc:276] 0709 14:54:22.292172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.292413 (+   241us) service_pool.cc:224] Handling call
0709 14:54:22.292794 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:22.644233 (+351439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:22.644296 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:22.652466 (+  8170us) log.cc:844] Serialized 26 byte log entry
0709 14:54:23.080632 (+428166us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:23.080677 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.080772 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:23.342027 (+261255us) raft_consensus.cc:1685] finished
0709 14:54:23.342051 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:23.342224 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30492,"raft.queue_time_us":213363,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":9211,"thread_start_us":650667,"threads_started":2,"wal-append.queue_time_us":457882}
I20260709 14:54:23.326659  9225 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.352677  9202 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.354640  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:23.354158  9213 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.661s	user 0.005s	sys 0.002s
I20260709 14:54:23.359268  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.359462  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:23.360054  2608 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:23.360113  2752 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 14:54:23.361963  9273 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 14:54:23.362298  9109 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:23.362732  9109 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.363642  9109 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.364933  9109 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.366540  9228 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16.
I20260709 14:54:23.362323  9242 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.329340  9020 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.339977  9235 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.375880  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.381644  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1747) took 1091 ms. Trace:
I20260709 14:54:23.356871  9069 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:23.383363  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.382462  3537 rpcz_store.cc:276] 0709 14:54:22.290313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.300731 (+ 10418us) service_pool.cc:224] Handling call
0709 14:54:22.340514 (+ 39783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:22.981648 (+641134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.381225 (+399577us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54004c4ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.381546 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415572864}
I20260709 14:54:23.360860  9008 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:23.397519  9217 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.801s	user 0.005s	sys 0.001s
I20260709 14:54:23.398941  9256 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.404639  9225 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.405817  9104 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:23.406397  9104 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:23.407469  9104 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.351934  9064 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.409523  9104 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:23.409549  9155 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:23.412366  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1189) took 2213 ms. Trace:
W20260709 14:54:23.412652  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1192) took 1950 ms (client timeout 1880 ms). Trace:
W20260709 14:54:23.413280  2754 rpcz_store.cc:269] 0709 14:54:21.462563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.462775 (+   212us) service_pool.cc:224] Handling call
0709 14:54:23.412442 (+1949667us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400571a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.412571 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199593216}
I20260709 14:54:23.414779  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.374804  9226 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:23.420670  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:23.393826  9274 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 14:54:23.417018  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.420460  2754 raft_consensus.cc:2393] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:23.412846  2681 rpcz_store.cc:276] 0709 14:54:21.198767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.199482 (+   715us) service_pool.cc:224] Handling call
0709 14:54:21.254728 (+ 55246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:21.677081 (+422353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.412023 (+1734942us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540012c854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.412289 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630465408}
I20260709 14:54:23.422927  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.424065  2607 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:23.433130  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.433786  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.434348  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.395689  9069 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.435261  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.437749  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.437640  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.441803  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.443471  3537 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.441601  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.351567  9238 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.445948  2681 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:23.447659  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.448925  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.451505  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.448390  3541 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:23.454010  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.428220  8921 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.415045  9202 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.455986  3541 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
W20260709 14:54:23.463495  9272 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 14:54:23.463526  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:23.464666  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.464603  3541 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:23.461203  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:23.465737  2681 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:23.460203  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.468887  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.469729  2752 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:23.472560  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1199) took 1367 ms. Trace:
I20260709 14:54:23.446552  9028 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.473321  2758 rpcz_store.cc:276] 0709 14:54:22.105611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.107606 (+  1995us) service_pool.cc:224] Handling call
0709 14:54:22.108113 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:22.117611 (+  9498us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:22.295145 (+177534us) log.cc:844] Serialized 14 byte log entry
0709 14:54:23.286246 (+991101us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18004745e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:23.337756 (+ 51510us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:23.342287 (+  4531us) log.cc:844] Serialized 26 byte log entry
0709 14:54:23.342460 (+   173us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:23.342495 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.342575 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:23.464188 (+121613us) raft_consensus.cc:1685] finished
0709 14:54:23.472207 (+  8019us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:23.472452 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8884,"prepare.queue_time_us":230385,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":15454,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":13642,"spinlock_wait_cycles":2908416,"thread_start_us":199706,"threads_started":2,"wal-append.queue_time_us":1047977}
I20260709 14:54:23.479877  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.481020  2758 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:23.483312  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.461800  9235 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.434958  9027 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.488682  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:23.481953  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.486778  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.440143  9256 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 14:54:23.493388  2752 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:23.490586  2758 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.495584  9246 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:23.503039  9269 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.296s	user 0.002s	sys 0.000s
W20260709 14:54:23.503444  9279 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 14:54:23.504735  9251 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.482686  8803 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.511081  9069 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.518229  9251 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.522229  9251 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.469293  8921 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.525997  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.537941  2752 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:23.519472  3546 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:23.488976  9020 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:23.532358  9251 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.504783  9021 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.541029  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.492790  9202 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.544845  9020 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.549445  3546 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:23.551749  8921 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.552098  9027 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.552178  9071 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.486328  9028 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:23.573207  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.593681  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.574318  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.581729  9116 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.587930  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1543) took 1262 ms. Trace:
I20260709 14:54:23.601701  3536 rpcz_store.cc:276] 0709 14:54:22.325395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.339497 (+ 14102us) service_pool.cc:224] Handling call
0709 14:54:22.441854 (+102357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:22.442764 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.587550 (+1144786us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002017d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.587821 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234421120}
I20260709 14:54:23.602984  3537 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:23.607705  9133 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:23.608748  9020 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:23.551743  9206 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:23.610419  9215 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 2.002s	user 0.002s	sys 0.008s
I20260709 14:54:23.615108  9115 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:23.615355  9028 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.620494  9115 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.621019  9071 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:23.619771  9249 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.621614  9115 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.623466  9115 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.626345  8885 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.627964  8885 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.629518  9263 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.631568  9154 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:23.632122  9154 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.632274  9263 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.632709  9249 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:23.633203  9154 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.638927  9214 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:23.643287  9214 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.644382  9214 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:23.645058  9237 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.014s
I20260709 14:54:23.634464  9250 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.636034  9044 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 14:54:23.649745  9044 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:23.645866  9214 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:23.634724  9154 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.645658  9237 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.642457  9244 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:23.705277  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.709074  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.709875  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:23.710397  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.712321  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.541321  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:23.716627  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 175204us
I20260709 14:54:23.724103  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:23.727073  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:23.637120  8803 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:23.732613  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.735678  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.736909  3541 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.739691  3545 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:23.740644  2681 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:23.751055  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.753562  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.752804  9044 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.755683  9011 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:23.758240  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.761606  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:23.759399  9192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.674s	user 0.005s	sys 0.003s
I20260709 14:54:23.780064  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:23.786860  2753 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 14:54:23.787659  2753 pending_rounds.cc:85] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 9
I20260709 14:54:23.788008  2753 pending_rounds.cc:107] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 14:54:23.788439  2753 raft_consensus.cc:2889] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:23.796957  9230 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.797063  9044 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 14:54:23.798264  9230 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.803628  9163 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.804150  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1663) took 1097 ms. Trace:
I20260709 14:54:23.804620  9163 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.805198  9250 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.806242  9011 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:23.809118  2677 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:23.804737  2682 rpcz_store.cc:276] 0709 14:54:22.706119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.711397 (+  5278us) service_pool.cc:224] Handling call
0709 14:54:23.803525 (+1092128us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400269654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.803874 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694571392}
W20260709 14:54:23.810958  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1647) took 2538 ms (client timeout 1917 ms). Trace:
I20260709 14:54:23.811197  9056 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:23.812270  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1709) took 1504 ms (client timeout 1992 ms). Trace:
I20260709 14:54:23.815728  2757 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:23.816522  9244 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.817015  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:23.819276  2679 rpcz_store.cc:269] 0709 14:54:21.272278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.550042 (+277764us) service_pool.cc:224] Handling call
0709 14:54:23.802060 (+2252018us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54002b3254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.810497 (+  8437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651213056}
W20260709 14:54:23.820729  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1737) took 1964 ms (client timeout 1953 ms). Trace:
I20260709 14:54:23.821682  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.822181  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.822888  9256 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:23.823339  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1188) took 2626 ms. Trace:
I20260709 14:54:23.824576  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1175) took 1782 ms. Trace:
I20260709 14:54:23.757194  9095 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:23.825935  9192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.762s	user 0.005s	sys 0.006s
W20260709 14:54:23.826027  2751 rpcz_store.cc:269] 0709 14:54:22.307230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.366412 (+ 59182us) service_pool.cc:224] Handling call
0709 14:54:23.811724 (+1445312us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54003363d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.811978 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041601920}
W20260709 14:54:23.828723  2609 rpcz_store.cc:269] 0709 14:54:21.856550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.856742 (+   192us) service_pool.cc:224] Handling call
0709 14:54:23.820014 (+1963272us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400135454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.820399 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459078528}
I20260709 14:54:23.830745  2686 rpcz_store.cc:276] 0709 14:54:21.196884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.197129 (+   245us) service_pool.cc:224] Handling call
0709 14:54:21.197604 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:22.062101 (+864497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.822488 (+1760387us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54002b3754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.823005 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985458560}
I20260709 14:54:23.822439  9019 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:23.850889  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:23.863557  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.864877  2752 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:23.865195  3537 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:23.831137  2605 rpcz_store.cc:276] 0709 14:54:22.041691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.041982 (+   291us) service_pool.cc:224] Handling call
0709 14:54:22.042564 (+   582us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:22.285556 (+242992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:22.285607 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:22.295346 (+  9739us) log.cc:844] Serialized 52 byte log entry
0709 14:54:22.466682 (+171336us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:22.467151 (+   469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.823483 (+1356332us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54001ebc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.823555 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:23.823656 (+   101us) raft_consensus.cc:1685] finished
0709 14:54:23.823676 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:23.824227 (+   551us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1053,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":13402,"spinlock_wait_cycles":535945600,"thread_start_us":115919,"threads_started":1,"wal-append.queue_time_us":303505,"wal-append.run_cpu_time_us":2872,"wal-append.run_wall_time_us":1165236}
I20260709 14:54:23.871569  9109 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:23.877209  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.879052  9240 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:23.880232  9240 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.883386  2684 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 14:54:23.884090  8578 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:23.884562  8578 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:23.889858  9056 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.890903  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.892894  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.893769  9056 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.895037  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:23.896059  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1382) took 2762 ms. Trace:
W20260709 14:54:23.896436  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1386) took 2668 ms (client timeout 1710 ms). Trace:
I20260709 14:54:23.897373  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:23.899036  9149 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:23.899376  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70523.9us
I20260709 14:54:23.868978  9095 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:23.899749  9149 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:23.899925  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.900278  9149 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:23.901296  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:23.902098  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:23.902765  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.905087  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1877) took 1709 ms. Trace:
I20260709 14:54:23.905634  2604 rpcz_store.cc:276] 0709 14:54:22.197197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.666293 (+469096us) service_pool.cc:224] Handling call
0709 14:54:22.745726 (+ 79433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:23.206543 (+460817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.903643 (+697100us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.904834 (+  1191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007371648}
I20260709 14:54:23.877128  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.873965  2682 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:23.909255  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:23.897396  3542 rpcz_store.cc:269] 0709 14:54:21.227695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.227946 (+   251us) service_pool.cc:224] Handling call
0709 14:54:23.895643 (+2667697us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002eb8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.896052 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331543296}
I20260709 14:54:23.899459  9011 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.912247  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.912978  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.913245  3542 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:23.913992  2609 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:23.915869  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:23.916255  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:23.918339  9097 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:23.919476  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.920383  9250 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:23.920738  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.920918  2604 raft_consensus.cc:2410] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 14:54:23.921267  9250 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:23.924441  9103 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:23.925251  9103 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:23.926880  2610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1730) took 2511 ms (client timeout 2121 ms). Trace:
W20260709 14:54:23.927495  2610 rpcz_store.cc:269] 0709 14:54:21.414804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.414977 (+   173us) service_pool.cc:224] Handling call
0709 14:54:21.923860 (+508883us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400125d20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e649e18 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.926369 (+2002509us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400098654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:23.926585 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634314368}
I20260709 14:54:23.929102  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.907621  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.850567  9192 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:23.934460  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.935137  9116 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:23.935680  9116 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:23.936070  9116 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.936429  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.937029  9219 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:23.937459  3545 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:23.938261  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.932425  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.938702  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.896586  3547 rpcz_store.cc:276] 0709 14:54:21.133980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.134177 (+   197us) service_pool.cc:224] Handling call
0709 14:54:21.159916 (+ 25739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:22.153166 (+993250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.895333 (+1742167us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54003df9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.895759 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282603264}
W20260709 14:54:23.945873  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.886108  8578 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:23.947777  9097 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:23.948686  2679 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:23.949206  8894 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:23.951457  8708 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:23.957180  9028 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:23.962852  9028 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:23.960842  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.961103  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1845) took 1649 ms. Trace:
I20260709 14:54:23.964713  2751 rpcz_store.cc:276] 0709 14:54:22.311792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.861802 (+1550010us) service_pool.cc:224] Handling call
0709 14:54:23.960089 (+ 98287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:23.960607 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:23.960633 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:23.960722 (+    89us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:23.960787 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.960854 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:23.961008 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:23.962037  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:23.945600  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:23.873034  9109 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:23.970153  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:23.972743  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:23.978888  3545 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:23.979182  2605 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:23.976747  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1837) took 2407 ms. Trace:
I20260709 14:54:23.967178  9028 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:23.986950  2650 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:23.988160  2753 rpcz_store.cc:276] 0709 14:54:21.569050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.884804 (+315754us) service_pool.cc:224] Handling call
0709 14:54:21.885124 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:23.789286 (+1904162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:23.963438 (+174152us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b5400423bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:23.976618 (+ 13180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4371396480}
I20260709 14:54:23.998662  3542 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:23.999922  9230 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.000327  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:23.993039  2611 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:23.996702  2681 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:24.004505  9237 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:24.007113  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:23.975665  9095 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.009455  2686 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:24.000326  2758 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.014s
W20260709 14:54:24.013181  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 101243us
W20260709 14:54:24.018524  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46201.4us
I20260709 14:54:24.020229  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1710) took 1712 ms. Trace:
I20260709 14:54:24.020875  2759 rpcz_store.cc:276] 0709 14:54:22.307560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.753343 (+445783us) service_pool.cc:224] Handling call
0709 14:54:24.013955 (+1260612us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.019939 (+  5984us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965525888}
I20260709 14:54:24.022190  9027 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:24.023999  2751 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:54:24.027083  9291 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:24.028167  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1182) took 1013 ms. Trace:
I20260709 14:54:24.028081  9291 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.028723  2611 rpcz_store.cc:276] 0709 14:54:23.014544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.012161 (+997617us) service_pool.cc:224] Handling call
0709 14:54:24.027310 (+ 15149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.027705 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.027737 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.027801 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:24.027848 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.027914 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.028071 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:24.030164  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:24.035684  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 157620us
I20260709 14:54:24.037161  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1843) took 1726 ms. Trace:
W20260709 14:54:24.038082  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.027645  9192 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.000000  9103 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.051147  8578 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:24.037811  2757 rpcz_store.cc:276] 0709 14:54:22.311042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.984584 (+673542us) service_pool.cc:224] Handling call
0709 14:54:23.815027 (+830443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:23.839445 (+ 24418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.036886 (+197441us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54003363d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.037061 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908668288}
I20260709 14:54:24.035832  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:24.051177  2756 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:24.053635  2679 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:24.062440  2609 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:24.063894  3541 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:24.066084  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:24.055197  2758 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:24.047371  9097 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.070271  9262 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.071362  9021 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.072093  9021 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:24.068128  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1180) took 1309 ms. Trace:
I20260709 14:54:24.076186  2610 rpcz_store.cc:276] 0709 14:54:22.752388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.940137 (+1187749us) service_pool.cc:224] Handling call
0709 14:54:24.016433 (+ 76296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.061066 (+ 44633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.061126 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.061209 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:24.061264 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.061366 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.061562 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:24.082108  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.055197  9250 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:24.057814  9240 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.091990  9008 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.094363  2678 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:24.095870  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:24.096490  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:24.096906  9202 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:24.097285  9103 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:24.100133  3542 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:24.108206  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 164056us
I20260709 14:54:24.108719  2753 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:24.109717  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1719) took 1025 ms. Trace:
I20260709 14:54:24.111773  2753 rpcz_store.cc:276] 0709 14:54:23.084832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.011422 (+926590us) service_pool.cc:224] Handling call
0709 14:54:24.077618 (+ 66196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.109317 (+ 31699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.109651 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:24.114207  2686 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:24.116259  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.116252  9008 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.124001  8894 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:24.123275  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1183) took 1059 ms. Trace:
I20260709 14:54:24.127844  2611 rpcz_store.cc:276] 0709 14:54:23.060864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.065703 (+1004839us) service_pool.cc:224] Handling call
0709 14:54:24.101165 (+ 35462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.103429 (+  2264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.103464 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.109882 (+  6418us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:24.119999 (+ 10117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.120059 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.120175 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:24.115398  9021 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.130081  9103 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:24.136576  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.138794  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:24.139961  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.139242  9103 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.146986  9103 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:24.150326  9263 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.155514  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:24.161490  2681 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 14:54:24.150853  9008 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.163419  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:24.166247  9263 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:24.168517  9109 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:24.169831  9263 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:24.170889  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:24.176918  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 91252.9us
W20260709 14:54:24.182315  9133 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.006s
W20260709 14:54:24.167919  8885 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.539s	user 0.016s	sys 0.014s
I20260709 14:54:24.168558  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:24.175870  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1184) took 1041 ms. Trace:
I20260709 14:54:24.139480  9202 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.193930  2610 rpcz_store.cc:276] 0709 14:54:23.123144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.081138 (+957994us) service_pool.cc:224] Handling call
0709 14:54:24.126283 (+ 45145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.160368 (+ 34085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.160401 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.163724 (+  3323us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:24.163780 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.163847 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.163958 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:24.201412  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.191280  9240 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:24.212189  2611 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:24.212481  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.217032  3545 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:24.220923  9256 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:24.221292  3540 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:24.221154  3541 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:24.221755  9256 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:24.222359  9256 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:24.224763  3541 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 14:54:24.226738  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.228777  3541 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:24.229475  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:24.232070  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43542.4us
W20260709 14:54:24.235677  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1706) took 1932 ms (client timeout 2053 ms). Trace:
W20260709 14:54:24.236171  2755 rpcz_store.cc:269] 0709 14:54:22.303538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.303679 (+   141us) service_pool.cc:224] Handling call
0709 14:54:24.235170 (+1931491us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54000d0a54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.235459 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167470848}
W20260709 14:54:24.235064  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.237932  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1171) took 2242 ms. Trace:
I20260709 14:54:24.247222  2606 rpcz_store.cc:276] 0709 14:54:21.995449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.995581 (+   132us) service_pool.cc:224] Handling call
0709 14:54:21.996034 (+   453us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:22.017037 (+ 21003us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 14:54:22.059959 (+ 42922us) log.cc:844] Serialized 14 byte log entry
0709 14:54:23.578193 (+1518234us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b1800023920. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1
0709 14:54:23.772067 (+193874us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b1800023920. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:24.009568 (+237501us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:24.025855 (+ 16287us) log.cc:844] Serialized 78 byte log entry
0709 14:54:24.026212 (+   357us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:24.026252 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.026419 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.190615 (+164196us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b1800023920. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:24.237326 (+ 46711us) raft_consensus.cc:1685] finished
0709 14:54:24.237363 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.237662 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64817,"prepare.queue_time_us":46726,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":119,"raft.queue_time_us":58895,"raft.run_cpu_time_us":701,"raft.run_wall_time_us":88986,"spinlock_wait_cycles":79638784,"thread_start_us":197790,"threads_started":1,"wal-append.queue_time_us":1989134}
W20260709 14:54:24.251972  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.239756  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1668) took 2961 ms (client timeout 2187 ms). Trace:
I20260709 14:54:24.253945  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:24.254770  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:24.241250  9020 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:24.255982  2753 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:24.256078  9020 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.256668  9020 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:24.260154  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:24.265985  2686 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:54:24.218808  2608 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.758s	user 0.005s	sys 0.018s
W20260709 14:54:24.254016  2683 rpcz_store.cc:269] 0709 14:54:21.278559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.780541 (+501982us) service_pool.cc:224] Handling call
0709 14:54:24.239330 (+2458789us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400267d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.239521 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2585634048}
I20260709 14:54:24.275604  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:24.277153  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80654us
I20260709 14:54:24.283788  2608 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:24.284287  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:24.284853  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1887) took 1014 ms. Trace:
I20260709 14:54:24.285514  2753 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:24.274458  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:24.285341  2608 rpcz_store.cc:276] 0709 14:54:23.270232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.350432 (+ 80200us) service_pool.cc:224] Handling call
0709 14:54:24.284720 (+934288us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:24.295032  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.295284  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 168009us
I20260709 14:54:24.296808  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.300449  9116 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:24.304780  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.287026  2610 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:24.310807  2686 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:24.312163  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:24.298228  9008 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.318811  2681 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:24.314904  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1705) took 2011 ms. Trace:
I20260709 14:54:24.329511  2610 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
W20260709 14:54:24.331064  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 227992us
I20260709 14:54:24.331861  9202 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:24.332479  2610 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 14:54:24.332661  9239 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:24.333333  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 237691us
I20260709 14:54:24.332499  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.333551  9239 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:24.335240  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.335767  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.335815  9049 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:24.336340  9049 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.336794  9049 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:24.337687  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:24.336366  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.346464  2605 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:24.320719  2756 rpcz_store.cc:276] 0709 14:54:22.303196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.303380 (+   184us) service_pool.cc:224] Handling call
0709 14:54:22.303752 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.066539 (+1762787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.304167 (+237628us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:24.314740 (+ 10573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3866090880}
W20260709 14:54:24.355305  9154 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.654s	user 0.014s	sys 0.005s
I20260709 14:54:24.356344  9154 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:54:24.356781  3543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1740) took 3606 ms (client timeout 1945 ms). Trace:
W20260709 14:54:24.357714  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61125.7us
I20260709 14:54:24.357934  2607 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 14:54:24.358930  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.359973  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:24.360975  2604 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:24.362254  8885 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 14:54:24.363003  9071 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.689s	user 0.002s	sys 0.025s
W20260709 14:54:24.357419  3543 rpcz_store.cc:269] 0709 14:54:20.749874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:20.796010 (+ 46136us) service_pool.cc:224] Handling call
0709 14:54:24.356155 (+3560145us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b5400200dd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.356498 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295637376}
I20260709 14:54:24.367080  9008 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.367659  9237 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:24.362646  2604 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 14:54:24.369968  9008 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.370949  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1672) took 1283 ms. Trace:
I20260709 14:54:24.371529  2685 rpcz_store.cc:276] 0709 14:54:23.087565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.087711 (+   146us) service_pool.cc:224] Handling call
0709 14:54:24.370432 (+1282721us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b54001d10d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.370679 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180403456}
I20260709 14:54:24.293792  9149 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:24.372932  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.373741  2685 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:24.376150  2760 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:24.377027  9225 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:24.378309  9225 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.379305  2680 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:24.379832  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.351765  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.383841  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1883) took 1602 ms. Trace:
I20260709 14:54:24.385106  2605 rpcz_store.cc:276] 0709 14:54:22.781856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.872778 (+1090922us) service_pool.cc:224] Handling call
0709 14:54:24.383761 (+510983us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:24.353770  3542 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
W20260709 14:54:24.381357  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100082us
I20260709 14:54:24.319249  8894 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:24.395781  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:24.398787  2681 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:24.401280  9004 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.401546  2683 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:24.404145  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.410960  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.395500  9109 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 14:54:24.419307  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.421945  9206 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.422876  9206 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.426225  9206 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.421216  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:24.428149  9163 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:24.428977  9163 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.429575  2760 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:24.428169  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.426997  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.422655  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:24.437268  2608 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:24.435719  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:24.445797  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1832) took 3893 ms. Trace:
W20260709 14:54:24.449704  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.446383  2760 rpcz_store.cc:276] 0709 14:54:20.537197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:21.028780 (+491583us) service_pool.cc:224] Handling call
0709 14:54:21.029247 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:21.453920 (+424673us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b5400310ea0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:54:24.375216 (+2921296us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b580047f6d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5a20aa 00007fa16e624447 00007fa16e63cfe7 00007fa16e63e7de 00007fa16e648b49 00007fa16e649416 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4
0709 14:54:24.430129 (+ 54913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.430456 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4569083904}
I20260709 14:54:24.433441  9149 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.414124  9060 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.493602  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:24.446076  8885 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:24.508490  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 125116us
W20260709 14:54:24.508805  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 93470.1us
I20260709 14:54:24.447501  9071 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:54:24.524927  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 89471.2us
W20260709 14:54:24.524904  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 102558us
I20260709 14:54:24.429602  9163 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:24.501502  9003 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 14:54:24.519651  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:24.432905  9225 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:24.532577  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.520367  9295 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 14:54:24.440375  9004 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.531368  9060 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.519668  9207 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:24.551088  9207 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:24.551635  2683 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 14:54:24.551914  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139108us
I20260709 14:54:24.552553  9207 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:24.557260  9225 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.540096  9263 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.552593  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1673) took 1147 ms. Trace:
I20260709 14:54:24.559854  2683 rpcz_store.cc:276] 0709 14:54:23.404841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.484372 (+1079531us) service_pool.cc:224] Handling call
0709 14:54:24.551005 (+ 66633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.552165 (+  1160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.552512 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:24.561971  9214 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.010s
I20260709 14:54:24.564566  9239 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:24.566026  9115 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:24.532810  9109 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:24.566768  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1720) took 1477 ms. Trace:
I20260709 14:54:24.566968  9109 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:24.567472  9109 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:24.570314  9163 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.571720  9116 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:24.567310  2759 rpcz_store.cc:276] 0709 14:54:23.089687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.022013 (+932326us) service_pool.cc:224] Handling call
0709 14:54:24.022710 (+   697us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.023408 (+   698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.023445 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.024180 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.290231 (+266051us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:24.290300 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.290407 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.566403 (+275996us) raft_consensus.cc:1685] finished
0709 14:54:24.566439 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.566675 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15830,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":193,"thread_start_us":265476,"threads_started":1,"wal-append.queue_time_us":521633}
W20260709 14:54:24.542945  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 135031us
W20260709 14:54:24.546178  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.558753  9263 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:24.583478  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 354129us
I20260709 14:54:24.575084  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1668) took 1439 ms. Trace:
I20260709 14:54:24.578612  2681 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:24.545802  2611 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:24.554416  9207 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:24.587808  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47436.3us
I20260709 14:54:24.550453  9060 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.589443  2684 rpcz_store.cc:276] 0709 14:54:23.135225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.886435 (+751210us) service_pool.cc:224] Handling call
0709 14:54:23.955645 (+ 69210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.023362 (+ 67717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.023406 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.024180 (+   774us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.166032 (+141852us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:24.166105 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.166195 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.574734 (+408539us) raft_consensus.cc:1685] finished
0709 14:54:24.574782 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.575007 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3085,"raft.queue_time_us":9242,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":185,"thread_start_us":138255,"threads_started":1,"wal-append.queue_time_us":477518}
I20260709 14:54:24.597385  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:24.595934  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 219110us
W20260709 14:54:24.593690  9044 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.017s
I20260709 14:54:24.614903  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1846) took 2294 ms. Trace:
W20260709 14:54:24.615159  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.616074  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.617333  3542 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:24.613518  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.602005  9097 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:24.631505  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.638613  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 110708us
I20260709 14:54:24.640086  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.622241  2752 rpcz_store.cc:276] 0709 14:54:22.320420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.866378 (+1545958us) service_pool.cc:224] Handling call
0709 14:54:23.958430 (+ 92052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.062188 (+103758us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:24.126972 (+ 64784us) log.cc:844] Serialized 14 byte log entry
0709 14:54:24.614331 (+487359us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.614422 (+    91us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:24.614493 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.614619 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.614799 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70914,"prepare.run_wall_time_us":1533,"raft.queue_time_us":12223,"spinlock_wait_cycles":6743936,"thread_start_us":447633,"threads_started":1}
W20260709 14:54:24.652884  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43745.1us
I20260709 14:54:24.626981  9071 raft_consensus.cc:2764] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 14:54:24.658666  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62500.8us
I20260709 14:54:24.660784  3543 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:24.606792  9256 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:24.673334  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.674593  3542 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:24.676406  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:24.613430  9020 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.679558  2759 pending_rounds.cc:85] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 10
I20260709 14:54:24.685331  2686 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:24.686390  2759 pending_rounds.cc:107] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 14:54:24.688198  2759 raft_consensus.cc:2889] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:24.691358  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51414.1us
I20260709 14:54:24.693462  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.694986  3542 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:54:24.696324  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.683997  9116 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:24.656791  9097 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:24.732286  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:24.733253  3542 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:24.710868  9028 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.003s
I20260709 14:54:24.727242  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.737473  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:24.738864  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1202) took 1557 ms. Trace:
I20260709 14:54:24.739382  2680 rpcz_store.cc:276] 0709 14:54:23.173712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.858207 (+684495us) service_pool.cc:224] Handling call
0709 14:54:24.731307 (+873100us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400268754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:24.731509 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926335872}
I20260709 14:54:24.722098  9192 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.687218  9214 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 14:54:24.751001  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:24.820573  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 71282.5us
I20260709 14:54:24.821622  3412 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:24.827482  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84715.1us
I20260709 14:54:24.831249  9244 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 14:54:24.832857  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78122.3us
I20260709 14:54:24.833391  9306 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:24.833913  9306 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.834321  9306 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:24.836503  9256 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:24.834918  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:24.843688  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116588us
I20260709 14:54:24.844954  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.847203  2680 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:24.847709  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 510208us
I20260709 14:54:24.847319  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1181) took 1838 ms. Trace:
I20260709 14:54:24.849097  2603 rpcz_store.cc:276] 0709 14:54:23.008745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.963285 (+954540us) service_pool.cc:224] Handling call
0709 14:54:24.007025 (+ 43740us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:24.336696 (+329671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.336733 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:24.337669 (+   936us) log.cc:844] Serialized 78 byte log entry
0709 14:54:24.616683 (+279014us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:24.616749 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.616835 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.846935 (+230100us) raft_consensus.cc:1685] finished
0709 14:54:24.846961 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.847206 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223266,"raft.queue_time_us":238078,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":170,"thread_start_us":269500,"threads_started":1,"wal-append.queue_time_us":370124}
W20260709 14:54:24.850464  9305 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 14:54:24.740119  9028 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.746537  9044 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:24.748715  8885 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 14:54:24.748456  9149 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 14:54:24.862124  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:24.873180  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1557) took 1150 ms. Trace:
I20260709 14:54:24.847189  9116 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:24.853003  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1552) took 1415 ms. Trace:
I20260709 14:54:24.850792  2576 leader_election.cc:400] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17.
I20260709 14:54:24.883623  2576 leader_election.cc:403] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 14:54:24.874120  3541 rpcz_store.cc:276] 0709 14:54:23.723351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.241292 (+517941us) service_pool.cc:224] Handling call
0709 14:54:24.241734 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:24.277073 (+ 35339us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:24.627846 (+350773us) log.cc:844] Serialized 14 byte log entry
0709 14:54:24.825464 (+197618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:24.825544 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:24.872836 (+ 47292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.872944 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.873098 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9229,"prepare.queue_time_us":390955,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":9787,"raft.queue_time_us":33333,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":147,"thread_start_us":433635,"threads_started":2}
I20260709 14:54:24.881075  9116 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.853286  9311 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 14:54:24.888218  9116 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:24.889293  9311 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:24.889724  9311 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:24.889753  9238 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:24.854369  9301 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 14:54:24.854099  9304 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 14:54:24.838423  9263 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.856479  9020 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:24.851823  9244 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:24.864837  8871 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:24.855402  9044 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:24.881531  3536 rpcz_store.cc:276] 0709 14:54:23.437987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.172253 (+734266us) service_pool.cc:224] Handling call
0709 14:54:24.172876 (+   623us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:24.173656 (+   780us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:24.173688 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:24.174589 (+   901us) log.cc:844] Serialized 52 byte log entry
0709 14:54:24.515778 (+341189us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:24.515848 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.515971 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.852658 (+336687us) raft_consensus.cc:1685] finished
0709 14:54:24.852689 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.852906 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1518,"spinlock_wait_cycles":194283264,"thread_start_us":339769,"threads_started":1,"wal-append.queue_time_us":463322}
I20260709 14:54:24.900838  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.896740  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.914400  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1723) took 1612 ms. Trace:
I20260709 14:54:24.915097  2751 rpcz_store.cc:276] 0709 14:54:23.301797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.025296 (+723499us) service_pool.cc:224] Handling call
0709 14:54:24.025914 (+   618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.059286 (+ 33372us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:24.515387 (+456101us) log.cc:844] Serialized 14 byte log entry
0709 14:54:24.690521 (+175134us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.715982 (+ 25461us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.716205 (+   223us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:24.716250 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.716340 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.857892 (+141552us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b24001cd0a8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b597fb0 00007fa16e20e488 00007fa16e20ebe8 00007fa16e1e6398 00007fa16e1e176a 00007fa16e1e0bb6
0709 14:54:24.908628 (+ 50736us) raft_consensus.cc:1685] finished
0709 14:54:24.914058 (+  5430us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.914297 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49701,"prepare.queue_time_us":529364,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":74,"raft.queue_time_us":16059,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":4944,"spinlock_wait_cycles":33574656,"thread_start_us":498041,"threads_started":2,"wal-append.queue_time_us":297282}
I20260709 14:54:24.917725  9096 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.900524  9311 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.920470  9311 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:24.921838  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1669) took 1786 ms. Trace:
W20260709 14:54:24.923105  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76370.1us
I20260709 14:54:24.923890  2683 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 14:54:24.914475  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64003.5us
I20260709 14:54:24.891170  9214 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:24.824084  9192 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:24.936080  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:24.893788  9163 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:24.921317  9103 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.774s	user 0.017s	sys 0.005s
I20260709 14:54:24.918476  9095 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17.
I20260709 14:54:24.906395  9044 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:24.945607  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:24.903441  8871 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:24.918390  9049 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.009s
I20260709 14:54:24.953876  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:24.922377  2682 rpcz_store.cc:276] 0709 14:54:23.135612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.911377 (+775765us) service_pool.cc:224] Handling call
0709 14:54:23.912051 (+   674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:23.972243 (+ 60192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:23.972288 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.043898 (+ 71610us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.528640 (+484742us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:24.528923 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.529195 (+   272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:24.921533 (+392338us) raft_consensus.cc:1685] finished
0709 14:54:24.921560 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:24.921766 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44664,"raft.queue_time_us":57891,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":34238,"thread_start_us":405144,"threads_started":1,"wal-append.queue_time_us":529445}
I20260709 14:54:24.959929  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:24.951278  8871 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:25.021895  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 85969.3us
W20260709 14:54:25.021144  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 159228us
I20260709 14:54:24.945286  9239 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:25.024420  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 82846.9us
I20260709 14:54:24.950850  9103 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.031816  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:25.034147  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:24.972790  9115 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:25.034878  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:25.038040  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:25.036142  9315 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 14:54:24.918566  9096 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:24.945010  9214 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.032517  9244 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:54:25.069976  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 210904us
I20260709 14:54:25.069783  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.031093  9020 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:24.956933  9049 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.076843  2685 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:25.028776  9095 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.028581  9163 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:25.081594  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1766) took 1356 ms. Trace:
I20260709 14:54:25.041368  9249 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.037664  9115 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:25.087177  3545 rpcz_store.cc:276] 0709 14:54:23.725566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.280842 (+555276us) service_pool.cc:224] Handling call
0709 14:54:24.302174 (+ 21332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.303469 (+  1295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.303509 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.304148 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.623120 (+318972us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:24.623172 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.623247 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:25.081302 (+458055us) raft_consensus.cc:1685] finished
0709 14:54:25.081328 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:25.081510 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"thread_start_us":318407,"threads_started":1,"wal-append.queue_time_us":423395}
I20260709 14:54:25.090389  9163 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.037344  8885 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:25.091696  9320 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:54:25.092121  9319 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:25.093334  9323 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 14:54:25.098574  9027 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:54:25.103180  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 70522.1us
I20260709 14:54:25.104182  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.127667  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1671) took 2041 ms. Trace:
I20260709 14:54:25.041941  9263 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:25.129455  3538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1394) took 2142 ms (client timeout 2013 ms). Trace:
W20260709 14:54:25.134322  2607 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1891) took 1814 ms (client timeout 1931 ms). Trace:
W20260709 14:54:25.135845  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101275us
I20260709 14:54:25.039904  9060 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 14:54:25.123492  9109 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.005s
I20260709 14:54:25.141330  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1185) took 1432 ms. Trace:
W20260709 14:54:25.178673  2607 rpcz_store.cc:269] 0709 14:54:23.319615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.726487 (+406872us) service_pool.cc:224] Handling call
0709 14:54:25.134017 (+1407530us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.134239 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1447172352}
I20260709 14:54:25.124605  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.084311  9097 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:25.177798  3538 rpcz_store.cc:269] 0709 14:54:22.986578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:22.986851 (+   273us) service_pool.cc:224] Handling call
0709 14:54:25.128860 (+2142009us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.129175 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339910400}
W20260709 14:54:25.205343  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.128711  9049 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.117800  9250 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 14:54:25.208297  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 83825us
I20260709 14:54:25.110172  9020 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:25.145700  2677 rpcz_store.cc:276] 0709 14:54:23.086215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.086768 (+   553us) service_pool.cc:224] Handling call
0709 14:54:23.802412 (+715644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:23.836279 (+ 33867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:25.127105 (+1290826us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54002b3250. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.127507 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1644546944}
I20260709 14:54:25.173085  8708 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:54:25.222270  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 134803us
I20260709 14:54:25.110347  9115 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:25.138921  9192 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.180667  2609 rpcz_store.cc:276] 0709 14:54:23.709427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.099226 (+389799us) service_pool.cc:224] Handling call
0709 14:54:24.332822 (+233596us) spinlock_profiling.cc:243] Waited 877 us on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:24.333054 (+   232us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:54:24.339053 (+  5999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:24.339105 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:54:24.443911 (+104806us) log.cc:844] Serialized 78 byte log entry
0709 14:54:24.704475 (+260564us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:24.704628 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.704900 (+   272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:25.140800 (+435900us) raft_consensus.cc:1685] finished
0709 14:54:25.140834 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:25.141230 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5239,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13555,"spinlock_wait_cycles":2018048,"thread_start_us":156181,"threads_started":1,"wal-append.queue_time_us":396660}
I20260709 14:54:25.172796  9027 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:25.143626  9249 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.238643  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.257711  3541 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.131844  9291 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.234463  8708 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:25.243052  9229 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:25.214040  9008 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:25.250991  9263 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:25.254153  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.124464  9071 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 14:54:25.136065  9103 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.137981  9214 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.268378  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.269678  8708 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.270955  9008 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.272341  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.254568  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1898) took 1166 ms. Trace:
I20260709 14:54:25.135272  9319 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.288504  3542 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:25.289621  3545 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:25.141633  9116 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.204082  9109 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:25.299757  3411 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.880s (SENT)
W20260709 14:54:25.276161  9207 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.003s
I20260709 14:54:25.229331  9250 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:25.277853  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.282390  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:25.289362  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 81528us
I20260709 14:54:25.110271  9095 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.132416  8885 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.273456  9291 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.261230  9244 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
W20260709 14:54:25.303876  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2518037399999002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 14:54:25.275667  9192 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.291656  2610 rpcz_store.cc:276] 0709 14:54:24.088422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.333721 (+245299us) service_pool.cc:224] Handling call
0709 14:54:24.335460 (+  1739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.383706 (+ 48246us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:24.383762 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:24.443293 (+ 59531us) log.cc:844] Serialized 26 byte log entry
0709 14:54:24.686558 (+243265us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:24.686613 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:24.689200 (+  2587us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:25.254208 (+565008us) raft_consensus.cc:1685] finished
0709 14:54:25.254242 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:25.254473 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13483,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":31588,"thread_start_us":157806,"threads_started":1,"wal-append.queue_time_us":534556}
W20260709 14:54:25.315795  2576 connection.cc:256] RPC call timeout handler was delayed by -1.1569998170000417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 14:54:25.282195  9028 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.315837  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.329463  3541 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:25.289228  9008 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:25.236455  9239 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 14:54:25.307884  8708 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:25.337143  2681 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:25.270705  9229 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.345664  9027 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.348834  8885 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.350369  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:25.352159  9263 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.359123  2603 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:25.331645  9207 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:54:25.359055  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 109672us
W20260709 14:54:25.338243  8578 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.271s	user 0.007s	sys 0.030s
I20260709 14:54:25.354050  9109 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.345750  2609 raft_consensus.cc:1217] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Deduplicated request from leader. Original: 12.8->[13.9-15.10]   Dedup: 13.9->[15.10-15.10]
I20260709 14:54:25.369014  2609 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 14:54:25.371542  2611 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:25.371819  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:25.373867  2608 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:25.374119  2611 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:25.375201  2608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:25.375500  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:25.376245  2608 raft_consensus.cc:606] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.376591  2604 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:25.376797  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:25.377243  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:25.377621  2606 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:25.376801  2608 raft_consensus.cc:614] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:25.376274  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.379647  9049 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:25.367662  3537 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:25.380789  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:25.360419  9306 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 14:54:25.330533  9291 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.392495  3542 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:25.381769  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:25.365485  8885 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:25.398236  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 120525us
I20260709 14:54:25.362977  9263 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.364964  9207 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.399943  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.395005  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.397262  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.412741  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:25.413594  2612 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.416026  9044 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.418871  9244 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:25.419474  9244 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:25.412770  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 125209us
I20260709 14:54:25.423239  9096 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.423105  9244 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.368480  9020 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:25.424490  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.424757  9244 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.427296  9095 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.397935  9306 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.430177  8871 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.433375  9306 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.437242  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.438055  3542 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:25.438117  9249 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:25.440858  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47150.4us
I20260709 14:54:25.386071  9163 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.445490  3545 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:25.449944  2678 raft_consensus.cc:2368] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:25.447646  9342 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:25.441308  9207 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.444419  8871 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.477484  9115 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.480600  9319 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.482318  9116 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.484201  9229 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.484711  9291 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.484975  9229 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.485581  9229 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.437273  9252 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:25.486848  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.487159  9116 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.487329  9252 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:25.487785  9116 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.488468  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.380684  9192 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:25.495204  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.474583  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.418669  9044 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.490063  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.495153  2650 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.497596  2612 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:25.489688  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.499179  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:25.504043  2759 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [17.12-18.13]
W20260709 14:54:25.504550  9238 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.614s	user 0.017s	sys 0.000s
I20260709 14:54:25.410876  9250 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.449025  8885 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:25.505501  9250 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.506947  9116 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.506453  8885 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:25.507472  9300 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.956s	user 0.006s	sys 0.000s
I20260709 14:54:25.464038  9163 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:25.509899  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.499341  3541 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:25.511864  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 131544us
I20260709 14:54:25.505528  9341 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:25.494803  9311 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.574s	user 0.014s	sys 0.000s
I20260709 14:54:25.520012  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.466519  9049 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:25.524716  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.518607  3542 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
W20260709 14:54:25.528343  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162348us
W20260709 14:54:25.533524  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.514516  9238 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:25.535678  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 262306us
I20260709 14:54:25.542182  2753 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:25.539580  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.524678  9116 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:25.537416  2754 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:25.545269  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:25.545267  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.546815  3541 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:25.544638  9311 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 14:54:25.553002  9343 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:25.553819  9343 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.556315  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.557212  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.558055  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.558988  2755 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:25.559525  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.558212  2757 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:25.560806  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.561455  2758 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:25.562355  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.563393  2677 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:25.565153  2755 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 14:54:25.567116  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 92801.1us
I20260709 14:54:25.567291  9207 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.568496  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.558992  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.559063  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.527733  9049 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:25.499560  9263 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.570992  2759 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:25.578373  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.579980  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.581980  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.587785  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.592134  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.593637  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.532958  9344 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:54:25.588671  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43514.2us
I20260709 14:54:25.544662   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:54:25.536201  9337 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:25.601272  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.597026  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.592684  2751 raft_consensus.cc:2376] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21. Re-sending same reply.
I20260709 14:54:25.604631  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.608759  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.488504  9252 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:25.610971  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.614385  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:25.608697  2760 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.583807  9192 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.617522  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.513576  9341 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.590060  9049 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:25.619395  3542 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:25.609376  9153 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:25.620018  2684 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:25.600332  9163 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.624245  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.625705  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.629351  9008 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.632124  9337 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:25.633484  9115 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:25.634402  3547 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 14:54:25.663630  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.637212  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.640172  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.668715  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.669812  2751 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:25.640342  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.671326  2759 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:25.641376  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.642207  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.673700  2754 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:25.549914  9238 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:25.644132  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.677820  9340 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:25.678674  9340 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.679405  3547 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:25.681435  9340 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.603547  9311 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:25.682217  9311 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.682699  9311 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.620383  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.617452  9207 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.613638  9027 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:25.655990  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 136071us
I20260709 14:54:25.656654  2752 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 19. Ops: []
I20260709 14:54:25.603894  9345 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:25.657843  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.650508  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.654361  3536 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:25.651254  2757 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:25.612618  9060 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:25.699867  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:25.705955  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.707022  2684 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:25.707674  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:25.684219  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:25.708703  9095 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:25.709575  9095 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:25.710011  9095 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.710402  9095 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:25.710975  9336 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:25.711277  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.713362  3538 raft_consensus.cc:1275] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 14:54:25.717980  3538 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
W20260709 14:54:25.722234  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.728044  2759 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 14:54:25.728660  2759 pending_rounds.cc:85] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 12
I20260709 14:54:25.728940  2759 pending_rounds.cc:107] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 14:54:25.729209  2759 raft_consensus.cc:2889] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:25.642573  9343 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.730307  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1683) took 1548 ms. Trace:
I20260709 14:54:25.732513  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1687) took 1454 ms. Trace:
I20260709 14:54:25.733031  2685 rpcz_store.cc:276] 0709 14:54:24.282529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.374602 (+ 92073us) service_pool.cc:224] Handling call
0709 14:54:25.732302 (+1357700us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.732439 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1516273536}
I20260709 14:54:25.741570  9095 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.745469  9229 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.702627  9249 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 14:54:25.684362  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.700222  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.705842  9060 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:25.711513  9336 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.703416  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1202) took 1051 ms. Trace:
I20260709 14:54:25.696077  2650 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.749008  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.755487  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.734534  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.773140  2757 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:25.755873  2753 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.762460  3543 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 14:54:25.763412  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79185.7us
I20260709 14:54:25.767450  2752 rpcz_store.cc:276] 0709 14:54:24.652541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.653164 (+1000623us) service_pool.cc:224] Handling call
0709 14:54:25.656425 (+  3261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:25.703003 (+ 46578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:25.703330 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:25.778347  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.730815  2683 rpcz_store.cc:276] 0709 14:54:24.181899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.571488 (+389589us) service_pool.cc:224] Handling call
0709 14:54:24.923440 (+351952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:24.924626 (+  1186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:25.730101 (+805475us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400343850. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.730247 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807603072}
W20260709 14:54:25.768235  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.760633  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.764094  3542 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:25.788057  3536 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:25.783234  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 87295.2us
W20260709 14:54:25.787395  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.785449  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:25.820797  2685 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.760082  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.825476  2686 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:25.818965  3537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1755) took 2315 ms (client timeout 1907 ms). Trace:
I20260709 14:54:25.832572  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:25.832619  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 364166us
W20260709 14:54:25.710564  9347 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 14:54:25.756673  9348 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 14:54:25.851011  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65644us
W20260709 14:54:25.852787  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65931.3us
W20260709 14:54:25.830193  3537 rpcz_store.cc:269] 0709 14:54:23.466040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:23.866375 (+400335us) service_pool.cc:224] Handling call
0709 14:54:25.780724 (+1914349us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:25.780925 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3355815808}
I20260709 14:54:25.856050  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:25.909962  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55495.1us
I20260709 14:54:25.911381  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:25.911388  2751 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
W20260709 14:54:25.856171  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 297325us
I20260709 14:54:25.861485  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.860476  3547 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 14:54:25.941689  3547 pending_rounds.cc:85] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 7
I20260709 14:54:25.942085  3547 pending_rounds.cc:107] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 14:54:25.942371  3547 raft_consensus.cc:2889] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:25.942746  3547 pending_rounds.cc:107] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 14:54:25.943039  3547 raft_consensus.cc:2889] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:25.817857  9153 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:25.920392  2683 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:25.923727  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:25.925087  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.925832  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.926035  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.945875  2678 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:25.763664  9115 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:25.947270  9115 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:25.947767  9115 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:25.944435  2757 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:25.945324  2681 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:25.928762  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:25.949836  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.746714  9238 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.747115  9027 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:25.956535  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.956128  9027 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.758692  9095 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:25.829222  9192 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:25.933849  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:25.945035  2752 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.951675  2681 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:25.758155  9263 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:25.765069  9008 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:25.945911  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:25.910182  9096 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:25.951452  2753 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:25.992249  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:25.937366  9163 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:25.759405  9249 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:25.962296  2759 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:25.766113  9336 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:25.946748  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:25.983600  2681 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 14:54:25.962960  2652 leader_election.cc:336] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.931s (SENT)
I20260709 14:54:25.791013  8708 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:25.986378  3537 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 14.
I20260709 14:54:25.953791  2683 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:25.910285  9291 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:25.986245  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:25.868120  9116 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:25.828810  9244 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 14:54:26.005789  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.014114  2753 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 14:54:25.953024  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.001554  2685 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:26.003832  3542 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:26.009745  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:26.018114  3538 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:26.026499  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.027792  2759 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:25.993834  9096 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.032855  2752 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:26.038223  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:26.045434  3542 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:25.987610  9153 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:26.042922  2681 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:26.046175  2753 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 14:54:26.000590  9353 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:26.051131  9351 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 14:54:26.047240  9291 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.053610  8885 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:26.054219  8885 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:25.765347  9337 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:26.071130  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 502792us
I20260709 14:54:26.083674  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.083928  2683 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:26.084422  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.048164  9096 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:26.001238  9249 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:26.054553  9291 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:26.094740  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.111871  2679 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:26.112627  3537 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:54:26.096395  2757 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:26.110713  2753 raft_consensus.cc:2393] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:26.108669  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 195473us
I20260709 14:54:26.124444  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.125535  2686 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:26.130522  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.131088  2681 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 14:54:26.131402  2752 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:26.134004  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.050762  9353 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.145784  2759 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:26.080441  9095 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 14:54:26.027405  8708 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:26.173537  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43486.3us
I20260709 14:54:26.178244  3546 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:26.156680  2753 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:26.158562  9249 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.144491  9291 consensus_queue.cc:1048] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 14:54:26.165124  9004 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:26.167035  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.188305  2760 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:26.171324  9153 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.146579  9343 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 14:54:26.103554  9345 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:26.212054  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.189577  3538 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:26.195389  3542 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:26.196205  9356 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:26.201481  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:26.217643  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 292867us
I20260709 14:54:26.217911  2760 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:26.218160  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.203889  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.101033  9149 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 14:54:26.180991  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.212975  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.224399  2759 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:26.214115  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.221024  2753 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:26.226099  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:26.226210  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.227370  2760 raft_consensus.cc:2368] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:26.237401  2757 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:26.222852  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.248970  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.248747  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.250433  2760 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:26.155184  9331 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 14:54:26.253288  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.219105  9249 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:26.261615  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:26.249459  2682 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:26.212801  9229 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:26.256096  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:26.266415  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48358.1us
I20260709 14:54:26.267148  2753 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:26.265712  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.268934  2760 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:26.130241  9116 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.224242  9004 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:26.279016  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:26.261061  9250 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.007s
W20260709 14:54:26.260533  9342 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.791s	user 0.012s	sys 0.003s
I20260709 14:54:26.276185  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:26.222951  9095 consensus_queue.cc:1175] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" responder_term: 17 status { last_received { term: 17 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 17 index: 9 } }
I20260709 14:54:26.280884  9229 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.281018  9207 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:26.281044  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:26.284885  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.292816  9354 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:26.293179  2760 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
W20260709 14:54:26.293695  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112823us
I20260709 14:54:26.324453  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.324308  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.325323  2760 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:26.216012  9115 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.298468  2752 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:54:26.290546  9049 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.604s	user 0.005s	sys 0.005s
I20260709 14:54:26.319502  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.333664  2757 raft_consensus.cc:2410] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 14:54:26.326788  2683 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:26.319690  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:26.333294  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.337181  2760 raft_consensus.cc:2376] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:26.255175  9364 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 14:54:26.323192  9354 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.347572  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.302202  9365 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 14:54:26.352140  9358 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 14:54:26.353497  8885 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:26.313637  9342 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:26.301635  9192 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:26.359658  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:54:26.296211  9263 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 14:54:26.359249  9311 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.654s	user 0.014s	sys 0.000s
I20260709 14:54:26.333228  9363 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:26.363466  9354 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:26.361716  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.394277  2756 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:26.321597  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.394729  2752 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:26.396052  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.397130  2752 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:26.334692  9207 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:26.376649  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.360694  9291 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:26.368940  9149 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:26.385887  9364 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:26.387418  9192 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:26.318591  9071 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:26.358029  9004 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 14:54:26.401976  2760 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:26.338713  9049 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.375376  8885 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:26.408066  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86623.9us
I20260709 14:54:26.477240  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:26.365768  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 155037us
I20260709 14:54:26.434767  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:26.436491  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.437044  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.479426  2752 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 20
W20260709 14:54:26.444806  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 227234us
I20260709 14:54:26.406347  9291 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:26.482420  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.434464  9364 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.385874  9027 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 14:54:26.404547  9353 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:26.486385  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.476714  9342 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:26.489848  2723 leader_election.cc:400] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:26.511039  2723 leader_election.cc:403] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 14:54:26.381284  9311 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:26.482883  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:26.480188  3409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 2.104s (SENT) [suppressed 32 similar messages]
I20260709 14:54:26.476347  9363 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:26.516091  2760 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:26.476423  8885 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:26.477598  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 216073us
I20260709 14:54:26.485282  9096 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:26.492517  9291 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:26.499897  9364 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:26.534310  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 196759us
I20260709 14:54:26.530552  2577 leader_election.cc:400] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
W20260709 14:54:26.531570  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51457.7us
I20260709 14:54:26.536636  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.507848  9071 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:26.537415  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.522403  9342 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:26.532258  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45078.2us
W20260709 14:54:26.534966  2685 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.001s
I20260709 14:54:26.540282  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.541154  9071 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:26.542816  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 218749us
I20260709 14:54:26.546470  9071 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:26.547106  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.547741  2760 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:26.524081  9363 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.530072  8885 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:26.552057  2752 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:26.547686  2685 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
W20260709 14:54:26.556212  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.556885  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:26.564579  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.541352  9353 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:26.566876  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:26.568913  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:26.548388  9342 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:26.540997  9352 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:26.584535  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:26.578519  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 869694us
I20260709 14:54:26.574704  9229 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:26.576084  9336 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 14:54:26.548285  9378 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:26.578424  9096 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:26.579409  9115 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:26.580484  9383 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:26.581408  9381 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:26.554889  8708 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:26.581439  9049 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:26.584095  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 94418us
I20260709 14:54:26.607872  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:26.587147  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.583495  9060 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:26.552629  9363 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:26.598976  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1208) took 1001 ms. Trace:
I20260709 14:54:26.547930  9377 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 14:54:26.584918  9352 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:26.580530  9004 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:54:26.637372  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50347us
I20260709 14:54:26.543885  9207 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.610090  9383 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:26.637462  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 97254.3us
I20260709 14:54:26.609910  9229 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:26.616801  2678 rpcz_store.cc:276] 0709 14:54:25.597664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.947153 (+349489us) service_pool.cc:224] Handling call
0709 14:54:25.947812 (+   659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:25.992692 (+ 44880us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 14:54:25.992731 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:26.017918 (+ 25187us) log.cc:844] Serialized 26 byte log entry
0709 14:54:26.313369 (+295451us) raft_consensus.cc:1645] Marking committed up to 6
0709 14:54:26.313430 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:26.313528 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:26.598462 (+284934us) raft_consensus.cc:1685] finished
0709 14:54:26.598499 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:26.598745 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39783,"raft.queue_time_us":77642,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":54148,"spinlock_wait_cycles":1117056,"thread_start_us":212238,"threads_started":1,"wal-append.queue_time_us":385481}
I20260709 14:54:26.614611  9244 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:26.647100  2682 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
W20260709 14:54:26.646704  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62268.8us
I20260709 14:54:26.648175  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.646921  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.632507  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.590217  9365 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 14:54:26.649106  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.650359  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:26.650481  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.655588  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:26.609634  9336 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:26.633723  9149 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 14:54:26.661315  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53562.3us
W20260709 14:54:26.695609  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 124622us
I20260709 14:54:26.675019  9387 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 14:54:26.678169  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:26.679683  9004 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 14:54:26.679922  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47549.8us
I20260709 14:54:26.707334  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.639115  9352 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:26.645443  9229 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.681686  9311 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:26.683003  9384 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:26.683419  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.681715  2759 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:26.684777  9291 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.686151  9071 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.694286  9238 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 14:54:26.689911  9360 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.463s	user 0.000s	sys 0.001s
I20260709 14:54:26.692495  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:26.696125  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:26.638521  9319 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:26.741401  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:26.697809  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 47422.8us
I20260709 14:54:26.700001  2678 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:26.700130  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:26.645009  9331 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:26.683533  9380 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 14:54:26.751101  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55128.7us
I20260709 14:54:26.713318  2757 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:26.657802  9115 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:26.705868  9244 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:26.717713  9378 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 14:54:26.718062  3547 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:26.706799  9004 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:26.722980  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.723272  9366 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:26.687659  9372 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 14:54:26.701843  9381 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 14:54:26.734078  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:26.731559  2683 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:26.746853  9071 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:26.748248  9341 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:26.750288  9336 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:26.751005  9331 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.644793  9383 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:26.744180  3536 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:26.758193  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.716768  9384 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.725520  9229 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:26.712311  9249 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:26.770551  2752 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 14:54:26.848110  2752 pending_rounds.cc:85] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 9
I20260709 14:54:26.848498  2752 pending_rounds.cc:107] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 14:54:26.848807  2752 raft_consensus.cc:2889] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:26.712033  9375 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:26.780237  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:26.851655  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79235.9us
I20260709 14:54:26.691131  9049 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:26.820791  9256 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:26.827499  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 115954us
I20260709 14:54:26.737138  9238 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:26.823752  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:26.755220  9345 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:26.820581  9377 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:26.836961  2759 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:26.884963  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:26.840318  3543 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:26.887310  9244 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.889269  2759 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:26.889861  2759 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:26.705585  9342 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:26.844937  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 103634us
I20260709 14:54:26.892236  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.894245  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.847106  9374 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.202s
I20260709 14:54:26.896087  2759 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.901165  9342 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:26.902125  9342 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:26.903033  9342 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:26.903678  9163 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:26.904273  9163 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:26.837196  9331 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:26.871452  9249 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.909046  9249 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:26.842124  9071 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:26.911137  9071 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:26.911660  9071 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:26.770355  9319 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:26.914261  9319 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:26.824646  9004 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:26.915474  9377 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.770262  9380 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:26.739648  9096 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 14:54:26.820811  9153 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:26.866954  2678 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:26.871811  2752 pending_rounds.cc:107] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 14:54:26.925091  2752 raft_consensus.cc:2889] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:26.875494  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:26.820390  9354 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.871881  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:26.903103  2759 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:26.903021  9391 raft_consensus.cc:2764] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 14:54:26.878904  8708 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:26.927682  2754 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:26.880133  9238 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:26.755858  9387 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:26.932140  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60372.7us
I20260709 14:54:26.756054  9250 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
I20260709 14:54:26.767253  9392 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:26.834095  9115 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 14:54:26.917934  9377 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:26.829612  9390 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 14:54:26.941030  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1215) took 1188 ms. Trace:
W20260709 14:54:26.954682  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 79311us
W20260709 14:54:26.958722  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 235921us
I20260709 14:54:26.953461  2754 rpcz_store.cc:276] 0709 14:54:25.752886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.872662 (+1119776us) service_pool.cc:224] Handling call
0709 14:54:26.922557 (+ 49895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:26.940688 (+ 18131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:26.940962 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:26.829214  9366 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:26.941946  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 245463us
I20260709 14:54:26.965184  2724 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.969911  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:26.971304  2612 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 14:54:26.940822  9381 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:26.932548  9372 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 14:54:26.883314  9345 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:26.956315  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:26.879958  9049 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:26.952682  8708 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:26.919220  9256 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:26.994086  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:26.977908  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:26.998962  2678 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:26.936599  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 176775us
W20260709 14:54:26.876343  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 170100us
I20260709 14:54:26.979566  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:26.985447  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:26.931764  9153 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:26.990209  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:26.990209  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.007452  2603 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:26.967209  2759 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 14:54:26.994086  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.006078  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:26.994419  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.008409  2759 pending_rounds.cc:85] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 10
I20260709 14:54:26.934113  9163 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.012038  9153 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.001523  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.008983  2612 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 14:54:26.991204  9345 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.009852  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:27.009313  2759 pending_rounds.cc:107] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 14:54:27.019577  2759 raft_consensus.cc:2889] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:27.014395  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:27.027908  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:27.026283  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:26.972486  9342 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 14:54:27.037221  2603 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:26.976126  9249 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 14:54:26.990543  9381 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:27.025964  9388 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:27.015455  9391 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:27.062819  9391 raft_consensus.cc:697] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:27.064035  9391 consensus_queue.cc:237] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.012104  9163 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.067123  9377 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:27.057041  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 162916us
I20260709 14:54:27.067188  9388 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:27.068533  9388 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:27.069345  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:27.058207  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.071524  9354 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.055845  9291 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.072870  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.074303  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:27.058434  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44147.3us
I20260709 14:54:27.035777  9250 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:27.075525  2759 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:27.077678  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:26.979157  9363 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.061965  9342 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:27.071211  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:27.080914  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 124750us
I20260709 14:54:27.081156  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:26.964444  9392 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.092880  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:27.099941  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:27.086383  8885 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.003s
I20260709 14:54:27.088271  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.113598  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.114797  2685 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:27.090564  9353 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.003s
I20260709 14:54:27.118906  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.121001  9306 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:27.109758  9008 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:27.122401  9008 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:27.123426  9008 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.024577  9364 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.072033  9366 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:27.136824  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123021us
I20260709 14:54:27.092207  9115 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.098429  9291 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:27.143090  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.105070  9250 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.093622  9354 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 14:54:27.116674  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:27.167469  2686 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:27.202211  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 85713.8us
I20260709 14:54:27.117130  9319 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:27.151517  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:27.155910  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:27.125136  9008 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:27.197696  2678 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:27.139762  9256 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.204555  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:27.136901  9393 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.246s	user 0.002s	sys 0.000s
I20260709 14:54:27.127655  8885 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.253347  3541 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 14:54:27.256923  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.213115  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.136772  9398 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:27.168085  9394 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 14:54:27.223107  2609 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:27.022380  9380 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:27.299464  3411 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.564s (SENT)
I20260709 14:54:27.300078  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.135535  9353 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:27.300990  9420 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:27.301118  9392 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.239385  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.302177  9374 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:27.231491  9384 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:27.304028  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1878) took 1045 ms. Trace:
I20260709 14:54:27.304540  2760 rpcz_store.cc:276] 0709 14:54:26.258652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.560408 (+301756us) service_pool.cc:224] Handling call
0709 14:54:26.560947 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:26.616100 (+ 55153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:26.616154 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:26.628987 (+ 12833us) log.cc:844] Serialized 26 byte log entry
0709 14:54:26.732344 (+103357us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:26.981788 (+249444us) log.cc:844] Serialized 14 byte log entry
0709 14:54:26.982177 (+   389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:26.982313 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:27.303747 (+321434us) raft_consensus.cc:1685] finished
0709 14:54:27.303775 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:27.303952 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":247,"prepare.queue_time_us":281467,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":120134,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":18025,"thread_start_us":320222,"threads_started":3,"wal-append.queue_time_us":196804}
I20260709 14:54:27.136148  8708 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.135442  9153 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 14:54:27.218624  9342 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:27.231935  9341 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:27.234445  9319 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:27.290266  9398 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.295336  9406 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:27.298013  9395 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 14:54:27.298707  9049 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.266189  9381 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.203548  9291 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.306797  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.342582  9377 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.343153  9004 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.343577  9377 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.346194  9399 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:27.346787  2759 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:27.347081  9399 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:27.348183  9399 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.351100  9364 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:27.351739  9405 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 14:54:27.354409  9367 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.919s	user 0.002s	sys 0.000s
I20260709 14:54:27.308619  9410 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:27.309644  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:27.312165  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 224001us
I20260709 14:54:27.369288  2751 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.370602  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:27.371706  2760 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.279981  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.219434  9250 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.319453  2677 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:27.315833  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61012.9us
I20260709 14:54:27.372920  2685 raft_consensus.cc:2368] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20.
I20260709 14:54:27.323258  2678 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:27.297214  9375 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:27.316181  9341 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:27.384109  9341 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:27.384660  9363 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.179239  9115 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.374577  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:27.375701  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.395328  2752 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:27.396459  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:27.397225  2752 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:27.373726  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:27.387962  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.367154  9368 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:27.398263  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:27.398661  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.399587  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:27.387475  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.404683  2678 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:27.404825  2685 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:27.374856  9383 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.001s
I20260709 14:54:27.416023  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.416499  3542 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:27.417665  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.420905  9115 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.422103  9396 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:27.422936  9396 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.331306  9238 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.430156  9238 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:27.433560  9291 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:27.405558  9374 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:27.401583  9368 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:27.436729  9368 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.438155  9368 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:27.435423  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.442061  2752 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:27.367396  9004 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.446367  9409 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.447259  9225 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.447176  9409 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.449942  9409 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.447897  9225 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.450596  9390 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:27.452661  8885 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:27.457139  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 82652.1us
I20260709 14:54:27.457062  3541 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:27.459180  3409 leader_election.cc:400] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20.
I20260709 14:54:27.459619  3409 leader_election.cc:403] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:54:27.460321  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 62167.3us
I20260709 14:54:27.462570  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.459928  9374 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:27.463708  9060 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:27.464262  9060 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:27.464049  9374 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.454977  9250 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.465456  9250 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20.
I20260709 14:54:27.467811  9417 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:27.402177  9406 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:27.469715  9406 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.472204  9390 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:27.473522  9390 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.464663  9060 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:54:27.445122  9352 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.000s
I20260709 14:54:27.524417  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:27.493753  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:27.525513  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 64329.1us
I20260709 14:54:27.527869  3541 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:27.528414  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.529343  2752 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20. Re-sending same reply.
W20260709 14:54:27.529846  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 70878.7us
I20260709 14:54:27.531540  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.535171  2752 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:27.537772  2685 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:27.539708  9225 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:27.540095  9357 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 14:54:27.540911  2760 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:27.542208  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.542857  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.545253  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.546366  2760 raft_consensus.cc:2368] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20.
I20260709 14:54:27.416144  9398 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:27.569875  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.533933  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:27.571967  9413 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 14:54:27.533052  9352 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.575870  9396 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:27.576695  9071 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.002s
I20260709 14:54:27.579460  9375 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:27.580636  2611 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 14:54:27.580822  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:27.581135  9096 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.580809  9291 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:27.581640  9365 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.583966  2603 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:27.584733  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:27.444896  9383 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:27.588012  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54185.7us
I20260709 14:54:27.587558  2752 raft_consensus.cc:2410] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:54:27.585721  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.588924  2685 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:27.642025  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.643110  2685 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:27.644272  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.592113  9238 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:54:27.580646  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:27.664350  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:27.664268  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.569612  8708 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:27.456880  9342 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.625828  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:27.648896  9415 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 14:54:27.616460  9071 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.651522  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.694010  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:27.698122  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:27.623922  9096 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:27.627228  9291 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.532554  9250 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:27.658731  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1740) took 2321 ms. Trace:
I20260709 14:54:27.644285  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:27.662109  2606 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:27.665856  2604 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:27.667770  2751 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
W20260709 14:54:27.647168  9229 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.007s
I20260709 14:54:27.695855  2754 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 14:54:27.568009  9319 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.735376  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.700224  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:27.701175  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:27.702486  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 76763.1us
I20260709 14:54:27.710005  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:27.663847  9238 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:27.706312  2610 rpcz_store.cc:276] 0709 14:54:25.337646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.341141 (+  3495us) service_pool.cc:224] Handling call
0709 14:54:25.361982 (+ 20841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:25.397273 (+ 35291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:25.397308 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:25.434415 (+ 37107us) log.cc:844] Serialized 26 byte log entry
0709 14:54:27.077641 (+1643226us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b18003f3a00. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:27.402103 (+324462us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:27.402175 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:27.402262 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:27.658346 (+256084us) raft_consensus.cc:1685] finished
0709 14:54:27.658371 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:27.658649 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":598,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7623,"spinlock_wait_cycles":12301696,"thread_start_us":324255,"threads_started":1,"wal-append.queue_time_us":2221683}
I20260709 14:54:27.654743  9060 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:27.721045  9377 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.721078  9096 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:27.739557  9238 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.755218  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1689) took 2064 ms. Trace:
I20260709 14:54:27.755960  2679 rpcz_store.cc:276] 0709 14:54:25.690472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.115993 (+425521us) service_pool.cc:224] Handling call
0709 14:54:26.155636 (+ 39643us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:26.198277 (+ 42641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:26.198329 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:26.211945 (+ 13616us) log.cc:844] Serialized 52 byte log entry
0709 14:54:26.755151 (+543206us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b180040ff60. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:27.197774 (+442623us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:27.239219 (+ 41445us) log.cc:844] Serialized 14 byte log entry
0709 14:54:27.239470 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:27.239566 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:27.754790 (+515224us) raft_consensus.cc:1685] finished
0709 14:54:27.754935 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:27.755127 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46152,"prepare.queue_time_us":41254,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":9222,"raft.queue_time_us":41076,"raft.run_cpu_time_us":509,"raft.run_wall_time_us":40285,"spinlock_wait_cycles":10545024,"thread_start_us":442342,"threads_started":1,"wal-append.queue_time_us":1228425}
I20260709 14:54:27.727079  9390 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:27.759709  9390 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:27.760370  9390 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:27.762776  9423 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 14:54:27.767118  9374 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:27.768116  2608 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 23
I20260709 14:54:27.630286  9375 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.740684  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.747192  2751 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:54:27.641149  9365 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.727393  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1693) took 1084 ms. Trace:
I20260709 14:54:27.772274  9374 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:27.772964  3409 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.557s (SENT)
I20260709 14:54:27.752385  9352 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:27.778515  2683 rpcz_store.cc:276] 0709 14:54:26.642605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.642759 (+   154us) service_pool.cc:224] Handling call
0709 14:54:27.727099 (+1084340us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:27.727301 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121478016}
I20260709 14:54:27.780412  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.706023  9250 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:27.713706  9381 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:27.766531  9388 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.697s	user 0.018s	sys 0.000s
I20260709 14:54:27.782136  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:27.792477  2685 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:27.793321  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.794420  2610 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:27.787858  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.788367  3411 leader_election.cc:400] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20.
I20260709 14:54:27.796564  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:27.799511  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:27.802491  2683 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 14:54:27.810580  2611 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:27.791396  9429 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:27.788918  9262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.581s	user 0.013s	sys 0.032s
I20260709 14:54:27.772384  9096 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:27.832024  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.790859  9250 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.743969  9229 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:27.841696  2683 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:27.854200  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.855273  2608 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:27.855683  2603 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 14.
I20260709 14:54:27.769476  8708 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 14:54:27.859958  2758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1214) took 2107 ms (client timeout 1768 ms). Trace:
W20260709 14:54:27.859215  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 111106us
W20260709 14:54:27.833094  9262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.652s	user 0.014s	sys 0.033s
W20260709 14:54:27.874289  2758 rpcz_store.cc:269] 0709 14:54:25.752401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.840064 (+2087663us) service_pool.cc:211] Skipping call since client already timed out
0709 14:54:27.859855 (+ 19791us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 14:54:27.920672  2679 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:27.786643  9341 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.948637  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:27.929539  2683 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 14:54:27.958954  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:27.963660  9426 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 14:54:27.983620  2685 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:27.963461  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.983846  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.986306  3538 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:27.955217  9429 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:27.810698  9388 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:27.860913  9096 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:27.977849  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:27.959482  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.980515  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.964247  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.965075  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:27.928635  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:27.998522  9060 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:28.000162  9388 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:28.002370  2683 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:27.990320  3540 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:28.003777  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:28.055239  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 282305us
I20260709 14:54:28.059435  3543 raft_consensus.cc:2368] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18.
I20260709 14:54:28.060600  3542 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 23
W20260709 14:54:27.979861  9368 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.004s
I20260709 14:54:28.065793  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:27.971040  9377 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 14:54:27.970311  9352 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 14:54:27.983865  9116 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:28.127995  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62253us
I20260709 14:54:28.071139  9060 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:28.136283  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:28.073550  9430 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 14:54:28.065971  9383 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:28.054978  9427 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:28.139148  2603 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:28.142474  3538 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:28.149734  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:28.140287  9433 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 14:54:28.175489  9435 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 14:54:28.177534  9437 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 14:54:28.180052  9345 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.145s	user 0.015s	sys 0.000s
I20260709 14:54:28.180132  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.181703  9388 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:28.145102  9427 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:28.182055  2608 raft_consensus.cc:2376] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18. Re-sending same reply.
I20260709 14:54:28.182309  9388 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:28.182957  9427 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:28.184265  9341 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:28.186241  2603 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 14:54:28.184875  9291 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.187495  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.188962  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.189268  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:28.184350  9388 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:28.144595  9071 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:28.180698  9345 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:28.210002  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.324749  2605 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:28.190624  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.246186  2755 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:28.198638  2606 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:28.206439  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:28.218532  9434 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 14:54:28.232959  2684 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 14:54:28.204871  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:28.215657  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1727) took 3825 ms (client timeout 1828 ms). Trace:
I20260709 14:54:28.190480  9377 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:28.185570  2683 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:28.345899  2684 pending_rounds.cc:85] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 12
W20260709 14:54:28.337971  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:28.251689  9438 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.425s	user 0.002s	sys 0.000s
I20260709 14:54:28.191290  9388 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:28.333376  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.323043  9341 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:28.349637  2684 pending_rounds.cc:107] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 14:54:28.346987  9442 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:28.345808  2756 rpcz_store.cc:269] 0709 14:54:24.390640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.489380 (+1098740us) service_pool.cc:224] Handling call
0709 14:54:28.215169 (+2725789us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b540040ecd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:28.215476 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032896256}
I20260709 14:54:28.368798  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.369571  2603 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:28.369748  2756 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 14:54:28.371523  2756 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 14:54:28.355363  2682 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:28.199021  9365 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:28.374986  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:28.376825  9238 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.001s
I20260709 14:54:28.378247  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1753) took 1408 ms. Trace:
I20260709 14:54:28.378723  2609 rpcz_store.cc:276] 0709 14:54:26.969784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.975455 (+  5671us) service_pool.cc:224] Handling call
0709 14:54:28.378000 (+1402545us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:28.378179 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478341120}
I20260709 14:54:28.384681  3542 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:28.386511  2605 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:28.250939  9429 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 14:54:28.191257  9374 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:28.200196  9229 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.240947  9096 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:28.233366  9362 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 2.061s	user 0.001s	sys 0.005s
I20260709 14:54:28.365782  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.367112  2684 raft_consensus.cc:2889] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:28.394302  2607 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:28.368053  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.368911  9442 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.196493  9427 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.364080  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.406877  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:28.392819  9331 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.486s	user 0.018s	sys 0.000s
I20260709 14:54:28.410759  2605 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:28.399361  9008 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.111s	user 0.011s	sys 0.011s
I20260709 14:54:28.411629  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.417559  2609 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:28.377406  9238 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:28.420050  2608 raft_consensus.cc:2368] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
I20260709 14:54:28.370697  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.400446  9311 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:28.424714  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:28.397462  9429 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:28.426259  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.393451  9341 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.436167  3536 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:28.438759  9341 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.441526  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1694) took 1798 ms. Trace:
I20260709 14:54:28.446105  2686 rpcz_store.cc:276] 0709 14:54:26.642995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.643136 (+   141us) service_pool.cc:224] Handling call
0709 14:54:26.643444 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:27.202247 (+558803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.441159 (+1238912us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54000c9ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:28.441441 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204583168}
I20260709 14:54:28.449435  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1207) took 2853 ms. Trace:
I20260709 14:54:28.449819  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.451599  2605 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:28.457935  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.451236  2680 rpcz_store.cc:276] 0709 14:54:25.596491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.644218 (+ 47727us) service_pool.cc:224] Handling call
0709 14:54:25.683974 (+ 39756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:25.691571 (+  7597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:25.691611 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:25.766565 (+ 74954us) log.cc:844] Serialized 26 byte log entry
0709 14:54:25.943154 (+176589us) raft_consensus.cc:1645] Marking committed up to 7
0709 14:54:25.943214 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:25.943320 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:28.232341 (+2289021us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:28.442551 (+210210us) raft_consensus.cc:1685] finished
0709 14:54:28.442580 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:28.449323 (+  6743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265825152,"thread_start_us":175876,"threads_started":1,"wal-append.queue_time_us":404575}
I20260709 14:54:28.433686  9020 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:28.434036  9429 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:28.479327  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1233) took 1508 ms. Trace:
I20260709 14:54:28.420416  9331 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.480120  2678 rpcz_store.cc:276] 0709 14:54:26.970424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.458535 (+488111us) service_pool.cc:224] Handling call
0709 14:54:27.458964 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:27.524686 (+ 65722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:27.524721 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:27.594625 (+ 69904us) log.cc:844] Serialized 52 byte log entry
0709 14:54:27.777000 (+182375us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:27.807926 (+ 30926us) log.cc:844] Serialized 14 byte log entry
0709 14:54:27.808236 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:27.808349 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:28.218044 (+409695us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b240021b9d8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:54:28.475095 (+257051us) raft_consensus.cc:1685] finished
0709 14:54:28.478899 (+  3804us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:28.479205 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10240,"prepare.queue_time_us":35342,"raft.queue_time_us":32433,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":30878,"spinlock_wait_cycles":138383104,"thread_start_us":135851,"threads_started":1,"wal-append.queue_time_us":437706}
I20260709 14:54:28.480679  2605 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:28.433825  9311 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:28.476469  9020 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:28.498603  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.500193  2605 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:28.490692  9381 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.011s
I20260709 14:54:28.517319  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.518558  2611 raft_consensus.cc:2376] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19. Re-sending same reply.
I20260709 14:54:28.520828  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.521872  2605 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:28.522625  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.523087  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.523726  2611 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:28.523893  2605 raft_consensus.cc:2376] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20. Re-sending same reply.
I20260709 14:54:28.525190  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.525863  2605 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:28.526351  2611 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:28.526923  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.008s	sys 0.025s
W20260709 14:54:28.527284  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.025s
W20260709 14:54:28.527940  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 728513us
I20260709 14:54:28.529054  2611 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:28.528039  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1227) took 2312 ms (client timeout 1557 ms). Trace:
I20260709 14:54:28.492588  9238 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:28.531591  2681 rpcz_store.cc:269] 0709 14:54:26.216216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.527810 (+2311594us) service_pool.cc:211] Skipping call since client already timed out
0709 14:54:28.527929 (+   119us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 14:54:28.517077  9381 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.004s	sys 0.011s
I20260709 14:54:28.515468  9020 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:28.547617  9442 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.550733  9311 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:28.551849  9331 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.527923  2605 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:28.538893  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1236) took 1270 ms. Trace:
I20260709 14:54:28.538444  2607 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:28.531598  9375 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.553445  2685 rpcz_store.cc:276] 0709 14:54:27.268468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.986227 (+717759us) service_pool.cc:224] Handling call
0709 14:54:27.986725 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:27.987121 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:28.329593 (+342472us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.534003 (+204410us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.534547 (+   544us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.537961 (+  3414us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.538261 (+   300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:28.538341 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:28.538435 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.538567 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:28.538768 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":203464,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":430,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12549,"spinlock_wait_cycles":36734720,"thread_start_us":443833,"threads_started":2}
W20260709 14:54:28.549248  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:28.539604  9345 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:28.556870  2607 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:28.557737  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:28.558563  2605 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:28.543206  9381 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:28.566250  2607 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 14:54:28.556268  9337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.163s	user 0.028s	sys 0.008s
I20260709 14:54:28.591519  2608 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:28.580328  9442 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:28.600831  2609 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:28.600509  9442 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:28.604630  9390 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.844s	user 0.017s	sys 0.001s
I20260709 14:54:28.605513  9390 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.612253  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1751) took 1818 ms. Trace:
I20260709 14:54:28.613050  2759 rpcz_store.cc:276] 0709 14:54:26.794017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.076393 (+282376us) service_pool.cc:224] Handling call
0709 14:54:27.076751 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:27.347296 (+270545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.611789 (+1264493us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:28.612140 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620523776}
I20260709 14:54:28.597319  2608 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:28.622584  2609 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 14:54:28.625272  2607 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 14:54:28.614391  9390 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:28.579964  9337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.189s	user 0.029s	sys 0.008s
I20260709 14:54:28.628177  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:28.638175  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:28.629197  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.578177  9427 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.631274  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:28.627673  9238 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:28.646054  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:28.629140  9451 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:28.628065  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:28.644793  2681 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:28.644546  2678 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:28.634222  2608 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 14:54:28.656206  2686 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:54:28.659997  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:28.660558  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.661069  2680 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:28.665141  9444 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:28.665454  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.665827  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:28.666730  9442 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:28.666930  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:28.667481  9442 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:28.667878  9442 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:28.644188  9445 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 14:54:28.668582  9442 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:28.668527  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.669076  9445 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:28.669503  9445 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:28.670590  2684 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 14:54:28.667840  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:28.673919  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:28.674744  2684 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:28.674414  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:28.679955  2681 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 14:54:28.692934  9356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.293s	user 0.020s	sys 0.010s
W20260709 14:54:28.693811  9356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.294s	user 0.021s	sys 0.010s
I20260709 14:54:28.705742  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.708112  2686 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:28.708771  2683 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 14:54:28.713565  9456 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 14:54:28.717645  9455 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 14:54:28.717718  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49980.2us
I20260709 14:54:28.720222  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:28.650539  9238 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:28.663022  9229 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:28.723510  9060 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:28.736689  2678 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:28.731990  9460 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 14:54:28.653774  9381 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:28.755712  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:28.709669  9389 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:28.768636  2680 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:28.750694  9250 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.833s	user 0.009s	sys 0.003s
W20260709 14:54:28.821157  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44233.2us
I20260709 14:54:28.761552  9390 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:28.766371  9388 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:54:28.826653  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 152287us
I20260709 14:54:28.827035  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1225) took 2761 ms. Trace:
I20260709 14:54:28.764671  9389 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:28.832692  2757 rpcz_store.cc:276] 0709 14:54:26.065537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.487592 (+422055us) service_pool.cc:224] Handling call
0709 14:54:26.713039 (+225447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:26.755220 (+ 42181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.826754 (+2071534us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400571050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:28.826966 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478581120}
W20260709 14:54:28.772657  9436 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.008s	user 0.000s	sys 0.011s
I20260709 14:54:28.768481  9230 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:28.840243  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1197) took 1251 ms. Trace:
I20260709 14:54:28.840822  2611 rpcz_store.cc:276] 0709 14:54:27.581716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.530006 (+948290us) service_pool.cc:224] Handling call
0709 14:54:28.530566 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.530820 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:28.630024 (+ 99204us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.832845 (+202821us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.833177 (+   332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:28.833241 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:28.833301 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.833377 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:28.833533 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30840,"prepare.queue_time_us":183297,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":4680,"spinlock_wait_cycles":5596928,"thread_start_us":241563,"threads_started":2}
W20260709 14:54:28.849205  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:28.851545  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.832007  9427 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:28.843050  9388 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:28.832106  9390 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:28.835026  9230 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:28.779723  9250 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:28.826287  9229 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 14:54:28.865402  9361 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 2.699s	user 0.006s	sys 0.000s
I20260709 14:54:28.865180  9442 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.847841  9356 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:28.865983  9390 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:28.654651  9451 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:28.883570  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:28.885738  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:28.886072  2611 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:28.872906  9230 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:28.899837  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.901270  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.860317  9374 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 14:54:28.904927  2650 connection.cc:256] RPC call timeout handler was delayed by -1.0851912400000856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 14:54:28.907316  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.909422  3538 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:28.936153  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:28.936991  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.937415  2683 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:28.937991  3542 raft_consensus.cc:2410] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 14:54:28.889036  9469 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 14:54:28.915913  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.919153  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:28.920003  9461 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 14:54:28.920627  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1203) took 1279 ms. Trace:
I20260709 14:54:28.973663  2606 rpcz_store.cc:276] 0709 14:54:27.641185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.709483 (+ 68298us) service_pool.cc:224] Handling call
0709 14:54:28.920348 (+1210865us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:28.920539 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923340672}
I20260709 14:54:28.923554  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1217) took 3169 ms. Trace:
I20260709 14:54:28.975186  2755 rpcz_store.cc:276] 0709 14:54:25.753714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:25.968173 (+214459us) service_pool.cc:224] Handling call
0709 14:54:28.245815 (+2277642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.332138 (+ 86323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.923200 (+591062us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b540040ecd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:28.923454 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5181989760}
W20260709 14:54:28.927506  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:28.977161  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:28.930192  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1697) took 2223 ms. Trace:
W20260709 14:54:28.932415  9457 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 14:54:28.919286  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:28.985147  3543 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:28.941844  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:28.986358  3541 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 14:54:28.881485  9356 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:28.867779  9427 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:28.670466  9445 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:28.946605  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:28.970679  3545 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:28.949719  3537 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:28.979194  2682 rpcz_store.cc:276] 0709 14:54:26.706286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.418676 (+712390us) service_pool.cc:224] Handling call
0709 14:54:27.441577 (+ 22901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.373028 (+931451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.929719 (+556691us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:28.930078 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100769024}
I20260709 14:54:28.988859  2684 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:28.972328  2575 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:28.990422  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:29.003336  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:29.003027  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:28.934625  9404 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:29.011988  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65522.4us
I20260709 14:54:28.987562  9336 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.007513  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:29.018008  8885 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:29.020398  3540 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:29.019052  9319 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:29.027830  9466 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 14:54:29.049695  2611 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:29.052939  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:29.067780  3541 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:29.055696  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:29.070586  2682 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:29.004328  9469 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.048696  2609 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 14:54:29.025547  9442 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.096802  3538 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 14:54:29.097306  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1769) took 4923 ms (client timeout 1923 ms). Trace:
I20260709 14:54:28.903260  9389 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.060976  9336 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:29.101104  3546 rpcz_store.cc:269] 0709 14:54:24.146708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:24.167996 (+ 21288us) service_pool.cc:224] Handling call
0709 14:54:29.069600 (+4901604us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400551254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:29.069723 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3426226944}
I20260709 14:54:29.072784  9445 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:29.109947  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 262617us
I20260709 14:54:29.004341  9388 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.040105  9404 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:29.102135  9457 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 14:54:29.146592  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.105289  8885 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:29.056988  9311 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 14:54:29.161062  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:29.099375  9244 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 14:54:29.103709  9469 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.073153  9356 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.119403  9471 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 14:54:29.238557  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 139127us
W20260709 14:54:29.241380  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 255300us
I20260709 14:54:29.234925  9356 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:29.246927  9020 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.692s	user 0.013s	sys 0.003s
I20260709 14:54:29.236238  9250 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:29.298471  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 137514us
I20260709 14:54:29.229420  9229 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 14:54:29.246760  8885 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:29.232884  9060 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 14:54:29.319391  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1923) took 1240 ms. Trace:
I20260709 14:54:29.320034  2607 rpcz_store.cc:276] 0709 14:54:28.079067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.641141 (+562074us) service_pool.cc:224] Handling call
0709 14:54:28.641675 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.642191 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:28.763926 (+121735us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.318674 (+554748us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.318932 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:29.318994 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:29.319067 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.319148 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.319316 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":123007,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":64,"thread_start_us":653285,"threads_started":3,"wal-append.queue_time_us":547085}
I20260709 14:54:29.298594  9238 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.328713  9450 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:29.304347  9374 consensus_queue.cc:1243] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:29.318552  9331 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 14:54:29.302624  9475 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 14:54:29.317209  8885 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.359300  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1235) took 2344 ms. Trace:
W20260709 14:54:29.368391  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 211183us
I20260709 14:54:29.382707  2679 rpcz_store.cc:276] 0709 14:54:27.015028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.922580 (+907552us) service_pool.cc:224] Handling call
0709 14:54:27.980088 (+ 57508us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:27.981579 (+  1491us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:28.240304 (+258725us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b1800483c40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:28.375705 (+135401us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.625838 (+250133us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:28.651289 (+ 25451us) log.cc:844] Serialized 52 byte log entry
0709 14:54:28.653087 (+  1798us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:28.664164 (+ 11077us) log.cc:844] Serialized 14 byte log entry
0709 14:54:28.664489 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:28.664617 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:29.358994 (+694377us) raft_consensus.cc:1685] finished
0709 14:54:29.359021 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.359222 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1411,"prepare.queue_time_us":199120,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":128,"raft.queue_time_us":3520,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":17907,"spinlock_wait_cycles":17322112,"thread_start_us":615891,"threads_started":2,"wal-append.queue_time_us":270024}
I20260709 14:54:29.386767  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:29.300738  9020 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.357698  8708 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:29.399158  9336 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:29.400442  9467 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 14:54:29.402026  9404 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:29.405848  9230 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.004s
I20260709 14:54:29.408133  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1881) took 2311 ms. Trace:
I20260709 14:54:29.431210  2751 rpcz_store.cc:276] 0709 14:54:27.097150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.734247 (+637097us) service_pool.cc:224] Handling call
0709 14:54:27.734864 (+   617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:27.775470 (+ 40606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.407724 (+1632254us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:29.408044 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27269888}
I20260709 14:54:29.318850  9476 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 14:54:29.418799  2682 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:29.359400  9450 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:29.396145   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 14:54:29.438612  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.335695  9451 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.441078  2685 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:29.443202  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 378619us
I20260709 14:54:29.443619  8708 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:29.451424  9432 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.680s	user 0.005s	sys 0.007s
I20260709 14:54:29.451237  8708 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.452694  9450 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.454015  9450 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:29.454412  8885 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.443549  9484 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.455775  9484 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:29.456990  2724 connection.cc:256] RPC call timeout handler was delayed by -1.4825567350000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 14:54:29.457578  2724 connection.cc:256] RPC call timeout handler was delayed by -1.4176331700000446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 14:54:29.459213  9484 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.460528  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:29.432169  9427 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.438419  9374 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.439383  9442 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:29.486218  9252 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:29.442301  9230 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:29.511154  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.493026  9402 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.497804  9411 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:29.535938  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51340.3us
I20260709 14:54:29.504295  9412 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:29.496347  9442 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.546942  3536 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: []
W20260709 14:54:29.548799  9429 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.059s	user 0.016s	sys 0.001s
W20260709 14:54:29.550210  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.549674  9429 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:29.554926  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:29.551931  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:29.564181  2612 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:29.552944  2604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 1761) took 2552 ms (client timeout 1666 ms). Trace:
I20260709 14:54:29.498641  9484 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 14:54:29.534673  9402 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.511269  9252 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:29.561205  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100790us
I20260709 14:54:29.578162  2607 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:29.578796  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:29.499229  9374 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.583487  9374 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.498253  9451 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 14:54:29.565783  2604 rpcz_store.cc:269] 0709 14:54:27.000903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.001108 (+   205us) service_pool.cc:224] Handling call
0709 14:54:29.552705 (+2551597us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54001eb754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:29.552858 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485385088}
I20260709 14:54:29.570242  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:29.518395  9319 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:29.585840  9319 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.588443  2603 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:29.506932  9391 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 14:54:29.601081  9483 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 14:54:29.554232  9469 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.608737  2612 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:29.574935  9388 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:29.615733  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1226967549999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2439s for a response
I20260709 14:54:29.592948  9357 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:29.623224  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:29.624560  9353 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:29.632844  9353 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.629890  9207 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:29.641397  9402 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:29.641867  9353 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.656567  9391 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:29.657936  9163 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:29.658752  9163 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:29.660105  9488 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 14:54:29.660310  9163 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:29.661866  9491 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 14:54:29.665529  9485 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 14:54:29.667788  9008 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:29.668803  9008 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:29.671643  9490 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 14:54:29.671792  9355 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:29.672959  9355 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:29.684682  9230 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.686882  9442 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:29.688390  9487 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 14:54:29.538709  9411 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:29.657116  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1239) took 2208 ms. Trace:
I20260709 14:54:29.665285  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:29.666946  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 96818.1us
I20260709 14:54:29.598404  9476 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 14:54:29.714396  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:29.715476  2756 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:29.716921  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:29.721907  2756 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:29.723526  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:29.728212  2607 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:29.562309  9412 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:29.731760  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66627.8us
I20260709 14:54:29.709827  2758 rpcz_store.cc:276] 0709 14:54:27.449111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.060900 (+611789us) service_pool.cc:224] Handling call
0709 14:54:28.061404 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.061745 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:28.455354 (+393609us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.609032 (+1153678us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.656449 (+ 47417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:29.656562 (+   113us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:29.656656 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.656773 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.657013 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1019741,"prepare.queue_time_us":654156,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":77,"raft.queue_time_us":1027069,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":156,"spinlock_wait_cycles":17443072,"thread_start_us":553956,"threads_started":3,"wal-append.queue_time_us":174007}
W20260709 14:54:29.730679  2726 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.953s (SENT)
I20260709 14:54:29.733836  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1240) took 2284 ms. Trace:
I20260709 14:54:29.734076  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:29.734125  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:29.733837  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:29.735157  2759 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:29.736469  2755 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 15: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:29.735291  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:29.726994  9445 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.011s
I20260709 14:54:29.738883  2757 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:29.690676  9429 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.627336  9357 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:29.662714  9163 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:29.712524  9020 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.751631  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:29.761338  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:29.756991  9386 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 14:54:29.774757  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:29.790237  2608 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
W20260709 14:54:29.768069  2726 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 2.121s (SENT)
I20260709 14:54:29.791617  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:29.722229  9263 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:29.728700  9342 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.792948  9391 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:29.792840  9263 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.796142  9263 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:29.753182  3547 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 14:54:29.800027  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.730755  9392 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:29.803298  9392 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:29.805852  9392 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.782919  9353 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 14:54:29.808324  2608 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 15.
I20260709 14:54:29.810146  9475 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:29.812278  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:29.813746  9463 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.973s	user 0.002s	sys 0.000s
I20260709 14:54:29.820964  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:29.822140  9386 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:29.822600  9386 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:29.822992  9461 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 14:54:29.730693  9207 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.674373  9355 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.737752  9252 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.775264  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:29.760982  9412 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.786314  2751 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:29.828883  2607 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:29.786409  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.792840  9342 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.730510  9406 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.781575  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1752) took 2987 ms. Trace:
I20260709 14:54:29.830761  2760 rpcz_store.cc:276] 0709 14:54:26.794313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.646520 (+852207us) service_pool.cc:224] Handling call
0709 14:54:27.646960 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:29.463788 (+1816828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:29.463840 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:29.471524 (+  7684us) log.cc:844] Serialized 26 byte log entry
0709 14:54:29.610481 (+138957us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:29.610532 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.610619 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:29.781176 (+170557us) raft_consensus.cc:1685] finished
0709 14:54:29.781215 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.781455 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1854500,"raft.queue_time_us":1857311,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":12300,"thread_start_us":117577,"threads_started":1,"wal-append.queue_time_us":291936}
I20260709 14:54:29.825811  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:29.833747  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:29.835294  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:29.837329  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:29.754580  9390 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.858s	user 0.014s	sys 0.003s
I20260709 14:54:29.810645  9475 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:29.827365  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:29.828053  2680 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:29.834735  2760 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:29.843775  2759 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:29.823637  9386 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.852310  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:29.787436  9357 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:29.670382  9008 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.828092  9207 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.894902  2607 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:54:29.861867  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:29.839809  9392 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:29.852582  9252 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:29.864944  9342 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.846997  9445 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:29.915558  9445 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:29.916666  9445 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.854698  9406 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:29.920039  9389 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:29.921415  9406 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.827998  9060 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 14:54:29.925345  2608 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 14:54:29.926533  9475 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.927834  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:29.929370  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:29.930287  2755 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:29.931172  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1880) took 2834 ms. Trace:
I20260709 14:54:29.856691  9429 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 14:54:29.931635  2752 rpcz_store.cc:276] 0709 14:54:27.096826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.736387 (+639561us) service_pool.cc:224] Handling call
0709 14:54:27.736966 (+   579us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:29.487601 (+1750635us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:29.586931 (+ 99330us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.740809 (+153878us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:29.769683 (+ 28874us) log.cc:844] Serialized 52 byte log entry
0709 14:54:29.769944 (+   261us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:29.769995 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.770102 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:29.930926 (+160824us) raft_consensus.cc:1685] finished
0709 14:54:29.930950 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.931110 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1807787,"prepare.queue_time_us":109598,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":1778816,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":20012,"thread_start_us":222114,"threads_started":2,"wal-append.queue_time_us":308589}
I20260709 14:54:29.933884  9207 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:29.873591  9390 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:29.871835  9386 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:29.936590  9390 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:29.852430  9356 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:29.951748  9356 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:29.906278  9020 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 14:54:29.954416  9356 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.881349  9244 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:29.884958  9388 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:29.886579  9446 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:29.888208  9411 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.888550  9027 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:29.890409  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:29.890906  2652 leader_election.cc:400] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
I20260709 14:54:30.040271  2652 leader_election.cc:403] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:29.893543  2751 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:29.900699  2757 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:29.900689  2760 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:29.902163  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:30.041208  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 150563us
I20260709 14:54:30.042685  2685 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:29.905130  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:29.898690  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:29.853703  9404 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:29.909085  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:29.914448  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 146414us
I20260709 14:54:29.928110  9475 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:29.878026  9412 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:29.940304  9391 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:29.880076  8885 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:29.960309  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1789) took 1059 ms. Trace:
I20260709 14:54:30.067704  3547 rpcz_store.cc:276] 0709 14:54:28.900821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.900968 (+   147us) service_pool.cc:224] Handling call
0709 14:54:29.944963 (+1043995us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54002dc3d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:29.960209 (+ 15246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956796800}
I20260709 14:54:29.964731  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:29.966140  2677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1692) took 3325 ms (client timeout 1831 ms). Trace:
I20260709 14:54:29.966434  2678 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 14:54:29.975380  2607 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:30.070076  2677 rpcz_store.cc:269] 0709 14:54:26.641136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:26.641347 (+   211us) service_pool.cc:224] Handling call
0709 14:54:29.965801 (+3324454us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54001d15d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:29.966050 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505650432}
W20260709 14:54:29.980453  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:29.983161  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.073582  2686 raft_consensus.cc:2410] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 20 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:29.986441  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1205) took 2274 ms. Trace:
I20260709 14:54:29.988873  2608 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 14:54:30.042109  9392 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.029582  9004 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:30.030740  9469 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:30.084127  2576 connection.cc:256] RPC call timeout handler was delayed by -1.5583690500000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 14:54:30.097225  2576 connection.cc:256] RPC call timeout handler was delayed by -1.2739690500000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 14:54:30.074764  2605 rpcz_store.cc:276] 0709 14:54:27.712095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.566138 (+854043us) service_pool.cc:224] Handling call
0709 14:54:28.566495 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.566784 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:28.651202 (+ 84418us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.465432 (+814230us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b180010b460. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a
0709 14:54:29.978239 (+512807us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.978776 (+   537us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.979002 (+   226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:29.979066 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:29.979157 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.979278 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.986357 (+  7079us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":515707,"prepare.queue_time_us":706545,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":7105,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":16353,"spinlock_wait_cycles":73212928,"thread_start_us":206823,"threads_started":2,"wal-append.queue_time_us":228008}
I20260709 14:54:30.027433  9244 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.088312  9446 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.065950  9391 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:30.041146  9071 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:30.056348  9442 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 14:54:30.113629  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:30.097819  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0417690499999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 14:54:30.075663  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.115454  9250 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:30.115962  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.116384  9250 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.118078  9402 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:30.107353  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 208816us
I20260709 14:54:30.109984  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.110348  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.120048  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.120569  2751 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:30.111742  2759 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:30.114524  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0345690499999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 14:54:30.122169  2576 connection.cc:256] RPC call timeout handler was delayed by -1.0174690500000452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 14:54:30.122689  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.123389  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:30.124430  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.126014  2576 leader_election.cc:400] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18.
I20260709 14:54:30.126596  2576 leader_election.cc:403] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 14:54:30.116436  2755 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:30.137207  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.090327  9319 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:30.153632  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.153965  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.155269  2759 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:30.068889  9404 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.160041  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.161110  2751 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:30.161554  3536 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 14:54:30.120829  9060 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:30.156769  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.165758  3537 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:30.112597  9429 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.121186  9442 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.119153  9262 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 14:54:30.178284  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1886) took 3076 ms. Trace:
I20260709 14:54:30.133304  9331 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:30.179551  9331 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:30.178810  2754 rpcz_store.cc:276] 0709 14:54:27.102052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:27.722256 (+620204us) service_pool.cc:224] Handling call
0709 14:54:27.722878 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:29.471832 (+1748954us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:29.471865 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:29.472587 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 14:54:30.027897 (+555310us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:30.027947 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:30.028041 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:30.177978 (+149937us) raft_consensus.cc:1685] finished
0709 14:54:30.178009 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:30.178206 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1736882,"raft.queue_time_us":1784264,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":18429,"thread_start_us":494527,"threads_started":1,"wal-append.queue_time_us":660841}
I20260709 14:54:30.180333  9244 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.180132  9331 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.181278  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.182124  2754 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:30.097795  9392 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.151718  9489 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.191059  9489 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.193694  9489 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:30.194075  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.195485  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.163389  9383 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18.
I20260709 14:54:30.113108  9469 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.205300  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:30.206861  9230 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:30.140298  9071 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.209057  2757 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:30.085457  8885 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.112994  9345 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
W20260709 14:54:30.172878  9504 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 14:54:30.172077  9429 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.216566  9357 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.218122  2758 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:30.220906  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.221877  2755 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:30.231207  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.232822  2684 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:30.249842  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.251060  3546 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:30.131711  9446 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.260588  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.256525  2684 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:30.257488  2682 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:30.266057  2751 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:30.272033  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.274982  2686 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:30.276674  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.278167  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.279456  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.281170  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.170678  9020 consensus_queue.cc:1243] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:30.210628  9071 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.283931  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.252409  9060 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.247571  9383 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:30.304569  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.304977  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.304725  9244 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:30.305589  2751 raft_consensus.cc:2376] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:30.305887  9244 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:30.306402  9244 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:30.306610  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.307775  2751 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 14:54:30.307638  9244 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.289342  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:30.309530  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 266222us
I20260709 14:54:30.290221  3541 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:30.306473  2682 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:30.311110  2752 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 14:54:30.311961  2752 pending_rounds.cc:85] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 10
I20260709 14:54:30.312296  2752 pending_rounds.cc:107] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 14:54:30.312628  2752 raft_consensus.cc:2889] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:30.242332  9027 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.299592  3543 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:30.258102  9230 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.163085  9319 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.241489  9345 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.258363  9388 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:30.347214  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.302201  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.303344  3540 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:30.298481  9020 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:30.243459  8885 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:30.295846  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:30.288108  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:30.355818  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53785.6us
I20260709 14:54:30.309356  9244 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:30.360812  3546 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:30.366266  3537 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 15: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:30.369441  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.370703  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.371027  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.348579  9383 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:30.350791  9357 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:30.378741  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.371663  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.304569  9060 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.378208  9008 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.381992  9008 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.352089  9250 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:30.364943  9386 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:30.387255  9386 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:30.387714  9386 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:30.348018  9230 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.391405  9230 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:30.355057  9511 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 14:54:30.392184  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.392937  2758 raft_consensus.cc:2376] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:30.362792  9331 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:30.393968  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.397199  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.398113  2679 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:30.355219  9388 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.398903  9513 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 14:54:30.400112  9383 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.400755  9513 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 14:54:30.401175  9388 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:30.401898  3542 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:30.375324  9406 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.403636  9406 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.404506  3542 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:30.405113  9375 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:30.405565  9375 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:30.406347  9406 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:30.375538  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.340408  9027 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.347838  9345 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.381184  2682 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:30.394661  2758 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:30.372057  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.426141  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.429250  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.387552  9319 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:30.453616  2759 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:30.367226  9020 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.407907  9469 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:30.393970  9404 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.463383  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.464098  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.455894  3546 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:30.486317  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.457613  2679 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 14:54:30.370720  8885 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 14:54:30.459232  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.457324  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:30.454818  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 328999us
I20260709 14:54:30.492743  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:30.466248  2677 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:30.471372  9391 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.496940  9391 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.500104  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.500977  3540 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:30.501868  9020 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:29.734344  2753 rpcz_store.cc:276] 0709 14:54:27.449472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.234164 (+784692us) service_pool.cc:224] Handling call
0709 14:54:28.234664 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.235100 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:28.489764 (+254664us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.709829 (+1220065us) log.cc:844] Serialized 14 byte log entry
0709 14:54:29.733417 (+ 23588us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:29.733502 (+    85us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:29.733560 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:29.733629 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:29.733767 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":928788,"prepare.queue_time_us":506640,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":932013,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":10574,"thread_start_us":550544,"threads_started":3,"wal-append.queue_time_us":161980}
I20260709 14:54:30.501935  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:30.504151  9468 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:30.504724  9468 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.504843  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.506264  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.392019  9337 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 14:54:30.504057  9513 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:30.490180  3542 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 14:54:30.506325  2758 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:54:30.509757  2756 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 14:54:30.407620  9514 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 14:54:30.406617  9375 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:30.514772  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:30.518114  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 370830us
W20260709 14:54:30.486214  9252 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.000s
I20260709 14:54:30.506022  9468 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.538364  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:30.504647  9027 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.552277  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.566848  3546 raft_consensus.cc:2368] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20.
I20260709 14:54:30.546759  2755 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:30.568233  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:30.555519  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.569270  3540 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:30.564683  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:30.572829  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:30.578076  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:30.523131  9319 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 14:54:30.580365  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:30.588562  2686 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:30.585131  2751 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:30.583477  3541 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 14:54:30.595985  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 400603us
I20260709 14:54:30.566061  9331 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:30.567715  9391 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:30.600802  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 266346us
I20260709 14:54:30.594313  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:30.607997  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.576624  9388 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:30.586681  9468 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:30.588960  9374 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:54:30.576130  9392 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.614595  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:30.590272  9469 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:30.625928  8708 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:30.626662  8708 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.627135  9475 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:30.627756  9475 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.627810  8708 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.628824  9475 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.633729  9375 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:30.634163  9250 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.634935  9250 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.637537  9230 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.638346  9230 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.641069  9230 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:30.643709  9250 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.648397  9250 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:30.599803  9450 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:30.665226  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:30.669842  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 304635us
I20260709 14:54:30.617296  9445 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 14:54:30.677860  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70662.3us
I20260709 14:54:30.651875  9446 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:30.668991  9517 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:30.617421  9391 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:30.676247  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 337541us
W20260709 14:54:30.687490  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.678164  9450 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.702435  2604 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:30.704608  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.705618  2612 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:30.707718  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:30.702747  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 139893us
I20260709 14:54:30.697537  2686 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:30.728623  2612 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:30.734055  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:30.738124  9444 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:30.738636  9444 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:30.739023  9444 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:30.742462  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.743083  9389 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:54:30.745843  9020 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:30.746359  9020 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:30.749225  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.752616  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.754024  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.691092  9345 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.696004  9517 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:30.756176  9517 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.757510  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:30.758008  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.758293  2680 raft_consensus.cc:2376] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23. Re-sending same reply.
I20260709 14:54:30.758846  9391 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.759368  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.760309  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1786) took 1861 ms. Trace:
I20260709 14:54:30.759995  9450 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.760699  3545 rpcz_store.cc:276] 0709 14:54:28.898532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.899527 (+   995us) service_pool.cc:224] Handling call
0709 14:54:28.906564 (+  7037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.999811 (+ 93247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:30.760030 (+1760219us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b54002bb454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:30.760219 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82652416}
I20260709 14:54:30.761593  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1250) took 1047 ms. Trace:
I20260709 14:54:30.762208  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.763078  9027 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:30.763270  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.763509  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.763728  9027 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.764098  3545 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:30.764789  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:30.765151  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.765442  9356 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:30.765992  9356 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:30.726351  9008 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:30.756611  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56431.8us
I20260709 14:54:30.762032  2752 rpcz_store.cc:276] 0709 14:54:29.714550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:29.933449 (+218899us) service_pool.cc:224] Handling call
0709 14:54:29.933804 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.313278 (+379474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:30.761331 (+448053us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:30.761525 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866487552}
I20260709 14:54:30.767017  9356 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:30.764019  3546 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:30.768590  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.769521  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.770583  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.699321  9230 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:30.765619  3542 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:30.754570  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.777333  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.749246  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.752800  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.771473  2752 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:30.766407  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.779662  3545 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:30.760655  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.771981  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:30.759752  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1713) took 1817 ms. Trace:
I20260709 14:54:30.781481  2678 rpcz_store.cc:276] 0709 14:54:28.942577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:29.414346 (+471769us) service_pool.cc:224] Handling call
0709 14:54:29.966027 (+551681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.070229 (+104202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:30.759539 (+689310us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54001d15d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:30.759689 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267579136}
W20260709 14:54:30.783690  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:30.717154  9508 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.100s	user 0.001s	sys 0.001s
I20260709 14:54:30.701365  9445 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:30.792126  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.780869  2680 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:30.777655  2679 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:30.798694  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.797839  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:30.805789  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.806917  2755 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:30.808634  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.787385  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:30.796334  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:30.810431  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.813241  2609 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:30.815419  2678 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:30.816434  2759 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:30.817759  2678 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:54:30.819243  2755 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:30.823323  2678 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 14:54:30.791975  9402 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.668s	user 0.015s	sys 0.000s
W20260709 14:54:30.829365  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 57470.7us
I20260709 14:54:30.836911  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:30.766448  9027 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:30.844302  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:30.842134  2685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.030s	sys 0.015s
W20260709 14:54:30.847940  2685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.030s	sys 0.015s
W20260709 14:54:30.849275  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.832670  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:30.849294  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:30.834342  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.861522  2759 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:30.865293  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.866699  2603 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:30.867580  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.720304  9252 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 14:54:30.790817  9374 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:30.884903  2677 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 14:54:30.891338  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47214.2us
I20260709 14:54:30.757651  9020 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:30.894196  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.842041  9389 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:30.885207  9444 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:30.871874  9402 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:30.871210  9483 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:30.920403  9483 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:30.921051  9483 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:30.921936  9244 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.000s
I20260709 14:54:30.923306  9531 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:30.895310  9374 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.925499  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.927135  9445 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.886621  9520 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:30.900094  2755 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:30.910559  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.915767  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:30.815215  9386 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.929011  3547 raft_consensus.cc:2410] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 14:54:30.903563  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:30.843837  9519 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 14:54:30.955670  2609 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
W20260709 14:54:30.969530  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53902.2us
I20260709 14:54:30.926601  9374 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:31.012603  2604 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 14:54:30.960803  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.004s	sys 0.019s
W20260709 14:54:31.013640  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.005s	sys 0.019s
I20260709 14:54:31.008093  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:31.020486  2755 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:30.936156  9520 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:30.915092  9389 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:31.019575  9250 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:30.936532  9391 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 14:54:30.929553  9008 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
W20260709 14:54:31.038054  9516 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 14:54:31.078553  3537 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:54:31.091926  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:30.968223  9515 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:31.066318  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:31.103595  2759 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:31.025650  9420 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:31.028589  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:30.967000  9331 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:31.070258  9524 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 14:54:31.120675  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:31.132818  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.015s
I20260709 14:54:31.049892  9389 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:31.046620  9386 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:31.135484  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:31.099355  9250 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:31.129787  2759 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:31.044306  9444 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:31.173079  2759 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 14:54:31.032015  9374 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 14:54:31.143322  2760 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:31.025840  9402 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 14:54:31.087430  9468 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:31.170682  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:31.137666  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.004s	sys 0.015s
I20260709 14:54:31.104967  9520 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:31.108211  9420 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:31.175457  2759 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [18.10-19.11]
I20260709 14:54:31.022578  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:31.194747  2755 raft_consensus.cc:2376] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19. Re-sending same reply.
I20260709 14:54:31.196573  2755 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:31.139557  9060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.033s	sys 0.036s
W20260709 14:54:31.187111  3536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.012s	sys 0.015s
I20260709 14:54:31.189465  2754 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 14:54:31.073848  9538 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
W20260709 14:54:31.190644  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.013s	sys 0.010s
W20260709 14:54:31.201968  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.014s	sys 0.010s
W20260709 14:54:31.202625  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 516203us
W20260709 14:54:31.202461  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 476571us
I20260709 14:54:31.173483  9250 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:31.202045  3536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.012s	sys 0.015s
I20260709 14:54:31.203269  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.199218  2755 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:31.199754  2759 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 14:54:31.186558  9475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.009s	sys 0.021s
I20260709 14:54:31.211136  2755 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:31.230746  2754 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 14:54:31.203593  8708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.012s	sys 0.011s
I20260709 14:54:31.200448  9420 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:31.240146  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 442382us
W20260709 14:54:31.229628  9475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.009s	sys 0.022s
I20260709 14:54:31.238742  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:31.240456  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:31.235647  8708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.013s	sys 0.012s
W20260709 14:54:31.264020  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:31.264803  9535 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 14:54:31.243993  9544 consensus_queue.cc:1059] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
W20260709 14:54:31.322132  9539 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:54:31.239848  9331 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:31.359297  9337 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 14:54:31.362423  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:31.264529  9027 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.372354  9027 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:31.381040  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 112235us
I20260709 14:54:31.383363  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:31.381095  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 119771us
W20260709 14:54:31.389401  2611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.023s	sys 0.009s
W20260709 14:54:31.389969  2611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.023s	sys 0.009s
W20260709 14:54:31.390326  9429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.018s	sys 0.013s
W20260709 14:54:31.390848  9429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.018s	sys 0.013s
I20260709 14:54:31.370428  9538 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:54:31.402302  3538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.037s
W20260709 14:54:31.405117  3538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.009s	sys 0.037s
I20260709 14:54:31.406757  9483 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.409193  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1586) took 1159 ms. Trace:
W20260709 14:54:31.410372  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.031s	sys 0.044s
I20260709 14:54:31.412494  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:31.415251  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.017s	sys 0.017s
I20260709 14:54:31.402051  9544 consensus_queue.cc:1243] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 19.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 14:54:31.389591  9244 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 14:54:31.447495  9027 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:31.401326  9252 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 14:54:31.452448  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1251) took 1194 ms. Trace:
W20260709 14:54:31.488971  2678 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.006s	sys 0.037s
I20260709 14:54:31.468727  3538 rpcz_store.cc:276] 0709 14:54:30.249637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.249753 (+   116us) service_pool.cc:224] Handling call
0709 14:54:30.299487 (+ 49734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:30.335540 (+ 36053us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:30.350638 (+ 15098us) log.cc:844] Serialized 14 byte log entry
0709 14:54:30.483773 (+133135us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b18003a1d40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:31.405665 (+921892us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.406443 (+   778us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.408481 (+  2038us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:31.408547 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.408657 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:31.408724 (+    67us) raft_consensus.cc:1685] finished
0709 14:54:31.408878 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.409100 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26621,"prepare.queue_time_us":23612,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":1128,"raft.queue_time_us":98433,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":10782,"spinlock_wait_cycles":33580160,"thread_start_us":889340,"threads_started":2,"wal-append.queue_time_us":698483}
W20260709 14:54:31.489720  2678 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.008s	sys 0.038s
I20260709 14:54:31.489570  2684 rpcz_store.cc:276] 0709 14:54:30.258312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.297965 (+ 39653us) service_pool.cc:224] Handling call
0709 14:54:30.300737 (+  2772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.499577 (+198840us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:30.499909 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 14:54:30.961901 (+461992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:30.961972 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:30.962033 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.452039 (+490006us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001d06d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:31.452125 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.452363 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":150,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":79424,"spinlock_wait_cycles":587317504,"thread_start_us":460111,"threads_started":1,"wal-append.queue_time_us":574426}
W20260709 14:54:31.415552  9450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.017s	sys 0.016s
I20260709 14:54:31.415141  9404 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:31.457321  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.013s	sys 0.022s
W20260709 14:54:31.505873  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.014s	sys 0.022s
W20260709 14:54:31.457626  2685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.010s	sys 0.026s
W20260709 14:54:31.507282  2685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.010s	sys 0.026s
I20260709 14:54:31.414472  9538 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:31.465099  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.017s	sys 0.017s
W20260709 14:54:31.465639  2612 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.009s	sys 0.029s
W20260709 14:54:31.465793  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53384.5us
I20260709 14:54:31.466259  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:31.509263  2612 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.010s	sys 0.029s
I20260709 14:54:31.511102  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1721) took 1281 ms. Trace:
W20260709 14:54:31.466663  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.031s	sys 0.045s
W20260709 14:54:31.512423  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 796524us
I20260709 14:54:31.415377  9515 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 14:54:31.473789  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.012s	sys 0.028s
I20260709 14:54:31.456975  9319 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:31.519160  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.520201  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:31.417289  9483 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:31.474785  3540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.016s	sys 0.020s
W20260709 14:54:31.523093  3540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.016s	sys 0.020s
I20260709 14:54:31.521044  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:31.516834  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.012s	sys 0.028s
I20260709 14:54:31.517395  2753 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:31.511597  2685 rpcz_store.cc:276] 0709 14:54:30.229802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.229962 (+   160us) service_pool.cc:224] Handling call
0709 14:54:30.268399 (+ 38437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.268750 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 14:54:30.269096 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.507818 (+1238722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:31.510389 (+  2571us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:31.510477 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.510860 (+   383us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.511036 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19612,"prepare.queue_time_us":16153,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":183365,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":48466,"spinlock_wait_cycles":9045760,"thread_start_us":1238161,"threads_started":2,"wal-append.queue_time_us":763215}
W20260709 14:54:31.525985  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.012s	sys 0.028s
W20260709 14:54:31.526569  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.028s
W20260709 14:54:31.515237  9450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.018s	sys 0.016s
I20260709 14:54:31.523275  2684 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:31.517066  9404 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:31.538105  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:31.501152  3542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.033s	sys 0.029s
W20260709 14:54:31.547034  3542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.034s	sys 0.029s
W20260709 14:54:31.517525  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.014s	sys 0.037s
W20260709 14:54:31.550761  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.014s	sys 0.037s
W20260709 14:54:31.527683  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 617192us
I20260709 14:54:31.723790  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.724753  2605 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:31.609191  9538 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:31.725888  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:31.726169  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:31.540153  3411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.027s
I20260709 14:54:31.726953  2682 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:31.540282  2606 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.011s	sys 0.017s
W20260709 14:54:31.730104  2606 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.012s	sys 0.018s
I20260709 14:54:31.732225  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1229) took 3464 ms. Trace:
I20260709 14:54:31.732797  2606 rpcz_store.cc:276] 0709 14:54:28.267892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:28.975388 (+707496us) service_pool.cc:224] Handling call
0709 14:54:28.975967 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:28.976386 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:29.012078 (+ 35692us) log.cc:844] Serialized 14 byte log entry
0709 14:54:30.389042 (+1376964us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b18004fcae0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:31.730610 (+1341568us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.731316 (+   706us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.731508 (+   192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:31.731581 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:31.731664 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.731769 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.732134 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1361,"prepare.queue_time_us":144716,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":108,"raft.queue_time_us":134051,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":23892,"spinlock_wait_cycles":71876352,"thread_start_us":1340934,"threads_started":2,"wal-append.queue_time_us":1924770}
I20260709 14:54:31.557613  2576 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:31.562816  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.031s
I20260709 14:54:31.736205  9004 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:31.736807  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.000s	sys 0.032s
W20260709 14:54:31.737182  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 179693us
I20260709 14:54:31.737026  9004 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.531140  9476 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:31.739033  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:31.739319  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:31.740235  2683 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:31.740582  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:31.741273  3538 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:31.574397  2726 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.524s (SENT)
I20260709 14:54:31.743134  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:31.743424  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:31.743811  3543 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 14:54:31.744095  9391 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:54:31.744247  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 169947us
I20260709 14:54:31.519054  9319 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:31.745172  9319 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:31.569064  9515 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:31.748486  9515 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:31.749655  9515 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:31.586254  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:31.751248  2755 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:31.569651  9546 raft_consensus.cc:3055] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:31.752282  9546 raft_consensus.cc:740] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:31.752521  9520 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.000s
W20260709 14:54:31.607291  9542 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 14:54:31.752926  3538 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:31.590930  9549 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:31.754501  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:31.754876  9549 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:31.597078  9252 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:31.612264  9445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.006s	sys 0.024s
I20260709 14:54:31.756016  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:31.612915  9552 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:31.756305  9356 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:31.701244  9579 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:31.756011  9252 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.757828  3538 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:31.757496  9579 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:31.759333  9252 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:31.760309  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1933) took 1016 ms. Trace:
I20260709 14:54:31.760776  2608 rpcz_store.cc:276] 0709 14:54:30.744402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.745804 (+  1402us) service_pool.cc:224] Handling call
0709 14:54:30.746250 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:30.794431 (+ 48181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:30.794462 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:30.795202 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.506581 (+711379us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:31.506649 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.506770 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:31.760018 (+253248us) raft_consensus.cc:1685] finished
0709 14:54:31.760041 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.760237 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48372,"raft.queue_time_us":358587,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":16471,"thread_start_us":678825,"threads_started":1,"wal-append.queue_time_us":923362}
I20260709 14:54:31.744308  3547 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:54:31.703656  9545 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 14:54:31.764411  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1583) took 1515 ms. Trace:
I20260709 14:54:31.763862  3547 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
W20260709 14:54:31.706002  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:31.658699  9547 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:31.766892  9547 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:31.767452  9547 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:31.715508  9556 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:31.738493  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.773514  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:31.766876  3547 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 14:54:31.775154  9563 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:31.776060  9563 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:31.776537  9563 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:31.778455  9391 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.779233  9391 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:31.780597  2760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1889) took 1671 ms (client timeout 2089 ms). Trace:
W20260709 14:54:31.781109  2760 rpcz_store.cc:269] 0709 14:54:30.109479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.109636 (+   157us) service_pool.cc:224] Handling call
0709 14:54:31.780380 (+1670744us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:31.780527 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845589248}
I20260709 14:54:31.782109  9531 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:31.781908  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43533.3us
I20260709 14:54:31.783360  9020 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:31.783820  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.784150  9579 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:31.781839  9391 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:31.785363  9531 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:31.787410  3543 pending_rounds.cc:85] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 12
I20260709 14:54:31.787583  9388 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.788019  3543 pending_rounds.cc:107] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 20.13
W20260709 14:54:31.727061  3411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.001s	sys 0.028s
I20260709 14:54:31.788215  2576 leader_election.cc:400] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20.
I20260709 14:54:31.788865  2576 leader_election.cc:403] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:31.788348  9388 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:31.789208  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 956594us
I20260709 14:54:31.790345  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.790731  9517 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:31.791335  9531 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 20
W20260709 14:54:31.745244  2726 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.886s (SENT) [suppressed 39 similar messages]
I20260709 14:54:31.752727  2755 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:31.793776  2724 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:31.599877  2680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.012s	sys 0.016s
W20260709 14:54:31.794898  2680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.013s	sys 0.016s
I20260709 14:54:31.795639  9442 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:31.797216  9442 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:31.799072  9579 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:31.801322  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.802717  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1801) took 1043 ms. Trace:
I20260709 14:54:31.803246  3540 rpcz_store.cc:276] 0709 14:54:30.759407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.769050 (+  9643us) service_pool.cc:224] Handling call
0709 14:54:30.769495 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:30.769961 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:30.769993 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:30.770547 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.523725 (+753178us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:31.523772 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.523846 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:31.802465 (+278619us) raft_consensus.cc:1685] finished
0709 14:54:31.802487 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.802650 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30485,"raft.queue_time_us":1770,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":15736,"spinlock_wait_cycles":1920,"thread_start_us":752717,"threads_started":1,"wal-append.queue_time_us":1027176}
W20260709 14:54:31.805173  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 1888) took 1696 ms (client timeout 1693 ms). Trace:
I20260709 14:54:31.805565  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:31.805761  2757 rpcz_store.cc:269] 0709 14:54:30.109020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.119732 (+ 10712us) service_pool.cc:224] Handling call
0709 14:54:31.804925 (+1685193us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:31.805082 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202947968}
I20260709 14:54:31.555495  9404 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:31.808651  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:31.810248  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1441) took 1340 ms. Trace:
W20260709 14:54:31.756253  9445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.006s	sys 0.024s
I20260709 14:54:31.811024  9569 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.811573  3546 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:31.812731  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:31.813606  2751 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:31.813535  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:31.814235  9543 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 14:54:31.797109  9388 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:31.810745  3537 rpcz_store.cc:276] 0709 14:54:30.470355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.484558 (+ 14203us) service_pool.cc:224] Handling call
0709 14:54:30.534111 (+ 49553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.098662 (+564551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.809889 (+711227us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54001a0254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:31.810170 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251000704}
W20260709 14:54:31.815531  9420 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.003s
I20260709 14:54:31.816021  9420 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.817126  2608 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:31.813593  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1804) took 1053 ms. Trace:
I20260709 14:54:31.595487  9550 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 14:54:31.787312  2758 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 14:54:31.538808  9250 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:30.625504   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:54:31.820327  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:31.826371  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:31.816613  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1254) took 1093 ms. Trace:
I20260709 14:54:31.851181  2760 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:31.883296  2604 rpcz_store.cc:276] 0709 14:54:30.723664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.725114 (+  1450us) service_pool.cc:224] Handling call
0709 14:54:30.725532 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.028246 (+302714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.816249 (+788003us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540016e4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:31.816541 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659337472}
I20260709 14:54:31.818517  3546 rpcz_store.cc:276] 0709 14:54:30.760421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.770497 (+ 10076us) service_pool.cc:224] Handling call
0709 14:54:30.796116 (+ 25619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.812211 (+1016095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.813331 (+  1120us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54004c4ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:31.813523 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334876416}
I20260709 14:54:31.788439  3543 raft_consensus.cc:2889] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:31.885376  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:31.876554  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1810) took 1114 ms. Trace:
W20260709 14:54:31.814321  2751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1769) took 1254 ms (client timeout 1603 ms). Trace:
W20260709 14:54:31.832291  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:31.887568  2751 rpcz_store.cc:269] 0709 14:54:30.560056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.812366 (+1252310us) service_pool.cc:224] Handling call
0709 14:54:31.814254 (+  1888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:31.853302  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.852196  2686 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:31.836373  2605 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:31.888374  3542 rpcz_store.cc:276] 0709 14:54:30.762142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.793812 (+ 31670us) service_pool.cc:224] Handling call
0709 14:54:30.794215 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:30.842761 (+ 48546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:30.842815 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:30.891615 (+ 48800us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.548601 (+656986us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:31.548671 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.548788 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:31.876135 (+327347us) raft_consensus.cc:1685] finished
0709 14:54:31.876173 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.876452 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55002,"raft.queue_time_us":51433,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":22083,"thread_start_us":624398,"threads_started":1,"wal-append.queue_time_us":908254}
W20260709 14:54:31.858807  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45372.7us
I20260709 14:54:31.891942  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:31.879539  2606 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 14:54:31.904991  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1267) took 1216 ms. Trace:
I20260709 14:54:31.905898  2759 rpcz_store.cc:276] 0709 14:54:30.688878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.215379 (+526501us) service_pool.cc:224] Handling call
0709 14:54:31.215856 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.242489 (+ 26633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.242518 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.243128 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.703815 (+460687us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:31.703867 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.703970 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:31.904500 (+200530us) raft_consensus.cc:1685] finished
0709 14:54:31.904534 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.904890 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11556,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":5492,"thread_start_us":388659,"threads_started":1,"wal-append.queue_time_us":551913}
I20260709 14:54:31.915405  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:31.538079  9530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.006s	sys 0.023s
W20260709 14:54:31.925181  2651 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.603s (SENT)
I20260709 14:54:31.941010  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.918673  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.942051  2605 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:54:31.883193  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.048s	sys 0.056s
I20260709 14:54:31.943248  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.820899  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.935773  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:31.944183  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1759) took 1389 ms. Trace:
I20260709 14:54:31.944927  2758 rpcz_store.cc:276] 0709 14:54:30.554508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.554639 (+   131us) service_pool.cc:224] Handling call
0709 14:54:31.787025 (+1232386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.882274 (+ 95249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.943860 (+ 61586us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000e6850. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:31.944093 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2761381120}
W20260709 14:54:31.943557  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.048s	sys 0.056s
W20260709 14:54:31.946941  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 706538us
I20260709 14:54:31.818554  9420 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:31.955060  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.954437  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.959088  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:31.868633  3536 rpcz_store.cc:276] 0709 14:54:30.248718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.248861 (+   143us) service_pool.cc:224] Handling call
0709 14:54:30.249255 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.249514 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:30.427810 (+178296us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.228990 (+801180us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.763052 (+534062us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18000e6620. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf 00007fa16e660038
0709 14:54:31.763305 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.763725 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.763895 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:31.763938 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:31.763992 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.764170 (+   178us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.764337 (+   167us) inbound_call.cc:177] Queueing success response
0709 14:54:31.821366 (+ 57029us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b18000e6620. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:31.823000 (+  1634us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18000e6620. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
Metrics: {"mutex_wait_us":575012,"prepare.queue_time_us":582785,"prepare.run_cpu_time_us":211,"prepare.run_wall_time_us":219,"raft.queue_time_us":24278,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":453,"spinlock_wait_cycles":30357248,"thread_start_us":954169,"threads_started":2,"wal-append.queue_time_us":1172317}
W20260709 14:54:31.952365  2603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.026s	sys 0.061s
W20260709 14:54:31.960942  2603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.026s	sys 0.061s
I20260709 14:54:31.954401  2683 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:31.947330  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:31.952800  2611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.017s	sys 0.061s
W20260709 14:54:31.963232  2611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.017s	sys 0.061s
I20260709 14:54:31.962499  2759 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:31.833972  9554 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:31.844828  9469 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:31.889313  9531 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20.
I20260709 14:54:31.826761  9355 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:31.917938  9244 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:31.846279  9337 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 14:54:31.819823  9004 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:32.030416  9004 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:31.910780  9027 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.911373  9484 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.911038  9008 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:31.977624  9331 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:32.060623  9476 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:32.063227  2431 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 99) took 65020 ms (client timeout 15000 ms). Trace:
I20260709 14:54:31.753360  9546 consensus_queue.cc:260] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:32.067518  9546 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:32.064000  2431 rpcz_store.cc:269] 0709 14:53:27.042751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:27.043126 (+   375us) service_pool.cc:224] Handling call
0709 14:54:32.062890 (+65019764us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:54:18.940865 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:54:18.941607 (+   742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:54:18.941681 (+    74us) write_op.cc:435] Acquired schema lock
0709 14:54:18.941700 (+    19us) tablet.cc:662] Decoding operations
0709 14:54:18.986960 (+ 45260us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:54:19.007292 (+ 20332us) write_op.cc:641] Partition lock acquired for write op
0709 14:54:19.015901 (+  8609us) tablet.cc:685] Acquiring locks for 11 operations
0709 14:54:19.427041 (+411140us) tablet.cc:701] Row locks acquired
0709 14:54:19.427126 (+    85us) write_op.cc:260] PREPARE: finished
0709 14:54:19.427414 (+   288us) write_op.cc:270] Start()
0709 14:54:19.427657 (+   243us) write_op.cc:276] Timestamp: P: 1783608859427313 usec, L: 0
0709 14:54:19.427717 (+    60us) op_driver.cc:348] REPLICATION: starting
0709 14:54:19.428983 (+  1266us) log.cc:844] Serialized 4421 byte log entry
0709 14:54:20.806644 (+1377661us) op_driver.cc:464] REPLICATION: finished
0709 14:54:21.421507 (+614863us) write_op.cc:301] APPLY: starting
0709 14:54:21.421686 (+   179us) tablet.cc:1367] starting BulkCheckPresence
0709 14:54:21.449393 (+ 27707us) tablet.cc:1370] finished BulkCheckPresence
0709 14:54:21.449526 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:54:22.633863 (+1184337us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:54:22.703241 (+ 69378us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 14:54:22.703316 (+    75us) write_op.cc:312] APPLY: finished
0709 14:54:22.703853 (+   537us) log.cc:844] Serialized 104 byte log entry
0709 14:54:23.053287 (+349434us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:23.053491 (+   204us) write_op.cc:454] Released schema lock
0709 14:54:23.053577 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 14:54:23.083493 (+ 29916us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:23.084858 (+  1365us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":545398,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":600618,"apply.run_cpu_time_us":82296,"apply.run_wall_time_us":1643389,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50085,"num_ops":11,"prepare.queue_time_us":223391,"prepare.run_cpu_time_us":89712,"prepare.run_wall_time_us":1224032,"raft.queue_time_us":319694,"raft.run_cpu_time_us":132583,"raft.run_wall_time_us":404610,"replication_time_us":1378737,"thread_start_us":1675644,"threads_started":5,"wal-append.queue_time_us":1170043,"wal-append.run_cpu_time_us":42677,"wal-append.run_wall_time_us":2755238}]]}
I20260709 14:54:32.069114  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.811659  9569 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.992058  9383 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:31.766876  3538 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 14:54:31.830368  9515 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:31.912145  9555 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:31.988009  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:31.993943  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.026027  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1256) took 1295 ms. Trace:
I20260709 14:54:31.753324  9520 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:31.956293  9446 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 14:54:31.950791  9468 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:31.970723  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.085721  2612 rpcz_store.cc:276] 0709 14:54:30.730416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.732314 (+  1898us) service_pool.cc:224] Handling call
0709 14:54:30.732775 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.733072 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:31.511651 (+778579us) log.cc:844] Serialized 14 byte log entry
0709 14:54:32.025409 (+513758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:32.025509 (+   100us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:32.025602 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.025712 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:32.025934 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111868,"prepare.queue_time_us":174874,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":6734,"raft.queue_time_us":69022,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":34842,"thread_start_us":1179529,"threads_started":2}
W20260709 14:54:31.940860  9530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.006s	sys 0.024s
I20260709 14:54:32.143951  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:31.855506  9549 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.140s
I20260709 14:54:32.034174  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.027588  9355 raft_consensus.cc:697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:32.072788  9561 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:31.979216  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1250) took 1276 ms. Trace:
I20260709 14:54:32.158597  2611 rpcz_store.cc:276] 0709 14:54:30.703232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.716033 (+ 12801us) service_pool.cc:224] Handling call
0709 14:54:30.716573 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:30.716911 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:31.390574 (+673663us) log.cc:844] Serialized 14 byte log entry
0709 14:54:31.978557 (+587983us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:31.978653 (+    96us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:31.978733 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.978871 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:31.979121 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46798,"prepare.queue_time_us":125331,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":4103,"raft.queue_time_us":29593,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":9101,"thread_start_us":1177132,"threads_started":2}
I20260709 14:54:32.073364  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:31.975164  9554 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:31.981544  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:32.085685  2610 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
W20260709 14:54:32.092799  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 110000us
I20260709 14:54:32.082367  9331 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:31.993515  9494 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:31.978012  9469 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:32.143980  2608 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:32.183725  2610 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 14:54:32.154728  2606 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 14:54:31.992798  2427 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:32.148439  9020 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:32.167544  3547 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 14:54:32.169030  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:31.973444  9374 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 14:54:32.059031  9027 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:32.081483  9383 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:32.182096  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.031630  9571 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:32.142549  9468 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:32.202236  2604 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:32.075363  9404 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.163223  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.212525  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:32.075363  9484 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:32.380416  9585 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 14:54:32.382882  9484 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:32.219226  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 237812us
I20260709 14:54:32.386382  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:32.387523  9517 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 14:54:32.387727  9574 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 14:54:32.388656  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:32.389748  9517 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:32.390120  9582 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 14:54:32.390259  9517 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:32.391161  9517 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:32.390362  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:32.392925  9579 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:54:32.393888  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:32.395108  9531 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:32.395223  9388 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.395690  9531 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:32.396353  9531 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:32.396158  9388 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.399132  9554 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:32.401594  9554 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:32.402770  9420 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.403509  9420 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:32.406097  9420 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:32.410907  9568 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.145418  9555 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:32.264575  9468 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:32.414294  9555 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:32.414422  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:32.261701  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:32.192328  9469 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:32.224747  9244 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:32.369817  9570 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 14:54:32.371099  9027 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:32.256752  9494 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:32.375041  9355 consensus_queue.cc:237] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:32.442052  2685 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:32.363569  2612 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:32.264974  9571 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:32.447130  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.005s	sys 0.013s
W20260709 14:54:32.451175  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.005s	sys 0.013s
I20260709 14:54:32.419250  2604 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 14:54:32.216816  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:32.452545  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 235856us
W20260709 14:54:32.453833  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 241543us
I20260709 14:54:32.428069  2606 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 14:54:32.454790  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:32.280114  9547 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.003s
I20260709 14:54:32.237725  9004 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:32.175591  9561 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:32.469950  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.012s	sys 0.006s
W20260709 14:54:32.470422  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.012s	sys 0.006s
W20260709 14:54:32.440723  2680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.006s	sys 0.011s
I20260709 14:54:32.450942  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.474278  2612 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:54:32.475914  2680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.007s	sys 0.012s
I20260709 14:54:32.457738  2611 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 14:54:32.457454  2726 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.003s	sys 0.014s
W20260709 14:54:32.478294  2726 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.003s	sys 0.014s
W20260709 14:54:32.440531  9429 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.007s	sys 0.008s
I20260709 14:54:32.435225  9244 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:32.426195  9530 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:32.488294  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.507619  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:32.478822  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 733571us
I20260709 14:54:32.417032  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:32.465739  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 672008us
I20260709 14:54:32.435002  9556 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:32.504499  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:32.505275  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50583.9us
I20260709 14:54:32.453570  9337 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 14:54:32.522318  2726 leader_election.cc:336] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.647s (SENT)
I20260709 14:54:32.523443  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:32.496351  9442 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.001s
I20260709 14:54:32.520802  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:32.495302  9429 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.007s	sys 0.010s
W20260709 14:54:32.521447  2684 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.010s	sys 0.009s
W20260709 14:54:32.532620  2684 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.011s	sys 0.009s
I20260709 14:54:32.467403  9555 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:32.503105  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:32.524912  2726 leader_election.cc:336] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.666s (SENT)
W20260709 14:54:32.519735  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 102827us
I20260709 14:54:32.480530  9445 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:32.515594  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:32.525664  3545 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.004s	sys 0.015s
W20260709 14:54:32.510640  9391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.011s	sys 0.008s
I20260709 14:54:32.540273  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:32.577531  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.010s	sys 0.013s
I20260709 14:54:32.543514  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:32.578223  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.010s	sys 0.014s
I20260709 14:54:32.579895  2760 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 14:54:32.581863  2760 pending_rounds.cc:85] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 9
I20260709 14:54:32.582301  2760 pending_rounds.cc:107] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 14:54:32.550663  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:32.493886  9494 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:32.504772  9527 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:32.435532  9568 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.612444  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:32.613246  2758 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:32.515172  9244 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:32.582629  2760 raft_consensus.cc:2889] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:32.487167  9004 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:32.536907  9060 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.515265  9374 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
W20260709 14:54:32.601029  9391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.011s	sys 0.009s
I20260709 14:54:32.516878  9027 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:32.525560  9420 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.487555  9547 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:32.604398  9319 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.842s	user 0.013s	sys 0.000s
W20260709 14:54:32.643338  3540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.021s	sys 0.026s
W20260709 14:54:32.644044  3540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.021s	sys 0.026s
W20260709 14:54:32.634153  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.015s	sys 0.013s
I20260709 14:54:32.563522  9445 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:32.596427  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:32.655355  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.016s	sys 0.013s
I20260709 14:54:32.628891  9555 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:32.667182  2609 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:32.571328  3545 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.005s	sys 0.016s
W20260709 14:54:32.639734  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.007s	sys 0.021s
W20260709 14:54:32.613556  9356 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.857s	user 0.009s	sys 0.006s
I20260709 14:54:32.454667  9331 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:32.520984  9556 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:32.685408  9556 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:32.687598  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.007s	sys 0.021s
W20260709 14:54:32.693131  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 772501us
I20260709 14:54:32.678252  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.700943  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:32.647459  9515 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.003s
W20260709 14:54:32.642205  9563 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.865s	user 0.016s	sys 0.000s
I20260709 14:54:32.704385  9429 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:32.621627  9527 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.765251  2682 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
W20260709 14:54:32.705979  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.057s	sys 0.050s
I20260709 14:54:32.638803  9027 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:32.765925  2682 pending_rounds.cc:85] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 9
W20260709 14:54:32.651881  9375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.009s	sys 0.019s
I20260709 14:54:32.509624  9383 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:32.727492  2651 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.016s	sys 0.018s
I20260709 14:54:32.660439  9569 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:32.647459  9319 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:32.653999  9549 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:32.613322  9494 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.815310  9004 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:32.817167  9580 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
W20260709 14:54:32.710435  8885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.015s	sys 0.019s
I20260709 14:54:32.640049  9060 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:32.705610  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.021s	sys 0.033s
W20260709 14:54:32.827493  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.021s	sys 0.033s
W20260709 14:54:32.827878  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 914402us
I20260709 14:54:32.295783  2427 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 14 to 18, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:54:32.766160  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.057s	sys 0.050s
I20260709 14:54:32.766264  2682 pending_rounds.cc:107] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 14:54:32.859205  2682 raft_consensus.cc:2889] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:32.656706  9445 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:32.775291  2651 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.016s	sys 0.019s
W20260709 14:54:32.880386  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 932728us
W20260709 14:54:32.881202  2651 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.975s (SENT)
I20260709 14:54:32.881692  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:32.774649  9027 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:32.643180  9420 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:32.756138  9390 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.006s	sys 0.032s
I20260709 14:54:32.828794  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:32.908599  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79930us
I20260709 14:54:32.909746  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:32.793816  9375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.009s	sys 0.019s
I20260709 14:54:32.925659  2604 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:32.795665  9546 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.003s
W20260709 14:54:32.883685  2651 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.025s (SENT)
I20260709 14:54:32.930042  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:32.888618  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:32.767197  9538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.014s	sys 0.028s
I20260709 14:54:32.901882  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:32.829396  9584 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.387s	user 0.000s	sys 0.002s
W20260709 14:54:32.832212  8885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.015s	sys 0.019s
W20260709 14:54:32.912632  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:32.980747  2605 raft_consensus.cc:1275] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
W20260709 14:54:32.945393  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61755.9us
W20260709 14:54:32.907495  9390 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.007s	sys 0.033s
I20260709 14:54:32.776260  9563 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:32.981386  2605 pending_rounds.cc:85] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:54:32.990914  2605 pending_rounds.cc:107] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 14:54:32.991307  2605 raft_consensus.cc:2889] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:32.859315  9404 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:32.898912  9515 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:32.989117  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.999871  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.000885  2609 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:33.001964  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.003684  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:32.989398  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:33.004807  9520 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:32.978881  9538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.019s	sys 0.029s
I20260709 14:54:32.997871  9606 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:33.005613  9520 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:33.006345  9606 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:33.008102  9527 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:33.008891  9494 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:32.945081  9374 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:33.009958  9374 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:33.010517  9374 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:33.008950  9515 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:33.012518  9515 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:33.013226  9476 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 14:54:33.039346  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:33.040982  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51678.9us
I20260709 14:54:33.039343  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.826946  9549 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.006230  9520 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:32.974344  9569 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:33.065616  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.987808  9546 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:33.072739  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:32.992920  9571 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:32.993588  9597 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:32.974663  9442 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 14:54:33.087387  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.089093  2609 raft_consensus.cc:2410] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 14:54:33.016419  9563 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:33.069865  9577 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.362s	user 0.003s	sys 0.003s
I20260709 14:54:33.013954  9515 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:33.101940  9556 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:33.103118  9556 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:33.103502  9388 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:33.103544  9556 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:33.105695  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:33.069048  9250 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:32.994859  9356 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 14:54:33.068477  9569 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:33.087481  9597 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:33.122397  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:33.129755  2681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.029s	sys 0.036s
W20260709 14:54:33.109139  9207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.019s	sys 0.045s
I20260709 14:54:33.071343  9527 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 14:54:33.158655  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:33.140944  2753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.026s	sys 0.050s
I20260709 14:54:33.146883  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:33.147854  2681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.029s	sys 0.038s
I20260709 14:54:33.124269  9568 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:33.147444  2608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.008s	sys 0.061s
I20260709 14:54:33.148130  2611 raft_consensus.cc:2410] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 14:54:33.072041  8885 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 14:54:33.132131  9569 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:33.126413  9319 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:33.105930  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:33.150626  9207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.335s	user 0.020s	sys 0.046s
W20260709 14:54:33.177446  2753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.202s	user 0.027s	sys 0.051s
I20260709 14:54:33.224375  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1777) took 1645 ms. Trace:
I20260709 14:54:33.081656  9404 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 14:54:33.146899  9446 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:33.141114  9547 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:33.184170  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.036s	sys 0.033s
W20260709 14:54:33.239135  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.037s	sys 0.033s
W20260709 14:54:33.176365  2608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.008s	sys 0.061s
I20260709 14:54:33.241488  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1734) took 1365 ms. Trace:
I20260709 14:54:33.242221  2681 rpcz_store.cc:276] 0709 14:54:31.875962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.876183 (+   221us) service_pool.cc:224] Handling call
0709 14:54:31.876669 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.877046 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:32.425568 (+548522us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.240910 (+815342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:33.241003 (+    93us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:33.241074 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.241177 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.241392 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62685,"prepare.queue_time_us":278726,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":45,"thread_start_us":1227597,"threads_started":2}
I20260709 14:54:33.119897  9555 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 14:54:33.263306  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.000s	sys 0.021s
W20260709 14:54:33.263960  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.000s	sys 0.021s
W20260709 14:54:33.264567  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 749015us
I20260709 14:54:33.226642  2609 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:33.227231  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.012s
I20260709 14:54:33.228642  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.150789  9597 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:33.228616  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.012s
W20260709 14:54:33.272578  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.013s	sys 0.012s
W20260709 14:54:33.273360  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 748436us
W20260709 14:54:33.274117  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:33.274385  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1942) took 2069 ms. Trace:
W20260709 14:54:33.162279  9550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.113s	user 0.014s	sys 0.026s
I20260709 14:54:33.224879  2753 rpcz_store.cc:276] 0709 14:54:31.579013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.580204 (+  1191us) service_pool.cc:224] Handling call
0709 14:54:31.580703 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.581044 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:31.974077 (+393033us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.223834 (+1249757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:33.223932 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:33.224006 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.224106 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.224292 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":207445,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":1641275,"threads_started":2}
W20260709 14:54:33.236687  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.013s	sys 0.008s
W20260709 14:54:33.298770  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.014s	sys 0.009s
W20260709 14:54:33.299544  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 778814us
I20260709 14:54:33.274856  2603 rpcz_store.cc:276] 0709 14:54:31.204952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.205191 (+   239us) service_pool.cc:224] Handling call
0709 14:54:31.234086 (+ 28895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.245104 (+ 11018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.245154 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.262085 (+ 16931us) log.cc:844] Serialized 26 byte log entry
0709 14:54:31.961767 (+699682us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:31.961839 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.968426 (+  6587us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.274110 (+1305684us) raft_consensus.cc:1685] finished
0709 14:54:33.274135 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.274306 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129280,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":696220,"threads_started":1,"wal-append.queue_time_us":1299548}
I20260709 14:54:33.300782  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:33.237250  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.004s	sys 0.021s
W20260709 14:54:33.301545  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.005s	sys 0.021s
W20260709 14:54:33.269375  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.013s	sys 0.012s
W20260709 14:54:33.270347  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.032s	sys 0.091s
W20260709 14:54:33.302352  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.033s	sys 0.092s
W20260709 14:54:33.302349  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.014s	sys 0.010s
W20260709 14:54:33.304425  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.014s	sys 0.010s
I20260709 14:54:33.229308  9356 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:33.301911  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 798893us
W20260709 14:54:33.312354  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.012s	sys 0.012s
I20260709 14:54:33.312168  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:33.313025  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.013s	sys 0.012s
I20260709 14:54:33.229125  9391 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:33.319262  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:33.322113  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:33.234577  9446 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:33.298292  9550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.263s	user 0.014s	sys 0.026s
I20260709 14:54:33.336102  3547 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:33.335695  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:33.331403  2610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.001s	sys 0.025s
I20260709 14:54:33.339460  2611 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:33.352234  2683 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
W20260709 14:54:33.352351  2726 connection.cc:256] RPC call timeout handler was delayed by -1.5278384779999215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 14:54:33.353148  2726 connection.cc:256] RPC call timeout handler was delayed by -1.4918384779999769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 14:54:33.353499  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3658384779998869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
I20260709 14:54:33.353678  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.353958  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:33.354275  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3532384779999802s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 14:54:33.265003  9442 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:33.355052  2726 leader_election.cc:336] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.768s (SENT)
I20260709 14:54:33.355618  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:33.355197  9442 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.358595  9442 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:33.372169  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.001s	sys 0.025s
W20260709 14:54:33.325016  9252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.005s	sys 0.023s
I20260709 14:54:33.359884  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:33.346264  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:33.395298  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:33.396332  2603 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:33.402166  3545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.007s	sys 0.022s
I20260709 14:54:33.348474  9446 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:33.191507  9556 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:33.390188  9252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.005s	sys 0.023s
I20260709 14:54:33.340515  9250 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 14:54:33.328241  9356 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:33.331672  9568 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 14:54:33.335948  9388 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 14:54:33.376148  9331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.010s	sys 0.019s
I20260709 14:54:33.370169  9420 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:33.412533  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66373us
W20260709 14:54:33.614240  2684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.023s	sys 0.024s
W20260709 14:54:33.374110  9517 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.982s	user 0.016s	sys 0.000s
W20260709 14:54:33.652989  2685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.018s	sys 0.033s
W20260709 14:54:33.653527  2685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.018s	sys 0.033s
I20260709 14:54:33.500042  2612 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:33.496411  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:33.627274  2757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.009s	sys 0.016s
I20260709 14:54:33.374343  9550 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:54:33.656526  2757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.010s	sys 0.016s
W20260709 14:54:33.651496  2684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.023s	sys 0.028s
I20260709 14:54:33.482151  2609 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
W20260709 14:54:33.627126  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.018s	sys 0.007s
I20260709 14:54:33.658373  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1733) took 1951 ms. Trace:
W20260709 14:54:33.658888  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.019s	sys 0.007s
I20260709 14:54:33.662714  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:33.665807  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:33.497260  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1771) took 1608 ms. Trace:
I20260709 14:54:33.669404  2605 rpcz_store.cc:276] 0709 14:54:31.889173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.952846 (+ 63673us) service_pool.cc:224] Handling call
0709 14:54:31.953260 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:32.991994 (+1038734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.496772 (+504778us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54004808d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:33.497166 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353531776}
I20260709 14:54:33.658914  2684 rpcz_store.cc:276] 0709 14:54:31.706905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.707130 (+   225us) service_pool.cc:224] Handling call
0709 14:54:31.707566 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.707905 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:32.533210 (+825305us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.657947 (+1124737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:33.658009 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:33.658078 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.658169 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.658306 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":324589,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":3694,"thread_start_us":1949006,"threads_started":2}
I20260709 14:54:33.432271  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:33.673676  2611 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:33.676725  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.039s	user 0.014s	sys 0.039s
W20260709 14:54:33.414800  3545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.007s	sys 0.022s
W20260709 14:54:33.472756  9590 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 14:54:33.415853  9468 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.003s	user 0.013s	sys 0.003s
W20260709 14:54:33.554605  9608 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.556s	user 0.004s	sys 0.000s
W20260709 14:54:33.455394  9591 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 14:54:33.538483  9556 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:33.446152  9605 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 14:54:33.649055  2756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.009s	sys 0.018s
I20260709 14:54:33.712821  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:33.761233  2756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.011s	sys 0.019s
I20260709 14:54:33.762640  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1778) took 2182 ms. Trace:
W20260709 14:54:33.493021  9583 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.742s	user 0.005s	sys 0.005s
I20260709 14:54:33.401084  9404 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:33.447829  9607 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
W20260709 14:54:33.483561  9610 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 14:54:33.740331  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:33.677419  9331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.011s	sys 0.020s
I20260709 14:54:33.409232  9494 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:33.677373  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.015s	sys 0.039s
I20260709 14:54:33.490499  9549 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:33.773887  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:33.820259  9561 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:33.820574  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1732) took 2307 ms. Trace:
I20260709 14:54:33.820732  2758 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:33.677385  9550 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:33.823105  9550 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:33.824070  9004 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:54:33.827075  9593 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.112s	user 0.003s	sys 0.002s
I20260709 14:54:33.827850  9621 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:33.734206  9544 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:33.828649  9544 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.831413  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1946) took 2571 ms. Trace:
I20260709 14:54:33.821141  2679 rpcz_store.cc:276] 0709 14:54:31.515711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.518395 (+  2684us) service_pool.cc:224] Handling call
0709 14:54:31.532364 (+ 13969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:32.008458 (+476094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:32.008518 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:32.033961 (+ 25443us) log.cc:844] Serialized 26 byte log entry
0709 14:54:32.859275 (+825314us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:32.859345 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.859480 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.482576 (+623096us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:33.820184 (+337608us) raft_consensus.cc:1685] finished
0709 14:54:33.820208 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.820500 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7846,"raft.queue_time_us":182350,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":149,"spinlock_wait_cycles":18048896,"thread_start_us":1151149,"threads_started":2,"wal-append.queue_time_us":1087223}
I20260709 14:54:33.831954  2607 rpcz_store.cc:276] 0709 14:54:31.260650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.727855 (+467205us) service_pool.cc:224] Handling call
0709 14:54:31.728292 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.728728 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.728761 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.729364 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 14:54:32.470843 (+741479us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:32.470887 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.470973 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.826267 (+1355294us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:33.831012 (+  4745us) raft_consensus.cc:1685] finished
0709 14:54:33.831045 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.831332 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2356,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"spinlock_wait_cycles":722084224,"thread_start_us":740935,"threads_started":1,"wal-append.queue_time_us":898242}
I20260709 14:54:33.458237  9319 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:33.837723  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.022s	sys 0.089s
I20260709 14:54:33.489954  9520 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:33.556146  9442 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:33.383447  9207 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.322s
I20260709 14:54:33.817494  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:33.840495  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.022s	sys 0.090s
W20260709 14:54:33.843204  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 902310us
W20260709 14:54:33.704172  9531 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.301s	user 0.015s	sys 0.007s
I20260709 14:54:33.763310  2756 rpcz_store.cc:276] 0709 14:54:31.579564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.581640 (+  2076us) service_pool.cc:224] Handling call
0709 14:54:31.582082 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:31.582400 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:31.957033 (+374633us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.761822 (+1804789us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:33.761917 (+    95us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:33.762006 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.762114 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.762317 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206358,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":350082,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":105,"thread_start_us":2167680,"threads_started":3,"wal-append.queue_time_us":1135172}
I20260709 14:54:33.841181  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1719) took 3063 ms. Trace:
I20260709 14:54:33.852335  2678 rpcz_store.cc:276] 0709 14:54:30.778108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.824700 (+ 46592us) service_pool.cc:224] Handling call
0709 14:54:30.825202 (+   502us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:31.490282 (+665080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.490326 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:31.494789 (+  4463us) log.cc:844] Serialized 52 byte log entry
0709 14:54:31.912957 (+418168us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:31.913004 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:31.913083 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.492151 (+1579068us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:33.832164 (+340013us) raft_consensus.cc:1685] finished
0709 14:54:33.832190 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.841085 (+  8895us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":203715,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12293,"spinlock_wait_cycles":597912448,"thread_start_us":1080951,"threads_started":2,"wal-append.queue_time_us":1110707}
I20260709 14:54:33.841931  2752 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:33.817898  9445 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.003s
I20260709 14:54:33.863015  9531 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:33.864091  9445 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.866550  9445 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:33.842211  2603 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
W20260709 14:54:33.842501  2760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.188s	user 0.009s	sys 0.066s
I20260709 14:54:33.815816  2753 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:33.850008  9442 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:33.870221  2753 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 14:54:33.871776  2760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.010s	sys 0.066s
I20260709 14:54:33.871939  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1780) took 2195 ms. Trace:
I20260709 14:54:33.872432  9442 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:33.872357  2754 rpcz_store.cc:276] 0709 14:54:31.676784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.677030 (+   246us) service_pool.cc:224] Handling call
0709 14:54:31.677589 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.678193 (+   604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.678238 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.678982 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 14:54:32.578885 (+899903us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:32.578962 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.579088 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.871700 (+1292612us) raft_consensus.cc:1685] finished
0709 14:54:33.871725 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.871877 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1142,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":9343,"thread_start_us":878332,"threads_started":1,"wal-append.queue_time_us":1381996}
I20260709 14:54:33.851456  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:33.856684  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:33.849386  9561 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:33.876748  2756 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:33.382560  2427 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 7 to 17, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:33.882194  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:33.860767  9600 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.849s	user 0.000s	sys 0.004s
I20260709 14:54:33.500195  9599 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.341s
I20260709 14:54:33.742005  9337 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:33.701478  9420 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 14:54:33.698369  9517 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:33.863422  9619 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:33.867955  9531 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:34.001989  9614 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 14:54:33.846004  9319 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:33.886085  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:33.919961  2758 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:34.002024  9619 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:33.757228  9468 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:33.858664  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:33.987316  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1605) took 2245 ms. Trace:
I20260709 14:54:33.992791  9337 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:33.877168  2576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.047s	sys 0.070s
I20260709 14:54:33.912691  9445 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:34.021715  2576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.047s	sys 0.070s
I20260709 14:54:34.009521  3540 rpcz_store.cc:276] 0709 14:54:31.741931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.806797 (+ 64866us) service_pool.cc:224] Handling call
0709 14:54:31.957112 (+150315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.966586 (+  9474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.966637 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.977786 (+ 11149us) log.cc:844] Serialized 26 byte log entry
0709 14:54:32.644644 (+666858us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:32.662611 (+ 17967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.662923 (+   312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.501813 (+838890us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:33.921590 (+419777us) raft_consensus.cc:1685] finished
0709 14:54:33.921627 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:33.987209 (+ 65582us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10780,"raft.queue_time_us":9770,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":16883,"spinlock_wait_cycles":140562304,"thread_start_us":570689,"threads_started":1,"wal-append.queue_time_us":969378}
I20260709 14:54:34.012264  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1276) took 2284 ms. Trace:
I20260709 14:54:34.012673  2612 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:34.023859  9356 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:34.023883  2682 rpcz_store.cc:276] 0709 14:54:31.728303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:31.728777 (+   474us) service_pool.cc:224] Handling call
0709 14:54:31.729153 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:32.860168 (+1131015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.011841 (+1151673us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400269b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:34.012175 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381537536}
W20260709 14:54:34.027904  9244 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.389s	user 0.011s	sys 0.004s
I20260709 14:54:34.028825  9244 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.914567  9568 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
I20260709 14:54:34.008515  9622 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:34.017329  2752 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:34.107683  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:33.834990  9404 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:33.934237  9561 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:34.153267  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.451s	user 0.015s	sys 0.042s
I20260709 14:54:34.155802  9494 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 14:54:34.202307  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 94735.3us
I20260709 14:54:34.327512  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:34.328598  2752 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:34.331018  2756 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 14:54:34.199427  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1270) took 2028 ms. Trace:
I20260709 14:54:34.334113  2604 rpcz_store.cc:276] 0709 14:54:32.171252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.456577 (+285325us) service_pool.cc:224] Handling call
0709 14:54:32.456937 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:32.938782 (+481845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.199063 (+1260281us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54001d8b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:34.199339 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030299520}
W20260709 14:54:34.217254  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:34.031832  9599 consensus_queue.cc:1243] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 14:54:34.196480  9627 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:34.083920  9060 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.278371  2683 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 14:54:34.080875  9445 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:34.337836  9445 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:34.170536  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:34.211905  9027 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.281s	user 0.017s	sys 0.007s
I20260709 14:54:34.205749  9563 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:34.339021  9563 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:34.322409  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.620s	user 0.015s	sys 0.060s
I20260709 14:54:34.340293  9563 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.342185  9563 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:34.340909  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.63872e+06us
W20260709 14:54:34.239583  9469 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.792s	user 0.011s	sys 0.006s
I20260709 14:54:34.343804  9469 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.346364  9469 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:34.256712  9554 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.854s	user 0.014s	sys 0.008s
I20260709 14:54:34.279758  9429 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:34.348757  9429 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:34.297794  9550 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:34.237886  9561 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.153854  9622 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:34.303153  9207 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:34.358016  9207 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:34.359753  9207 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.361836  9207 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:34.278568  9390 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.201s	user 0.034s	sys 0.035s
I20260709 14:54:34.304462  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1261) took 2338 ms. Trace:
I20260709 14:54:34.367087  2610 rpcz_store.cc:276] 0709 14:54:31.966422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.190245 (+223823us) service_pool.cc:224] Handling call
0709 14:54:32.211746 (+ 21501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:32.267725 (+ 55979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:32.365107 (+ 97382us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:32.467266 (+102159us) log.cc:844] Serialized 26 byte log entry
0709 14:54:33.374499 (+907233us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:33.374567 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.374650 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:34.304153 (+929503us) raft_consensus.cc:1685] finished
0709 14:54:34.304201 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.304392 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31758,"raft.queue_time_us":43138,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":39504,"thread_start_us":761766,"threads_started":1,"wal-append.queue_time_us":1749416}
W20260709 14:54:34.253525  9355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.024s	sys 0.042s
W20260709 14:54:34.369994  9355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.024s	sys 0.042s
I20260709 14:54:34.249627  9547 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:34.371647  2751 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:34.312253  3545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1809) took 3550 ms. Trace:
I20260709 14:54:34.306882  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1730) took 3626 ms. Trace:
I20260709 14:54:34.373028  2680 rpcz_store.cc:276] 0709 14:54:30.680726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.817396 (+136670us) service_pool.cc:224] Handling call
0709 14:54:30.817854 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.795508 (+977654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:31.795546 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:31.796172 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 14:54:32.476787 (+680615us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:32.476855 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:32.497945 (+ 21090us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.859886 (+1361941us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:34.286622 (+426736us) raft_consensus.cc:1685] finished
0709 14:54:34.286657 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.306756 (+ 20099us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140612,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":44764,"spinlock_wait_cycles":112433920,"thread_start_us":1656868,"threads_started":2,"wal-append.queue_time_us":1101652}
I20260709 14:54:34.377466  9561 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.372872  3545 rpcz_store.cc:276] 0709 14:54:30.761837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:30.780468 (+ 18631us) service_pool.cc:224] Handling call
0709 14:54:30.813182 (+ 32714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:31.241669 (+428487us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:32.671960 (+1430291us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.748668 (+1076708us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.748911 (+   243us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:33.765616 (+ 16705us) log.cc:844] Serialized 26 byte log entry
0709 14:54:33.812442 (+ 46826us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:33.812506 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.216694 (+404188us) spinlock_profiling.cc:243] Waited 7.93 ms on lock 0x7b5400318154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:34.216781 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:34.311853 (+ 95072us) raft_consensus.cc:1685] finished
0709 14:54:34.311907 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.312115 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":596490,"prepare.queue_time_us":531592,"prepare.run_cpu_time_us":352,"prepare.run_wall_time_us":1681,"raft.queue_time_us":249477,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":9774,"spinlock_wait_cycles":18249088,"thread_start_us":2319986,"threads_started":3,"wal-append.queue_time_us":1458734}
W20260709 14:54:34.366499  9390 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.035s	sys 0.035s
I20260709 14:54:34.378465  9561 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.412284  2758 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 14:54:34.320600  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.030s	sys 0.024s
W20260709 14:54:34.413594  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.030s	sys 0.025s
I20260709 14:54:34.415467  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:34.150072  9620 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:34.150377  9549 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:34.343617  9027 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:34.155062  9544 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.018404  9616 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.267s
W20260709 14:54:34.430384  2723 connection.cc:256] RPC call timeout handler was delayed by -1.1020645469999408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 14:54:34.477711  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:34.428584  9569 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.188s	user 0.015s	sys 0.000s
I20260709 14:54:34.411229  9356 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.419430  9531 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:34.476240  9530 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.959s	user 0.010s	sys 0.012s
I20260709 14:54:34.430727  9547 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:34.391196  9561 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:34.512591  9624 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.473s	user 0.000s	sys 0.001s
I20260709 14:54:34.451164  9517 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:34.511456  9625 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.262s	user 0.002s	sys 0.000s
I20260709 14:54:34.275732  9619 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.535051  9599 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:34.514671  9632 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.433317  9549 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:34.567142  9620 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:34.573427  9530 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:34.536171  9569 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:34.587160  9527 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.348s	user 0.038s	sys 0.050s
I20260709 14:54:34.570267  9531 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:34.597545  9597 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.277s	user 0.011s	sys 0.006s
I20260709 14:54:34.598457  9619 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:34.598042  9527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.038s	sys 0.051s
W20260709 14:54:34.606070  9555 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.292s	user 0.031s	sys 0.057s
W20260709 14:54:34.609643  9556 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.001s
I20260709 14:54:34.610646  9527 consensus_queue.cc:1243] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 14:54:34.614876  9555 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.032s	sys 0.057s
I20260709 14:54:34.610030  9597 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:34.638383  9633 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 14:54:34.639998  9634 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260709 14:54:34.280514  2427 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 12 to 16, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 16 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:54:34.698683  2605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.020s	sys 0.007s
W20260709 14:54:34.699702  2605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.022s	sys 0.008s
W20260709 14:54:34.700111  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 284536us
W20260709 14:54:34.701954  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.030s	user 0.016s	sys 0.014s
W20260709 14:54:34.702595  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.016s	sys 0.014s
W20260709 14:54:34.321308  2608 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.070s	user 0.012s	sys 0.042s
W20260709 14:54:34.705520  2608 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.454s	user 0.013s	sys 0.042s
W20260709 14:54:34.708811  2681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.396s	user 0.013s	sys 0.051s
W20260709 14:54:34.709539  2681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.402s	user 0.014s	sys 0.051s
W20260709 14:54:34.710290  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 232530us
W20260709 14:54:34.711143  2686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.000s	sys 0.020s
W20260709 14:54:34.711755  2686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.000s	sys 0.020s
W20260709 14:54:34.712862  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.376s	user 0.009s	sys 0.071s
W20260709 14:54:34.713460  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.384s	user 0.010s	sys 0.071s
W20260709 14:54:34.714013  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.41027e+06us
W20260709 14:54:34.715129  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.348s	user 0.015s	sys 0.016s
W20260709 14:54:34.715728  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.015s	sys 0.016s
W20260709 14:54:34.718334  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.36323e+06us
I20260709 14:54:34.719393  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1260) took 2767 ms. Trace:
I20260709 14:54:34.720489  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1917) took 2165 ms. Trace:
W20260709 14:54:34.721505  2759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.390s	user 0.007s	sys 0.024s
W20260709 14:54:34.724134  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:34.724977  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.363s	user 0.030s	sys 0.056s
W20260709 14:54:34.726236  9020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.235s	user 0.035s	sys 0.038s
W20260709 14:54:34.727247  9252 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.017s	sys 0.020s
I20260709 14:54:34.728107  2758 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:34.452417  9355 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:34.736692  2760 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.005s	sys 0.004s
W20260709 14:54:34.743933  2759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.413s	user 0.008s	sys 0.024s
W20260709 14:54:34.749352  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.000s	sys 0.007s
W20260709 14:54:34.750079  2685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.003s	sys 0.002s
I20260709 14:54:34.752444  9549 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:34.745095  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.383s	user 0.030s	sys 0.056s
W20260709 14:54:34.756176  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:34.622078  9620 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.760850  9654 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:34.587209  9632 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.766731  9655 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:34.767673  9404 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:34.799494  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.139s	user 0.000s	sys 0.008s
I20260709 14:54:34.768639  2677 raft_consensus.cc:1240] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:34.768491  9631 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.771453  9653 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:34.773123  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:34.773655  9517 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.549913  9244 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:34.638415  9561 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.774398  9538 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.552016  9356 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:34.775658  9544 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 14:54:34.776065  9515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.027s	sys 0.078s
I20260709 14:54:34.776603  2680 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:54:34.776805  9546 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:34.778052  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:34.778398  9576 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:34.780182  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:34.780779  9597 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:34.605805  9547 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.845317  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:34.783566  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:34.784873  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:34.788058  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:34.849397  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:34.851079  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:34.851709  2679 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:34.852468  2684 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:34.480778  9319 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:34.759057  9252 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.017s	sys 0.021s
W20260709 14:54:34.757480  9020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.035s	sys 0.039s
W20260709 14:54:34.801900  2760 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.005s	sys 0.005s
W20260709 14:54:34.801826  2685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.003s	sys 0.002s
I20260709 14:54:34.856449  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:34.807163  9627 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.859057  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:34.859395  9627 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.860188  2679 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:34.861773  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:34.808557  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 1.14789e+06us
I20260709 14:54:34.862761  2679 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:34.817184  9632 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.818039  3545 raft_consensus.cc:1240] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:34.819425  9637 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.819690  9355 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:34.820708  9442 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:34.778661  9531 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.821710  2756 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:34.871646  9442 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.875636  9637 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.821911  9643 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.879601  9643 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:34.881279  9627 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.823915  9027 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:34.827117  9616 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:34.644021  9569 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.733039  2755 rpcz_store.cc:276] 0709 14:54:32.555047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.555232 (+   185us) service_pool.cc:224] Handling call
0709 14:54:32.555719 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:32.556221 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:32.556264 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:32.585650 (+ 29386us) log.cc:844] Serialized 26 byte log entry
0709 14:54:33.819197 (+1233547us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:33.819246 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:33.819330 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:33.822874 (+  3544us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:54:34.719350 (+896476us) raft_consensus.cc:1685] finished
0709 14:54:34.719411 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.720113 (+   702us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":326314,"raft.run_cpu_time_us":5400,"raft.run_wall_time_us":924977,"spinlock_wait_cycles":48123648,"thread_start_us":1181399,"threads_started":1,"wal-append.queue_time_us":1677666}
I20260709 14:54:34.884366  9616 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:34.790750  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:34.822536  9653 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:34.791901  9337 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.870704  9355 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.896397  9643 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:34.885291  9442 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:34.902034  9653 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:34.897732  9616 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:34.842406  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 106662us
W20260709 14:54:34.906997  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120311us
W20260709 14:54:34.910887  2757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.976s	user 0.006s	sys 0.010s
I20260709 14:54:34.725461  2427 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:34.846887  2680 raft_consensus.cc:2376] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19. Re-sending same reply.
W20260709 14:54:34.932581  2757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.007s	sys 0.012s
W20260709 14:54:34.913990  9515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.565s	user 0.027s	sys 0.078s
W20260709 14:54:34.910584  2726 connection.cc:256] RPC call timeout handler was delayed by -1.3334384779999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
I20260709 14:54:34.907465  9544 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:34.940483  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 170869us
I20260709 14:54:34.940907  9060 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.947446  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:34.947718  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:34.948251  9538 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:34.948827  3537 raft_consensus.cc:2410] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 14:54:34.951189  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1956) took 1643 ms. Trace:
W20260709 14:54:34.952185  9611 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 14:54:34.954278  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:34.955042  9390 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.955367  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:34.956399  9652 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:34.956959  3546 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:34.940936  2682 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:54:34.963048  2427 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 11 to 18, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:34.965806   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:54:34.948753  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.012s	sys 0.006s
I20260709 14:54:34.942073  9060 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:34.979085  2726 connection.cc:256] RPC call timeout handler was delayed by -1.2531912729999704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 14:54:34.981671  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.657s	user 0.012s	sys 0.007s
I20260709 14:54:34.977777  2678 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:34.732916  2608 rpcz_store.cc:276] 0709 14:54:31.952286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.184065 (+231779us) service_pool.cc:224] Handling call
0709 14:54:32.194620 (+ 10555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:32.195758 (+  1138us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:32.231173 (+ 35415us) log.cc:844] Serialized 14 byte log entry
0709 14:54:34.706493 (+2475320us) log.cc:844] Serialized 14 byte log entry
0709 14:54:34.707131 (+   638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:34.707275 (+   144us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:34.707502 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.707703 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.719163 (+ 11460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6908,"prepare.queue_time_us":1423208,"prepare.run_cpu_time_us":11820,"prepare.run_wall_time_us":86196,"raft.queue_time_us":13749,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":820,"thread_start_us":2465690,"threads_started":2,"wal-append.queue_time_us":1502656}
I20260709 14:54:34.937059  9544 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:34.989571  2726 connection.cc:256] RPC call timeout handler was delayed by -1.1784252609999157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 14:54:34.995286  9530 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:34.997304  3543 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:34.998034  3542 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 14:54:34.998577  3542 pending_rounds.cc:85] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 11
I20260709 14:54:34.951952  2605 rpcz_store.cc:276] 0709 14:54:33.308248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:33.671038 (+362790us) service_pool.cc:224] Handling call
0709 14:54:33.671462 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:33.671745 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:34.700582 (+1028837us) log.cc:844] Serialized 14 byte log entry
0709 14:54:34.950207 (+249625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:34.950315 (+   108us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:34.950506 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.950689 (+   183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:34.951049 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109805,"prepare.queue_time_us":966001,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":116196,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":286,"thread_start_us":1138320,"threads_started":2}
I20260709 14:54:34.999784  9383 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.005016  9586 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:35.007318  9468 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.011683  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1918) took 2454 ms. Trace:
I20260709 14:54:35.000072  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.012518  2760 rpcz_store.cc:276] 0709 14:54:32.557660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.580987 (+ 23327us) service_pool.cc:224] Handling call
0709 14:54:32.581551 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:32.615962 (+ 34411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:32.615998 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:32.619490 (+  3492us) log.cc:844] Serialized 26 byte log entry
0709 14:54:33.872381 (+1252891us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:34.856703 (+984322us) log.cc:844] Serialized 14 byte log entry
0709 14:54:35.011057 (+154354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:35.011218 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:35.011304 (+    86us) raft_consensus.cc:1685] finished
0709 14:54:35.011316 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.011591 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121305,"prepare.queue_time_us":1036050,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":14567,"raft.queue_time_us":127841,"spinlock_wait_cycles":54509312,"thread_start_us":2201753,"threads_started":2,"wal-append.queue_time_us":1926002}
W20260709 14:54:35.003376  2726 connection.cc:256] RPC call timeout handler was delayed by -1.0704745780000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3024s for a response
I20260709 14:54:35.016198  9599 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:34.996721  9530 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.021878  9494 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:34.966641  2427 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 8 to 18. New cstate: current_term: 18 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:54:35.023893  9538 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.024794  9609 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:35.027048  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.027806  9587 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:35.028546  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.029747  9576 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.031747  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.033617  9008 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:34.959872  9652 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.048777  9008 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:34.963086  9617 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:35.050566  9008 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.052122  9060 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.053402  9599 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:35.055151  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.75434e+06us
W20260709 14:54:35.028620  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90257.8us
I20260709 14:54:35.044842  9637 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:35.035358  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:34.990983  9631 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.045555  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1735) took 2520 ms. Trace:
I20260709 14:54:34.956023  9390 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.047876  9375 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:35.035367  9621 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 14:54:35.050804  9617 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:35.071342  9446 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.600s	user 0.012s	sys 0.016s
I20260709 14:54:35.071656  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:35.072057  9446 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.072898  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1736) took 1768 ms. Trace:
I20260709 14:54:35.073381  2681 rpcz_store.cc:276] 0709 14:54:33.304881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:33.306265 (+  1384us) service_pool.cc:224] Handling call
0709 14:54:33.306692 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:34.710430 (+1403738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:34.710483 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:34.717549 (+  7066us) log.cc:844] Serialized 26 byte log entry
0709 14:54:34.835352 (+117803us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:34.835428 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.844038 (+  8610us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:35.072583 (+228545us) raft_consensus.cc:1685] finished
0709 14:54:35.072610 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.072807 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":535759,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":62157,"thread_start_us":1495753,"threads_started":2,"wal-append.queue_time_us":297998}
I20260709 14:54:35.075574  9446 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.077081  9404 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:35.078042  9404 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:35.002389  3542 pending_rounds.cc:107] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 21.12
W20260709 14:54:35.045982  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60218.9us
I20260709 14:54:35.052754  9008 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:35.065714  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.056159  9587 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:35.078766  9404 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:35.080322  2684 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:35.080628  3542 raft_consensus.cc:2889] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:35.082765  3543 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:35.084115  9587 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.093111  9587 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:35.084385  2610 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:35.067862  9494 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.075800  2606 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 14:54:35.099119  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.100716  2682 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:35.100807  9331 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.101868  9331 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.040046  2756 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:35.055328  9576 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.110101  9576 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:34.980334  9517 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.113850  9517 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.115525  9383 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:35.116251  9383 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:35.122684  3545 raft_consensus.cc:1275] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 14:54:35.078505  9429 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.729s	user 0.016s	sys 0.005s
I20260709 14:54:35.085520  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.132761  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.102069  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:35.138175  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60145.4us
I20260709 14:54:35.117864  9383 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.136546  2682 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:35.145884  2681 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:35.152343  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.158006  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.159096  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.161247  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.161440  9446 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:35.169246  9576 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:35.170797  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.172031  3547 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:35.173285  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.174038  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:35.132398  9494 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.176466  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:35.176808  9390 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.121865  9631 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.158046  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:35.063702  9621 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:35.180588  9375 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:35.181360  9375 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:35.181201  9621 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.182914  9538 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:35.169277  9628 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:35.183826  9538 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.184031  9628 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.214226  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.215884  3546 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 14:54:35.214018  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.085235  9530 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.231492  2680 raft_consensus.cc:2410] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 14:54:35.059083  2685 rpcz_store.cc:276] 0709 14:54:32.519980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.520390 (+   410us) service_pool.cc:224] Handling call
0709 14:54:32.565299 (+ 44909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:33.654067 (+1088768us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:33.756288 (+102221us) log.cc:844] Serialized 14 byte log entry
0709 14:54:34.998624 (+1242336us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:34.999517 (+   893us) log.cc:844] Serialized 26 byte log entry
0709 14:54:34.999756 (+   239us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:34.999797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:34.999940 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:35.036962 (+ 37022us) raft_consensus.cc:1685] finished
0709 14:54:35.036989 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.040005 (+  3016us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188191,"prepare.queue_time_us":102172,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":15100,"raft.queue_time_us":286668,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":31441,"thread_start_us":2046512,"threads_started":2,"wal-append.queue_time_us":985112}
I20260709 14:54:35.230360  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.186164  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1913) took 2640 ms. Trace:
I20260709 14:54:35.234894  2681 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:35.182762  9621 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 14 FOLLOWER]: Advancing to term 15
I20260709 14:54:35.236222  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.176533  9494 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.255445  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.257511  3537 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:35.261381  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:35.261978  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 100908us
I20260709 14:54:35.262991  9556 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:35.263173  2427 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 9 to 12, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:35.264356  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:35.264578  9494 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.265265  2755 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:35.267482  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.268286  2682 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:35.272285  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.272715  9020 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:35.273463  9020 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.243474  9375 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.274786  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.283311  3547 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 14. Current term is 15. Ops: []
W20260709 14:54:35.279659  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94791.3us
W20260709 14:54:35.285074  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:35.293782  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:35.289719  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.304075  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.308517  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.309456  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.157063  9429 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.235674  2759 rpcz_store.cc:276] 0709 14:54:32.546200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.546477 (+   277us) service_pool.cc:224] Handling call
0709 14:54:32.546983 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:32.547354 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:32.609194 (+ 61840us) log.cc:844] Serialized 14 byte log entry
0709 14:54:34.936512 (+2327318us) log.cc:844] Serialized 14 byte log entry
0709 14:54:35.044028 (+107516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:35.056773 (+ 12745us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:35.133632 (+ 76859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:35.146575 (+ 12943us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.186084 (+ 39509us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80971,"prepare.queue_time_us":292915,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":97,"raft.queue_time_us":586999,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":287,"thread_start_us":2231082,"threads_started":3,"wal-append.queue_time_us":1270902}
I20260709 14:54:35.251526  9652 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:35.314288  2685 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 14:54:35.270596  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.320597  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.322959  9494 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.281984  9538 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.323645  9429 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:35.323884  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.325366  3541 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:35.326290  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:35.333698  2754 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
W20260709 14:54:35.275889  9445 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.007s
I20260709 14:54:35.327586  2427 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 3 to 15, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:35.331856  9661 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.332762  3546 raft_consensus.cc:2393] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:35.327054  2680 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 16.
I20260709 14:54:35.278263  9020 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.337450  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.275473  9655 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:35.280473  9517 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:35.372280  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:35.382541  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65327us
I20260709 14:54:35.378963  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.357404  9661 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.385221  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.369635  9631 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 14:54:35.396021  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.400103  2680 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
W20260709 14:54:35.400374  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 129943us
I20260709 14:54:35.401646  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.401937  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:35.403049  9563 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.908s	user 0.012s	sys 0.017s
I20260709 14:54:35.403879  9636 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.404187  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.404727  9636 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:35.406092  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 131554us
I20260709 14:54:35.365067  9445 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.407400  9641 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:35.407958  9641 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:35.408320  9641 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:35.409610  9445 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.410892  3538 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 14:54:35.389228  9494 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.411864  9563 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:35.412434  9563 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:35.412809  9563 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:35.417057  9561 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:35.416385  2427 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 12 to 19, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:35.411962  9494 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:35.410557  9374 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 2.337s	user 0.008s	sys 0.020s
I20260709 14:54:35.417831  9561 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:35.357728  9538 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 14:54:35.436300  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.394047  9661 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:35.446728  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77456.8us
I20260709 14:54:35.445271  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.434381  9494 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.460539  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.461755  2680 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:35.468561  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.474568  2682 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:35.449085  9383 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.479355  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:35.480243  9207 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.118s	user 0.018s	sys 0.005s
W20260709 14:54:35.483407  9619 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.872s	user 0.014s	sys 0.008s
I20260709 14:54:35.486672  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.487062  9337 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.487771  9337 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.488729  9530 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:35.489168  9530 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.489547  9530 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:35.448343  9374 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.491215  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.492077  9636 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.493054  9374 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.498091  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.498982  2685 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:35.500309  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.501363  2685 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 14:54:35.501801  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.488106  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.497432  2427 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 14 to 17, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:35.505707  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.492111  2679 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:35.443878  9404 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.517236  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.523314  9538 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:35.526805  9619 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.527753  9619 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.503903  9337 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.531818  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1268) took 3433 ms. Trace:
I20260709 14:54:35.532388  2606 rpcz_store.cc:276] 0709 14:54:32.098462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.455569 (+357107us) service_pool.cc:224] Handling call
0709 14:54:32.455912 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:35.099431 (+2643519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:35.531448 (+432017us) spinlock_profiling.cc:243] Waited 2.62 s on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:35.531743 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6023303552}
W20260709 14:54:35.534341  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:35.536928  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:35.539155  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:35.539364  9207 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:35.539855  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 86088.3us
I20260709 14:54:35.541458  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.542559  9207 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:35.544203  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1737) took 2239 ms. Trace:
I20260709 14:54:35.544689  2686 rpcz_store.cc:276] 0709 14:54:33.305226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:33.306018 (+   792us) service_pool.cc:224] Handling call
0709 14:54:33.310208 (+  4190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:34.712595 (+1402387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:34.712688 (+    93us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:34.712785 (+    97us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:34.712875 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:35.543904 (+831029us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b540024f9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:35.543961 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.544138 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":739329,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":32489,"spinlock_wait_cycles":978010880,"thread_start_us":976203,"threads_started":1}
I20260709 14:54:35.547286  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.544725  9599 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.501523  9561 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.553224  9404 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:35.564409  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76449.1us
I20260709 14:54:35.513674  9587 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.581317  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.588511  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.597613  2685 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 14:54:35.582073  9544 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.005s
I20260709 14:54:35.566641  9619 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.603587  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.609817  2678 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:35.597348  9599 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:35.597316  9538 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:54:35.627627  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61518.3us
W20260709 14:54:35.631142  3411 connection.cc:256] RPC call timeout handler was delayed by -1.1567981099999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 14:54:35.632419  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51247.2us
I20260709 14:54:35.633852  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.635140  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.636195  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.637111  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.639073  9576 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:35.639719  9549 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:35.644948  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.604665  9587 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:35.655081  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 107924us
I20260709 14:54:35.658087  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:35.658394  9576 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.659236  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.670095  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.668004  2604 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:35.612177  9544 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.677223  2679 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:35.685678  2609 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:35.673218  9576 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:35.683314  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.692473  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.692660  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.679639  9538 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:35.703310  2605 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:35.701187  2427 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 13 to 17, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:35.704803  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.707630  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.709002  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.709509  9027 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:35.710247  9027 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:35.710779  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.719867  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.710916  9599 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:35.718959  9619 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.727341  9619 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:35.727749  9619 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.729104  9628 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.731532  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.732417  2608 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:35.747385  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.748926  2605 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 14:54:35.725883  9446 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 14:54:35.706300  2427 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 8 to 17, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:35.751550  9554 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:35.752072  9554 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:35.753372  9554 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.726718  9621 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.755581  9621 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:35.756068  9621 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:35.756304  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45690.4us
I20260709 14:54:35.757403  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.759121  9207 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:35.759915  3411 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.796s (SENT)
I20260709 14:54:35.760203  9207 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:35.760529  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:35.760710  9207 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:35.761204  9207 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:35.763088  9429 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:35.711792  9027 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.763654  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1740) took 2269 ms. Trace:
I20260709 14:54:35.764164  2683 rpcz_store.cc:276] 0709 14:54:33.493948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:33.496083 (+  2135us) service_pool.cc:224] Handling call
0709 14:54:35.763164 (+2267081us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:35.763426 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388704512}
I20260709 14:54:35.765740  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.758658  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.748490  9530 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.714440  9587 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.778153  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.781201  2427 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 8 to 15, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:35.786634  2603 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:35.787873  9530 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:35.789391  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 69138us
I20260709 14:54:35.707721  9661 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.802683  2679 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:54:35.800904  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.807441  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.810557  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.816864  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.817981  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.820458  9576 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.849327  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.850193  3536 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:35.810792  3545 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 14:54:35.852213  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:35.853910  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.855535  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:35.857751  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.822680  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.823041  9628 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:35.824288  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:35.860189  3540 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:35.826983  2603 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:35.861245  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.861364  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.862066  3540 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:35.862995  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.864440  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:35.827229  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68667.4us
I20260709 14:54:35.828112  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.866107  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.866999  9547 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:35.867153  9628 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:35.867496  9547 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:35.867606  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.868948  9355 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:35.868629  9547 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:35.869385  9355 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:35.869777  9355 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:35.871413  8885 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:35.871920  8885 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:35.873028  9544 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.872809  8885 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.873725  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.873960  8885 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:35.875216  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:35.833727  9629 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.318s	user 0.008s	sys 0.013s
I20260709 14:54:35.857877  2427 catalog_manager.cc:5697] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 4 to 16, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:54:35.876271  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44329.5us
I20260709 14:54:35.877728  9250 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:35.878914  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1916) took 3326 ms. Trace:
I20260709 14:54:35.879585  9250 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:35.724753  9494 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.840116  2608 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:35.843992  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.845067  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.847713  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.848493  9404 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:35.850698  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 198422us
I20260709 14:54:35.875407  3547 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:35.877990  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:35.836597  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.880573  9609 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:35.879302  2757 rpcz_store.cc:276] 0709 14:54:32.552512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.552700 (+   188us) service_pool.cc:224] Handling call
0709 14:54:32.553259 (+   559us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:32.553860 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:32.636746 (+ 82886us) log.cc:844] Serialized 14 byte log entry
0709 14:54:33.660331 (+1023585us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:33.755867 (+ 95536us) log.cc:844] Serialized 52 byte log entry
0709 14:54:33.776495 (+ 20628us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:35.004522 (+1228027us) log.cc:844] Serialized 14 byte log entry
0709 14:54:35.048342 (+ 43820us) log.cc:844] Serialized 14 byte log entry
0709 14:54:35.048694 (+   352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:35.048824 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:35.832631 (+783807us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:35.878587 (+ 45956us) raft_consensus.cc:1685] finished
0709 14:54:35.878612 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:35.878813 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90057,"prepare.queue_time_us":1055210,"prepare.run_cpu_time_us":318,"prepare.run_wall_time_us":18815,"raft.queue_time_us":520440,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":48257,"spinlock_wait_cycles":45546752,"thread_start_us":2159025,"threads_started":2,"wal-append.queue_time_us":1667018}
I20260709 14:54:35.881958  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.882414  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.858661  2686 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:35.908144  2605 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:35.883420  9554 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:35.883528  9563 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.884934  9641 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.796698  9337 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 14:54:35.916036  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 111121us
I20260709 14:54:35.887259  9538 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:35.888252  9576 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.854633  3545 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:35.860241  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.889976  9617 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:35.890309  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:35.890333  9661 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.829010  9619 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.835798  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:35.891654  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 371720us
I20260709 14:54:35.892670  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.896641  2683 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:35.901640  2604 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:35.908306  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:35.877627  2427 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 6 to 10, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 10 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:35.915198  3542 raft_consensus.cc:2393] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:35.923512  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:35.924449  9356 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.924327  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.925347  2603 raft_consensus.cc:2376] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20. Re-sending same reply.
I20260709 14:54:35.926710  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.910564  9609 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:35.869804  9547 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.932130  2576 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.934626  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:35.934165  9374 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:35.946687  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.950500  9004 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:35.952005  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:35.952625  9619 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.952847  9599 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.954102  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:35.959123  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50935us
W20260709 14:54:35.959398  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 77588us
I20260709 14:54:35.960048  9617 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:35.960783  9331 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:35.961387  9622 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:35.963168  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 70565.7us
I20260709 14:54:35.989606  2686 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:35.964047  9576 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:35.964722  9550 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 17 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:35.968695  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.992571  2679 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:35.919836  9404 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:35.971222  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.972012  2605 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:35.972409  2610 raft_consensus.cc:2393] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:35.973237  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:35.975461  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:35.975718  2576 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.890811  9207 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:35.993198  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.999081  2683 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:35.963212  9494 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:35.982003  9628 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:36.003537  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:35.983387  9599 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:35.983235  9617 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:35.976961  9250 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 14:54:36.025620  2684 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:36.019554  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.028384  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:35.995347  9576 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:36.027817  9661 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:36.035511  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:36.021761  9599 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.047297  9446 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:36.049932  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.051244  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:36.025962  9617 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:35.988957  9004 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:36.025688  9619 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:36.056679  2603 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
W20260709 14:54:36.057302  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53905.8us
I20260709 14:54:36.052386  2680 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:36.057233  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.036324  9576 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.061887  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:36.064427  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:36.067066  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.073329  9544 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:36.080619  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1729) took 1031 ms. Trace:
I20260709 14:54:36.074908  9546 consensus_queue.cc:1048] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:36.092571  3542 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:36.054862  9446 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.090562  2576 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.093889  3536 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:36.097780  2686 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:36.099627  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:36.100102  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.101200  2684 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.099371  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:36.079589  9616 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:36.068560  9331 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:36.103693  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.106009  9641 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:36.092401  9563 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:36.096402  9446 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:36.115410  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.122959  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.119115  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.119541  2683 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:36.095222  9337 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:36.120568  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:36.127082  3546 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:36.127995  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:36.106078  9319 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:36.133881  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.136761  2756 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:36.134951  2608 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:36.137430  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:36.139786  9429 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:36.140345  9429 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.140690  9429 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:36.134610  2757 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:36.143389  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1726) took 1095 ms. Trace:
I20260709 14:54:36.143898  2678 rpcz_store.cc:276] 0709 14:54:35.048018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.057103 (+  9085us) service_pool.cc:224] Handling call
0709 14:54:36.143174 (+1086071us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b540024f4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:36.143288 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702279040}
I20260709 14:54:36.144304  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1914) took 3595 ms. Trace:
I20260709 14:54:36.144991  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.145486  2610 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:36.146097  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1749) took 1041 ms. Trace:
I20260709 14:54:36.146191  9374 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:36.147620  2610 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 14:54:36.147713  9374 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.150229  2603 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:36.146658  2678 rpcz_store.cc:276] 0709 14:54:35.104474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.144929 (+1040455us) service_pool.cc:224] Handling call
0709 14:54:36.145291 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.145616 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:36.145642 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:36.145709 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:36.145798 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.145863 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.146008 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.112703  9616 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:36.154548  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:36.154232  2680 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:36.125355  9445 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:36.158965  2681 raft_consensus.cc:2410] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 14:54:36.144857  2751 rpcz_store.cc:276] 0709 14:54:32.548652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:32.548887 (+   235us) service_pool.cc:224] Handling call
0709 14:54:32.612008 (+ 63121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:34.372454 (+1760446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.143463 (+1771009us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54003f1454. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:36.143785 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4046145408}
I20260709 14:54:36.166954  2756 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:36.161748  2683 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:36.169900  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.171716  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1745) took 1069 ms. Trace:
I20260709 14:54:36.172446  2683 rpcz_store.cc:276] 0709 14:54:35.102642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.132657 (+1030015us) service_pool.cc:224] Handling call
0709 14:54:36.151705 (+ 19048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.168494 (+ 16789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.171622 (+  3128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.152415  9374 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:36.183768  2757 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:36.200094  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:36.202121  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:36.172010  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:36.220292  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:36.148510  9550 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:36.225821  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50492.2us
I20260709 14:54:36.159391  9637 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 14:54:36.227138  8708 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.133s
I20260709 14:54:36.153601  9621 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:36.192096  9701 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 14:54:36.236896  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73780.5us
W20260709 14:54:36.236925  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.241906  2678 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:36.234213  2760 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:36.173527  9445 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:36.243117  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1752) took 1137 ms. Trace:
I20260709 14:54:36.169392  9563 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.060s
W20260709 14:54:36.257253  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.235098  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.258847  9628 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:36.259410  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 307602us
I20260709 14:54:36.261241  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:36.261559  8708 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:36.262032  2760 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:36.262733  9620 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:36.263470  9620 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:36.264755  9620 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:36.266407  9620 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:36.091336  2685 rpcz_store.cc:276] 0709 14:54:35.049670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.601053 (+551383us) service_pool.cc:224] Handling call
0709 14:54:35.601503 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:35.611366 (+  9863us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:35.673987 (+ 62621us) log.cc:844] Serialized 14 byte log entry
0709 14:54:36.063508 (+389521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:36.063581 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:36.071275 (+  7694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.080343 (+  9068us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.080532 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21776,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":7061,"raft.queue_time_us":9548,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":24174,"thread_start_us":336919,"threads_started":1}
I20260709 14:54:36.274358  9549 raft_consensus.cc:3055] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:36.274956  9549 raft_consensus.cc:740] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:36.275645  3411 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.942s (SENT)
W20260709 14:54:36.277061  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46214.3us
I20260709 14:54:36.277390  9550 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:36.278805  9550 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:36.255031  9641 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:36.282493  9641 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.284668  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1741) took 2790 ms. Trace:
I20260709 14:54:36.294044  2685 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:36.285331  9621 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:36.285557  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.287147  9207 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:36.288017  9616 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:36.289772  9699 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:54:36.257170  2678 rpcz_store.cc:276] 0709 14:54:35.105743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.218338 (+1112595us) service_pool.cc:224] Handling call
0709 14:54:36.241608 (+ 23270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.242688 (+  1080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.243025 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.224355  9404 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:36.160638  9355 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:36.295509  2677 rpcz_store.cc:276] 0709 14:54:33.494264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:33.494654 (+   390us) service_pool.cc:224] Handling call
0709 14:54:33.495089 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:34.807800 (+1312711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.284335 (+1476535us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540022c754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:36.284591 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907273472}
I20260709 14:54:36.286625  9004 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:36.301288  9641 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:36.357228  9621 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:36.360363  9703 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:36.361071  9703 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:36.363159  9004 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:36.363734  9004 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:36.364040  9703 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:36.364801  9004 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:36.366259  9004 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:36.369568  9617 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:36.375228  9617 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:36.376799  9698 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 14:54:36.381151  9429 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:36.312147  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51560.5us
I20260709 14:54:36.322664  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.260104  9628 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 14:54:36.256950  9445 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.329717  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.340747  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1733) took 1072 ms. Trace:
I20260709 14:54:36.333904  2678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a"
dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:44984
I20260709 14:54:36.353695  2677 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 14:54:36.355872  9355 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:36.387431  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:36.408560  2756 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.310001  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:36.389740  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58543.8us
I20260709 14:54:36.390053  2757 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:36.404188  2678 raft_consensus.cc:606] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:36.411339  2678 raft_consensus.cc:614] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:36.404809  2685 rpcz_store.cc:276] 0709 14:54:35.267984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.274654 (+1006670us) service_pool.cc:224] Handling call
0709 14:54:36.293363 (+ 18709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.340161 (+ 46798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.340657 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.412748  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1735) took 1143 ms. Trace:
I20260709 14:54:36.414011  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1761) took 1027 ms. Trace:
I20260709 14:54:36.413803  2677 rpcz_store.cc:276] 0709 14:54:35.268855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.343165 (+1074310us) service_pool.cc:224] Handling call
0709 14:54:36.353281 (+ 10116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:36.412077 (+ 58796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.412445 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.416033  2679 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:36.414661  2685 rpcz_store.cc:276] 0709 14:54:35.386228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.412835 (+1026607us) service_pool.cc:224] Handling call
0709 14:54:36.413251 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.413545 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:36.413584 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:36.413649 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:36.413696 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.413772 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.413930 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:36.417711  2684 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
W20260709 14:54:36.426539  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116604us
I20260709 14:54:36.426539  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.448746  2751 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 14:54:36.457592  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 154725us
I20260709 14:54:36.427659  2650 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.453799  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.465376  2679 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:36.423125  9617 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 14:54:36.452029  9554 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:36.466439  9627 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:36.498733  2677 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 14:54:36.512481  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84914.1us
I20260709 14:54:36.518445  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:36.519250  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 371652us
I20260709 14:54:36.483924  9531 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:36.483668  9060 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:36.523468  9060 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.523957  9060 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:36.524941  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.418896  9337 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 14:54:36.530323  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:36.488657  9617 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:36.427659  9561 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 14:54:36.543444  2609 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:36.547144  2650 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.563300  2754 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 14:54:36.563547  9561 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:36.564800  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.510521  9627 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:36.532037  9609 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.003s
W20260709 14:54:36.573776  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 113709us
I20260709 14:54:36.575125  2677 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:36.572240  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:36.577131  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.536058  9531 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:36.577445  2756 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:36.580963  2758 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:36.582935  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.564211  9561 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:36.591467  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.592312  2650 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:36.595754  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:36.598592  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.574911  9627 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.604574  2677 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:36.581739  9703 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 14:54:36.584331  9621 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:36.584146  9356 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:36.606324  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 76112.7us
W20260709 14:54:36.611495  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.582291  9531 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:36.610713  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.597364  8708 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:36.613234  9356 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.619270  9586 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:36.619800  9586 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:36.579165  9620 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 14:54:36.622876  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:36.627170  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.623936  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:36.629504  2610 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:36.624373  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.634155  2677 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 11. (term mismatch)
I20260709 14:54:36.635073  2677 pending_rounds.cc:85] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 10
I20260709 14:54:36.586520  9609 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:36.638283  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.636233  2752 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:36.642515  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 488097us
I20260709 14:54:36.643261  2754 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:36.635471  2677 pending_rounds.cc:107] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 23.11
I20260709 14:54:36.653852  2677 raft_consensus.cc:2889] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:36.614681  9337 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:36.617000  9531 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:36.647094  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.619338  9356 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:36.648877  9637 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:36.650185  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:36.653000  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52017.9us
I20260709 14:54:36.654179  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:36.625267  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:36.662556  2605 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:36.670102  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94119.2us
I20260709 14:54:36.616801  9621 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:36.677699  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52493us
I20260709 14:54:36.640050  9561 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:36.655630  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:36.704437  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48959us
I20260709 14:54:36.691247  2760 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:36.691555  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.681269  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1757) took 1456 ms. Trace:
I20260709 14:54:36.663798  9337 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:36.652798  9538 consensus_queue.cc:1059] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 14:54:36.696285  9319 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:36.724838  9319 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:36.645762  9429 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:36.725630  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.726115  9429 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:54:36.726574  2752 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:36.726663  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.727406  9429 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:36.727813  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:36.728116  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.622793  9374 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:36.728834  2752 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 17: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:36.730674  9374 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:36.731578  9374 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:54:36.665041  9331 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 14:54:36.690732  9546 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:36.653069  9517 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 14:54:36.734537  9546 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:36.691231  8885 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.806s	user 0.017s	sys 0.000s
I20260709 14:54:36.736375  8885 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:36.737473  8885 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:36.738618  2682 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:36.723589  9337 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:36.719681  9697 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.241s	user 0.001s	sys 0.001s
I20260709 14:54:36.721781  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.690908  9710 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:36.620812  9586 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:36.727911  9429 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:36.732492  9027 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:36.711056  9636 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:36.732722  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.749848  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1759) took 1430 ms. Trace:
I20260709 14:54:36.741907  2754 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:36.723341  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:36.709865  2686 rpcz_store.cc:276] 0709 14:54:35.225181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.222047 (+996866us) service_pool.cc:224] Handling call
0709 14:54:36.230789 (+  8742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.232453 (+  1664us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:36.509014 (+276561us) log.cc:844] Serialized 14 byte log entry
0709 14:54:36.680681 (+171667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:36.680775 (+    94us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:36.680860 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.680960 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.681175 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":367731,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":40858,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":11677,"thread_start_us":341289,"threads_started":2}
W20260709 14:54:36.701004  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 90387.3us
I20260709 14:54:36.754611  2677 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:36.754703  2756 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 14:54:36.758798  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:36.753901  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:36.730033  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:36.747967  2760 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:36.750563  2683 rpcz_store.cc:276] 0709 14:54:35.319551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.344912 (+1025361us) service_pool.cc:224] Handling call
0709 14:54:36.393613 (+ 48701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.393911 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:36.596899 (+202988us) log.cc:844] Serialized 14 byte log entry
0709 14:54:36.744177 (+147278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:36.744279 (+   102us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:36.744364 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.744480 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.747762 (+  3282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22809,"prepare.queue_time_us":307118,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":15545,"raft.queue_time_us":19735,"thread_start_us":265593,"threads_started":2}
I20260709 14:54:36.780421  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.782080  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:36.789288  2754 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:36.723459  9544 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 14:54:36.794981  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55903.8us
W20260709 14:54:36.795765  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.800911  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.801282  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.802749  2686 raft_consensus.cc:2376] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21. Re-sending same reply.
I20260709 14:54:36.803356  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:36.804581  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 50845.5us
W20260709 14:54:36.813776  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 108442us
I20260709 14:54:36.815903  2756 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:36.816787  2760 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
W20260709 14:54:36.822880  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.754503  9703 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:36.789877  9561 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:36.831069  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49154.8us
I20260709 14:54:36.825845  2752 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:36.780231  9517 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:36.827742  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:36.830271  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:36.781306  9710 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:36.832098  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:36.806012  9547 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.815s	user 0.006s	sys 0.010s
I20260709 14:54:36.836463  2754 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 14:54:36.796099  9544 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.755539  9538 consensus_queue.cc:1243] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 14:54:36.780360  9636 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:36.843880  9579 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 14:54:36.794858  8885 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 14:54:36.802012  9355 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:36.825863  9706 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.155s	user 0.004s	sys 0.000s
I20260709 14:54:36.816296  9404 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:36.821923  9722 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:36.822341  9637 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:36.906167  9250 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:36.823612  9609 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:36.831323  9619 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:36.838532  9661 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:36.838042  9517 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:36.849691  9620 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:36.920537  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:36.907792  9641 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.551s	user 0.015s	sys 0.000s
W20260709 14:54:36.907065  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 179022us
I20260709 14:54:36.905861  9544 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:36.927937  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 169221us
W20260709 14:54:36.910956  9446 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.759s	user 0.019s	sys 0.004s
W20260709 14:54:36.936658  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116280us
I20260709 14:54:36.837548  9703 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:36.951622  2683 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:36.970963  9356 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:36.974290  9356 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:36.975993  9531 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:36.964619  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:36.978722  9531 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:36.932893  9712 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:36.921481  9355 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:37.057042  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92540.7us
I20260709 14:54:37.060782  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:37.060623  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:36.955180  9517 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:36.940778  9250 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:37.075114  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:36.950678  9620 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:37.066730  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:36.929086  9722 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.087502  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.079500  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1753) took 1973 ms. Trace:
I20260709 14:54:37.093842  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.088753  2681 rpcz_store.cc:276] 0709 14:54:35.106148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.223258 (+1117110us) service_pool.cc:224] Handling call
0709 14:54:36.244476 (+ 21218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.294686 (+ 50210us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:36.696930 (+402244us) log.cc:844] Serialized 14 byte log entry
0709 14:54:37.079068 (+382138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:37.079132 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:37.079193 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.079280 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:37.079433 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":324734,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":692905,"threads_started":3}
W20260709 14:54:37.095621  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:37.098755  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.110648  3546 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:36.999033  9723 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:37.077921  9719 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.169s	user 0.018s	sys 0.009s
I20260709 14:54:37.136168  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:36.982739  9446 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:37.125074  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68022.7us
I20260709 14:54:36.980525  9641 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.069240  9661 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:36.931061  9429 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.079401  9250 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:37.137434  2681 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:37.124253  2754 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:37.123872  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:37.088347  9616 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 14:54:37.087787  9531 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:37.143167  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 187641us
W20260709 14:54:37.117308  9599 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.023s	user 0.011s	sys 0.006s
I20260709 14:54:37.000043  9619 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.166697  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1282) took 1346 ms. Trace:
I20260709 14:54:37.095145  9617 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:37.148279  2683 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
W20260709 14:54:37.180405  9718 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 14:54:37.182464  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1763) took 1468 ms. Trace:
I20260709 14:54:37.183221  9722 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:37.185281  9715 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 14:54:36.997030  9710 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.187275  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:37.190096  9445 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.001s
W20260709 14:54:37.191776  9717 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 14:54:37.194373  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:37.195101  2678 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.172809  2611 rpcz_store.cc:276] 0709 14:54:35.633134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.640732 (+  7598us) service_pool.cc:224] Handling call
0709 14:54:35.641300 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:35.647705 (+  6405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:35.647748 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:35.683719 (+ 35971us) log.cc:844] Serialized 26 byte log entry
0709 14:54:35.830537 (+146818us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b480026cdd8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:54:36.067801 (+237264us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:36.067855 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.067934 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:36.949226 (+881292us) raft_consensus.cc:1685] finished
0709 14:54:36.949267 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:36.978964 (+ 29697us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5861,"raft.queue_time_us":22723,"raft.run_cpu_time_us":2504,"raft.run_wall_time_us":161829,"spinlock_wait_cycles":10356480,"thread_start_us":358763,"threads_started":1,"wal-append.queue_time_us":634594}
I20260709 14:54:37.194061  9712 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:37.200124  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:37.201665  9641 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:37.101056  9004 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.005s
I20260709 14:54:37.158489  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:37.172495  9637 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.143705  9621 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:37.201983  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:37.140879  9723 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.183127  2685 rpcz_store.cc:276] 0709 14:54:35.714209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.425022 (+710813us) service_pool.cc:224] Handling call
0709 14:54:36.434328 (+  9306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:36.473467 (+ 39139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:36.473512 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:36.502165 (+ 28653us) log.cc:844] Serialized 26 byte log entry
0709 14:54:36.790228 (+288063us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:36.790294 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.790402 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:37.182118 (+391716us) raft_consensus.cc:1685] finished
0709 14:54:37.182147 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:37.182367 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":205957,"threads_started":1,"wal-append.queue_time_us":365952}
W20260709 14:54:37.220431  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:37.239998  2680 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:37.324259  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1768) took 1094 ms. Trace:
I20260709 14:54:37.246650  2683 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:37.137192  9636 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:37.349854  2610 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.001s
W20260709 14:54:37.215020  9576 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.132s	user 0.015s	sys 0.002s
I20260709 14:54:37.245136  2684 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:37.158867  2752 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:37.248157  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1286) took 1601 ms. Trace:
I20260709 14:54:37.202248  9356 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:37.327486  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:37.254405  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:37.364070  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 109755us
I20260709 14:54:37.262595  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.251307  9546 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.269014  9250 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.263859  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:37.251288  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 82836.9us
I20260709 14:54:37.208112  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1298) took 1442 ms. Trace:
I20260709 14:54:37.254900  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:37.324839  2680 rpcz_store.cc:276] 0709 14:54:36.230177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.239176 (+1008999us) service_pool.cc:224] Handling call
0709 14:54:37.239599 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:37.323801 (+ 84202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.324157 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:37.249655  9374 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:37.247498  9729 log.cc:927] Time spent T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.002s
W20260709 14:54:37.384234  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:37.384563  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 129833us
W20260709 14:54:37.354605  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:37.385427  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:37.385669  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:37.246660  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:37.248173  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 1977) took 1134 ms. Trace:
W20260709 14:54:37.253355  9711 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.606s	user 0.004s	sys 0.001s
I20260709 14:54:37.204836  9446 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.249732  9628 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:37.202545  9641 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:37.244292  9722 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:37.362607  2683 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:37.356598  2609 rpcz_store.cc:276] 0709 14:54:35.646428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.690780 (+ 44352us) service_pool.cc:224] Handling call
0709 14:54:35.717125 (+ 26345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.557647 (+840522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.247909 (+690262us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:37.248089 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899768192}
I20260709 14:54:37.233881  9531 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.211485  9616 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:37.366218  9707 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.742s	user 0.002s	sys 0.001s
I20260709 14:54:37.311309  9445 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.367189  2682 rpcz_store.cc:276] 0709 14:54:35.765632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.877688 (+112056us) service_pool.cc:224] Handling call
0709 14:54:37.207698 (+1330010us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001c0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:37.208024 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906683648}
I20260709 14:54:37.248320  9619 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:37.425089  9445 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.362156  9576 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.426419  9744 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:37.427284  9744 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.427568  9576 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:37.428419  2677 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 14:54:37.429440  9616 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.239773  9620 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.253266  9733 consensus_queue.cc:1059] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.152s
I20260709 14:54:37.433264  9620 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:37.434356  9733 raft_consensus.cc:3055] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:37.434777  9733 raft_consensus.cc:740] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:37.210568  9712 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.387084  2605 rpcz_store.cc:276] 0709 14:54:36.114028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.146519 (+ 32491us) service_pool.cc:224] Handling call
0709 14:54:36.287838 (+141319us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:36.288106 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.679343 (+391237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.223564 (+544221us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54002ce054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:37.248094 (+ 24530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864975872}
I20260709 14:54:37.366197  9636 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 14:54:37.386309  3411 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.510s (SENT)
I20260709 14:54:37.353052  9355 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:37.390504  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:37.252552  9547 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 14:54:37.169584  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:37.352152  2610 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:37.401057  2681 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:54:37.383486  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:37.501368  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117961us
I20260709 14:54:37.215196  9319 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.404605  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.507983  3542 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:37.405336  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.405607  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.407773  3546 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:37.509758  2678 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:37.512059  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:37.409816  2685 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 14:54:37.408268  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.250875  9599 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.391834  9628 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:37.418684  9724 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 14:54:37.405519  9641 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:37.419697  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:37.537770  9728 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.342s	user 0.002s	sys 0.002s
W20260709 14:54:37.538501  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 118948us
I20260709 14:54:37.205750  9731 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.183s
I20260709 14:54:37.420181  2683 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
W20260709 14:54:37.427346  9561 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.581s	user 0.004s	sys 0.008s
I20260709 14:54:37.253319  9723 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:37.249311  9703 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:37.441728  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:37.445885  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:37.455155  9576 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:37.389832  9374 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.458701  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:37.565623  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.523568  9355 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:37.461651  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:37.435791  9733 consensus_queue.cc:260] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.214560  9661 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:37.478018  9627 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.861s	user 0.014s	sys 0.000s
I20260709 14:54:37.575384  9627 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.479425  9207 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 14:54:37.517310  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 348397us
W20260709 14:54:37.520517  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 130191us
I20260709 14:54:37.509066  3537 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:37.557507  8885 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:37.429435  9531 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.531932  9250 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.532114  9599 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:37.586546  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.015s	sys 0.006s
I20260709 14:54:37.510511  9636 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:37.588002  9636 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.533722  9628 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.591533  9628 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:37.534667  9712 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:37.535261  9747 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:37.595709  9374 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:37.528198  9446 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:54:37.560019  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 118433us
I20260709 14:54:37.565414  9576 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.597214  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:37.598686  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:37.568586  2759 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:37.554440  9619 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:37.573077  2604 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:37.428437  9744 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:37.575771  9696 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.087s	user 0.004s	sys 0.000s
I20260709 14:54:37.613436  9547 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:37.479408  9517 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:37.609375  2576 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 1.504s (SENT) [suppressed 20 similar messages]
I20260709 14:54:37.620812  9517 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:37.622123  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1281) took 1989 ms. Trace:
I20260709 14:54:37.622680  2612 rpcz_store.cc:276] 0709 14:54:35.632693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:35.637754 (+  5061us) service_pool.cc:224] Handling call
0709 14:54:35.638302 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:35.653112 (+ 14810us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:35.692101 (+ 38989us) log.cc:844] Serialized 14 byte log entry
0709 14:54:36.063082 (+370981us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:36.092583 (+ 29501us) log.cc:844] Serialized 26 byte log entry
0709 14:54:36.092758 (+   175us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:36.092959 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:36.093059 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:37.539885 (+1446826us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:37.621584 (+ 81699us) raft_consensus.cc:1685] finished
0709 14:54:37.621710 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:37.621910 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29688,"prepare.queue_time_us":31608,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":19221,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":19259,"spinlock_wait_cycles":22649728,"thread_start_us":351355,"threads_started":1,"wal-append.queue_time_us":614622}
I20260709 14:54:37.219095  9549 consensus_queue.cc:260] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:37.575428  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 189156us
I20260709 14:54:37.579128  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:37.625854  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:37.626052  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47040.5us
I20260709 14:54:37.626148  9549 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:37.569607  9355 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:37.580484  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:37.581573  9703 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:37.582605  9004 consensus_queue.cc:1048] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:37.583305  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:37.580621  9207 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:37.584875  9627 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:37.585986  9060 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.062s	user 0.008s	sys 0.012s
W20260709 14:54:37.519738  9711 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 14:54:37.602622  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1796) took 1016 ms. Trace:
W20260709 14:54:37.603595  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.015s	sys 0.007s
W20260709 14:54:37.607239  2760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.008s	sys 0.010s
W20260709 14:54:37.660548  2760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.009s	sys 0.011s
I20260709 14:54:37.659976  2759 rpcz_store.cc:276] 0709 14:54:36.585925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.593336 (+  7411us) service_pool.cc:224] Handling call
0709 14:54:37.461461 (+868125us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:37.602515 (+141054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5017344}
W20260709 14:54:37.610870  9331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.009s	sys 0.010s
I20260709 14:54:37.611668  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:37.571364  9723 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:37.614780  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:37.599524  9531 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:37.562657  9736 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:37.633536  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:37.645649  2751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.018s	sys 0.003s
I20260709 14:54:37.647646  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:37.646870  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.013s	sys 0.008s
W20260709 14:54:37.677523  2751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.018s	sys 0.004s
I20260709 14:54:37.679483  3543 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:37.677138  2756 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.678409  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:37.683882  2610 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:37.685283  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.014s	sys 0.008s
I20260709 14:54:37.608707  8885 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.599546  9576 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:37.668718  2754 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:37.644155  9355 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:37.672086  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91680.1us
I20260709 14:54:37.668344  2755 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:37.694496  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:37.658977  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 275480us
I20260709 14:54:37.649973  9250 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 14:54:37.644661  9337 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.899s	user 0.015s	sys 0.000s
I20260709 14:54:37.657234  9517 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:37.666097  9331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.010s	sys 0.010s
I20260709 14:54:37.707466  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.649745  9445 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 14:54:37.649024  9374 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 14:54:37.709616  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.692272  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:37.708719  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.698499  2685 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:37.695334  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:37.708616  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:37.731770  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:37.673696  9531 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:37.695827  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:37.740190  2605 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 14:54:37.738497  3541 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 14:54:37.739046  2609 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:37.677417  9446 raft_consensus.cc:2764] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 14:54:37.745126  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.746027  3545 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 14:54:37.664409  9060 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.696714  9730 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:54:37.747996  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46277.8us
W20260709 14:54:37.759977  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 150633us
W20260709 14:54:37.755177  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 923096us
W20260709 14:54:37.755319  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 172098us
I20260709 14:54:37.693807  9429 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:37.656764  9747 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 14:54:37.749890  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:37.751479  2603 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:37.769249  2610 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.758890  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:37.754276  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58546.1us
I20260709 14:54:37.748761  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:37.756747  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 48261.3us
I20260709 14:54:37.770794  2611 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:37.775374  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:37.743268  9723 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:37.734774  9561 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:37.675475  9750 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 14:54:37.787899  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.790985  2756 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:37.745189  9250 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.747411  9531 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:37.777536  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 150885us
I20260709 14:54:37.793761  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:37.805462  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.780544  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:37.798125  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:37.806478  2609 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:37.783048  9544 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.000s
I20260709 14:54:37.731449  9337 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.768484  9616 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:37.767623  9429 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:37.774324  8708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.012s	sys 0.035s
I20260709 14:54:37.774379  9550 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:37.748390  9637 consensus_queue.cc:1059] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 14:54:37.750293  9517 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.845453  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:37.739625  9736 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 14:54:37.847519  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 86472.8us
I20260709 14:54:37.860457  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:37.789589  9561 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:37.861809  9561 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:37.795605  9563 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:37.731626  9207 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:37.863576  9207 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.794440  9620 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:37.806561  9730 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.839053  9446 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:37.869587  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:37.893565  3538 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:37.855057  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.789654  9374 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 14:54:37.868916  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:37.885703  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68595.8us
I20260709 14:54:37.845923  9429 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:37.852820  8708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.013s	sys 0.035s
I20260709 14:54:37.847470  9550 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:37.902876  2606 raft_consensus.cc:606] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:37.897538  3541 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:37.833313  9544 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.867352  9620 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.889452  9747 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:37.933434  9703 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:37.933813  9617 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:37.934418  9617 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:37.934222  9703 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.918571  9554 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:37.938961  9747 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.939560  9554 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:37.940164  9747 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.941320  9375 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:37.941701  9747 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.941838  9375 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:37.897673  9744 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:37.942418  9620 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.943342  9744 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.944129  9620 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:37.946013  9744 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:37.851619  9576 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 14:54:37.946996  2607 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:37.927289  9550 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:37.951025  9750 raft_consensus.cc:3055] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:37.951612  9750 raft_consensus.cc:740] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:37.953007  9750 consensus_queue.cc:260] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.956535  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:37.956337  9250 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:37.959131  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:37.971853  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 237499us
W20260709 14:54:37.974582  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:37.855001  9745 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.281s
I20260709 14:54:37.906610  9637 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:37.996767  2605 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:37.954701  9750 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:37.925212  9446 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:37.967031  9703 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:38.086627  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.093879  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 225174us
W20260709 14:54:38.098327  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.120169  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:38.108431  9617 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:37.985931  9004 raft_consensus.cc:2764] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 14:54:38.130703  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43306 (request call id 27) took 1070 ms. Trace:
W20260709 14:54:38.130995  3411 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.026s (SENT)
I20260709 14:54:37.938426  9641 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:38.057786  9207 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:38.129329  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 350812us
I20260709 14:54:38.131194  2606 rpcz_store.cc:276] 0709 14:54:37.006032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.074977 (+ 68945us) service_pool.cc:224] Handling call
0709 14:54:38.066240 (+991263us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b540016dfd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e640853 00007fa17360e60a 00007fa16cb947d4 00007fa16cb94762 00007fa16cb94691 00007fa16cb9460c 00007fa16cb938c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:38.066368 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711833472}
W20260709 14:54:38.160190  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 236806us
W20260709 14:54:38.160799  9628 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.006s
I20260709 14:54:38.063278  9637 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.144541  8885 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:38.184317  9726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.007s	sys 0.027s
I20260709 14:54:38.158779  9207 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:38.177673  9641 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:38.205456  9754 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 14:54:38.230721  2756 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
W20260709 14:54:38.209430  9705 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.357s	user 0.005s	sys 0.000s
I20260709 14:54:38.055243  9375 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.237483  9375 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:38.215834  9726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.007s	sys 0.028s
I20260709 14:54:38.255012  9760 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:38.255908  9753 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:38.127719  9730 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.253304  9735 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:38.182771  9741 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.435s
I20260709 14:54:38.274211  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:38.280133  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 155332us
W20260709 14:54:38.302773  2611 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.003s
I20260709 14:54:38.304618  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:54:38.285511  3545 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.001s
W20260709 14:54:38.274358  9710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.026s	sys 0.010s
W20260709 14:54:38.314545  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.104s (SENT)
I20260709 14:54:38.315776  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:38.326112  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.316877  3545 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:38.345844  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:38.359153  9749 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:38.309731  9735 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:38.361727  9756 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.431s	user 0.002s	sys 0.000s
W20260709 14:54:38.363643  9779 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 14:54:38.318377  9710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.027s	sys 0.011s
I20260709 14:54:38.216254  9446 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.387660  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:38.383633  9544 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:38.309916  9785 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:38.402913  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 283107us
W20260709 14:54:38.364231  9778 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 14:54:38.412501  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:38.404979  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:38.400228  3540 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.003s
I20260709 14:54:38.409468  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:38.409916  2609 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.003s
I20260709 14:54:38.359079  9768 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:38.388710  9561 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.004s
W20260709 14:54:38.358964  9757 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.471s	user 0.001s	sys 0.001s
I20260709 14:54:38.414525  3540 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:38.410084  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.415176  3545 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:38.367153  9517 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:38.358968  9786 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:38.292189  9760 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.486117  2755 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:38.419242  2609 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:54:38.430483  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 182898us
I20260709 14:54:38.467402  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:38.491770  3540 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 14:54:38.514117  2723 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.798s (SENT)
I20260709 14:54:38.478355  2759 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 14:54:38.514638  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:38.513772  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46535.1us
W20260709 14:54:38.518551  9772 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 14:54:38.519490  9760 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:38.519778  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 214016us
W20260709 14:54:38.510758  9723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.016s	sys 0.035s
I20260709 14:54:38.520684  9207 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:38.520869  9723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.017s	sys 0.036s
I20260709 14:54:38.492282  9563 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:38.521697  9563 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:38.521453  9723 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:38.522469  9763 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.317s	user 0.000s	sys 0.002s
I20260709 14:54:38.524052  9784 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:38.524578  9784 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:38.523602  9723 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:38.525460  9762 log.cc:927] Time spent T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 14:54:38.525960  9628 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:54:38.526280  9748 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.441s	user 0.002s	sys 0.000s
I20260709 14:54:38.526777  9375 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:38.517221  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:38.417325  9768 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:38.481192  9752 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 14:54:38.473783  9755 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:38.572778  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.573920  2755 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 14:54:38.548290  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.549495  2603 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:38.554503  2757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.000s	sys 0.006s
I20260709 14:54:38.555698  2609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:38.557693  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:38.579493  2609 raft_consensus.cc:606] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:38.580341  2609 raft_consensus.cc:614] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:38.583094  3540 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 14:54:38.583902  3540 pending_rounds.cc:85] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 10
I20260709 14:54:38.584224  3540 pending_rounds.cc:107] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 14:54:38.584544  3540 raft_consensus.cc:2889] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:38.576387  2757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.000s	sys 0.006s
I20260709 14:54:38.559556  9517 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:38.522269  9563 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:38.577747  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:38.587627  9734 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:38.588881  9703 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:38.589406  9703 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:38.589838  9703 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:38.590428  3545 raft_consensus.cc:1240] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:38.468114  9561 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.491295  9793 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:38.542001  9207 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:38.640172  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:38.602996  9775 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
I20260709 14:54:38.577414  9735 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:38.577427  9710 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:38.572865  9755 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:38.667692  2682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.008s	sys 0.023s
I20260709 14:54:38.658406  3537 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 14:54:38.682638  2682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.008s	sys 0.024s
I20260709 14:54:38.644047  9731 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:38.624466  9741 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 14:54:38.678726  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.240394  9331 consensus_queue.cc:1175] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 24 status { last_received { term: 23 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 14:54:38.683437  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1314) took 1749 ms. Trace:
I20260709 14:54:38.644208  9743 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:38.693367  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.685292  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 97457.3us
I20260709 14:54:38.484524  9786 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:38.699536  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142156us
I20260709 14:54:38.689822  2755 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 14:54:38.696964  9790 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 14:54:38.688696  9768 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:38.697705  9531 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.861s	user 0.010s	sys 0.003s
I20260709 14:54:38.697769  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.681406  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1950) took 1921 ms. Trace:
I20260709 14:54:38.708159  2759 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 14:54:38.693292  2682 rpcz_store.cc:276] 0709 14:54:36.933932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.425366 (+491434us) service_pool.cc:224] Handling call
0709 14:54:37.425796 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:37.426063 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:37.753290 (+327227us) log.cc:844] Serialized 14 byte log entry
0709 14:54:37.992650 (+239360us) log.cc:844] Serialized 14 byte log entry
0709 14:54:38.683041 (+690391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:38.683100 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:38.683156 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:38.683228 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:38.683388 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23514,"prepare.queue_time_us":608146,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":108,"raft.queue_time_us":477499,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":37406,"spinlock_wait_cycles":80809600,"thread_start_us":1222927,"threads_started":3,"wal-append.queue_time_us":758722}
I20260709 14:54:38.620378  9517 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.699574  9743 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:38.707367  2751 rpcz_store.cc:276] 0709 14:54:36.753007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.844923 (+ 91916us) service_pool.cc:224] Handling call
0709 14:54:36.845462 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:37.679669 (+834207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:37.679705 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:37.700019 (+ 20314us) log.cc:844] Serialized 26 byte log entry
0709 14:54:37.907277 (+207258us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:37.907339 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.907462 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:38.665616 (+758154us) raft_consensus.cc:1685] finished
0709 14:54:38.665649 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:38.673938 (+  8289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241821,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":19102,"thread_start_us":972047,"threads_started":2,"wal-append.queue_time_us":435706}
I20260709 14:54:38.656590  9207 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:38.714253  9722 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.281s	user 0.015s	sys 0.002s
I20260709 14:54:38.699223  9731 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:38.739275  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.740156  2758 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:38.739789  9731 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:38.740451  9550 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.000s
W20260709 14:54:38.731523  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.741039  9550 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:38.723384  9743 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:38.742484  9549 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.115s	user 0.014s	sys 0.000s
I20260709 14:54:38.745087  9616 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:38.734114  9747 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.001s
I20260709 14:54:38.738042  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.682133  9710 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:38.748311  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.747958  9616 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:38.724054  9768 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:38.749563  9742 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.553s	user 0.009s	sys 0.000s
I20260709 14:54:38.750277  2755 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:38.749619  9703 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:38.747455  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:38.751588  9786 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:38.753445  9207 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:38.754501  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.754767  2759 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:38.767551  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:38.756647  9641 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.001s
I20260709 14:54:38.768808  3547 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:38.771328  2759 raft_consensus.cc:1240] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:38.710238  9531 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:38.727213  9741 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 14:54:38.776170  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.778127  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.782088  2758 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:38.788444  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.789598  2758 raft_consensus.cc:2410] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:38.767266  9812 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:38.767812  9799 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:38.792227  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.798461  9812 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:38.798905  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.801780  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.800304  9812 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:38.806723  3411 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.685s (SENT)
I20260709 14:54:38.807422  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:38.808334  9747 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:38.808647  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 1954) took 1370 ms. Trace:
I20260709 14:54:38.809115  2752 rpcz_store.cc:276] 0709 14:54:37.438864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.439060 (+   196us) service_pool.cc:224] Handling call
0709 14:54:37.499299 (+ 60239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:37.583973 (+ 84674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:37.584018 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:37.584699 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 14:54:37.824616 (+239917us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:37.824682 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.945166 (+120484us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:37.945240 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:38.808287 (+863047us) raft_consensus.cc:1685] finished
0709 14:54:38.808310 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:38.808578 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10971,"raft.queue_time_us":18349,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"spinlock_wait_cycles":70898944,"thread_start_us":238506,"threads_started":1,"wal-append.queue_time_us":1187584}
I20260709 14:54:38.811311  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.810964  9799 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:38.812073  2752 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:38.813278  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.814179  2758 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:38.816378  9561 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:38.823263  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.822768  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.824494  2754 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:38.828771  3411 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.077s (SENT)
I20260709 14:54:38.829581  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:38.832561  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.839007  2751 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 14:54:38.762149  9755 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:38.843324  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.858233  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.858748  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.844203  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:38.830921  9355 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.131s	user 0.014s	sys 0.002s
I20260709 14:54:38.849211  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.862543  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:38.867038  2752 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 14:54:38.843998  9429 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.921s	user 0.016s	sys 0.001s
I20260709 14:54:38.869359  2754 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:38.836406  9641 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:38.874039  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:38.873157  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44459.1us
I20260709 14:54:38.876296  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:38.878712  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.877147  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.880692  2752 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 14:54:38.881548  2754 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
W20260709 14:54:38.883996  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:38.856546  9805 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:38.889318  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.893216  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:38.894973  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:38.867964  9803 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:38.897069  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:38.901619  2681 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:54:38.903468  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:38.904608  2683 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:38.912669  9799 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:38.886415  9549 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:38.892263  9755 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:38.919428  9549 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.923067  9550 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:38.923339  9517 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:38.924459  9531 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:38.958011  2752 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:38.875661  9355 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:38.922461  9549 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:38.967586  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 1982) took 1175 ms. Trace:
I20260709 14:54:38.968017  2605 rpcz_store.cc:276] 0709 14:54:37.755418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.773836 (+ 18418us) service_pool.cc:224] Handling call
0709 14:54:38.930135 (+1156299us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54003791d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:38.930404 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429593984}
I20260709 14:54:38.877933  9429 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:38.883635  9641 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:38.978443  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1488) took 1274 ms. Trace:
I20260709 14:54:38.978482  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.981746  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.982350  2756 raft_consensus.cc:2368] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
W20260709 14:54:38.966977  9732 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.769s	user 0.007s	sys 0.007s
W20260709 14:54:38.993409  3411 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.887s (SENT)
W20260709 14:54:38.997977  3411 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.932s (SENT)
I20260709 14:54:38.998392  2751 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:38.999460  2758 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 14:54:38.999815  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.000408  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:38.979105  3541 rpcz_store.cc:276] 0709 14:54:37.697896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.962368 (+264472us) service_pool.cc:224] Handling call
0709 14:54:37.996920 (+ 34552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:38.123184 (+126264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:38.123246 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:38.136685 (+ 13439us) log.cc:844] Serialized 26 byte log entry
0709 14:54:38.492778 (+356093us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:38.549137 (+ 56359us) log.cc:844] Serialized 14 byte log entry
0709 14:54:38.568367 (+ 19230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:38.568473 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:38.970157 (+401684us) raft_consensus.cc:1685] finished
0709 14:54:38.970183 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:38.972046 (+  1863us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144487,"prepare.queue_time_us":56345,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":185225,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":52108,"spinlock_wait_cycles":66280320,"thread_start_us":268363,"threads_started":1,"wal-append.queue_time_us":574795}
I20260709 14:54:39.001194  2752 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:39.000018  2758 pending_rounds.cc:85] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 11
I20260709 14:54:39.002239  2758 pending_rounds.cc:107] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 14:54:38.899080  9805 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.002530  2758 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:38.984695  9703 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:38.984632  9446 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.013893  9554 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.022691  9429 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.024147  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1481) took 1335 ms. Trace:
I20260709 14:54:39.024688  3547 rpcz_store.cc:276] 0709 14:54:37.688658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.688799 (+   141us) service_pool.cc:224] Handling call
0709 14:54:39.023897 (+1335098us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400003554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:39.024034 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380292480}
I20260709 14:54:39.026113  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.019096  9319 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:39.026808  3547 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:39.010910  3411 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.063s (SENT)
I20260709 14:54:39.042328  9355 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.047719  9446 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.032411  9743 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.031080  2685 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
W20260709 14:54:39.087266  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.027597  9805 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.088235  9743 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.075659  9749 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.027920  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.089843  9749 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.028457  3537 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:39.092496  9749 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.047021  9641 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.075743  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1495) took 1221 ms. Trace:
I20260709 14:54:39.096027  3542 rpcz_store.cc:276] 0709 14:54:37.854501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:38.571602 (+717101us) service_pool.cc:224] Handling call
0709 14:54:38.581741 (+ 10139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.048292 (+466551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:39.048330 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:39.048381 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:39.048428 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.050552 (+  2124us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54004b41d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:39.050621 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:39.075620 (+ 24999us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41338,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":1071991040}
W20260709 14:54:39.079891  9750 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.956s	user 0.014s	sys 0.000s
I20260709 14:54:39.044061  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.099793  3540 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.103024  9780 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.103678  9780 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.106079  9780 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.090646  2753 raft_consensus.cc:2376] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21. Re-sending same reply.
I20260709 14:54:39.108130  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.108942  3540 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:39.110941  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.112720  9355 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:39.113407  9355 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:39.114503  9355 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.116135  9750 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.116780  9750 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.047349  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.041756  2754 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:39.119457  2751 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:39.119918  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.120556  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.121222  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.121917  2754 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:39.122057  3542 raft_consensus.cc:1275] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 14:54:39.043401  9765 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.501s	user 0.004s	sys 0.000s
I20260709 14:54:39.123186  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.123996  2754 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:39.124265  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1310) took 2268 ms. Trace:
I20260709 14:54:39.124791  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.125548  2754 raft_consensus.cc:2376] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22. Re-sending same reply.
I20260709 14:54:39.124668  2684 rpcz_store.cc:276] 0709 14:54:36.855873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.373427 (+517554us) service_pool.cc:224] Handling call
0709 14:54:37.404762 (+ 31335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:37.435199 (+ 30437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:37.435246 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:37.435894 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 14:54:37.782053 (+346159us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:37.782119 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:37.782212 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:39.034131 (+1251919us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:39.123967 (+ 89836us) raft_consensus.cc:1685] finished
0709 14:54:39.123991 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:39.124203 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2229,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":46114,"spinlock_wait_cycles":47049472,"thread_start_us":345531,"threads_started":1,"wal-append.queue_time_us":966420}
I20260709 14:54:39.125806  9004 consensus_queue.cc:1048] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:39.126940  2751 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:39.128114  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.094874  9446 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:39.035862  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.092844  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1498) took 1032 ms. Trace:
I20260709 14:54:39.129304  3537 rpcz_store.cc:276] 0709 14:54:38.060381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:38.687972 (+627591us) service_pool.cc:224] Handling call
0709 14:54:39.028047 (+340075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.091422 (+ 63375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.092566 (+  1144us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400003550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:39.092760 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745507072}
I20260709 14:54:39.098233  2759 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:39.094515  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:39.043730  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:39.131435  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120611us
I20260709 14:54:39.050123  9563 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.133276  9780 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:39.124816  3541 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:39.135033  2682 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:39.126492  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.136057  2754 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:39.136153  9563 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.136590  3537 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:39.139894  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.141739  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.142529  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.131516  2758 raft_consensus.cc:2376] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21. Re-sending same reply.
I20260709 14:54:39.146188  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.146806  2759 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:39.143255  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.147140  3541 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:39.144126  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.144312  9749 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.145220  9641 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.145366  9750 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:39.143568  9446 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.148874  3537 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:39.157655  2754 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:39.155457  2751 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:39.155781  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 184042us
I20260709 14:54:39.156315  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.159878  2758 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:39.156601  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53825.3us
I20260709 14:54:39.160298  3538 pending_rounds.cc:85] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 10
I20260709 14:54:39.109678  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.160820  3538 pending_rounds.cc:107] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 14:54:39.161235  3538 raft_consensus.cc:2889] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:39.161316  2753 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:39.161834  3547 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:39.164752  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.165253  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.163288  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.168316  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:39.165880  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.169663  2754 raft_consensus.cc:2410] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 14:54:39.171550  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:39.170516  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.151554  9743 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:39.167152  9794 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 14:54:39.181615  9812 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:39.182335  9538 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 14:54:39.183635  9782 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.969s	user 0.003s	sys 0.000s
I20260709 14:54:39.186121  9637 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:39.187022  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1790) took 2650 ms. Trace:
I20260709 14:54:39.190328  2757 rpcz_store.cc:276] 0709 14:54:36.536790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.548240 (+ 11450us) service_pool.cc:224] Handling call
0709 14:54:36.583668 (+ 35428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:36.614160 (+ 30492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:36.614195 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:36.638249 (+ 24054us) log.cc:844] Serialized 26 byte log entry
0709 14:54:37.833889 (+1195640us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800336e80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:37.907664 (+ 73775us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:38.585735 (+678071us) log.cc:844] Serialized 14 byte log entry
0709 14:54:38.586093 (+   358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:38.586215 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:39.182916 (+596701us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:39.186679 (+  3763us) raft_consensus.cc:1685] finished
0709 14:54:39.186706 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:39.186949 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":804846,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":35307,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":26985,"spinlock_wait_cycles":41954816,"thread_start_us":670726,"threads_started":2,"wal-append.queue_time_us":1478516}
I20260709 14:54:39.196105  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.196880  3540 raft_consensus.cc:2368] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
I20260709 14:54:39.175464  9810 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:39.175143  9554 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:39.157938  9749 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.203078  9749 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.205617  9736 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.194409  9750 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 14:54:39.206386  9736 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.206947  9710 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:39.208179  9708 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:39.208707  9708 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:39.188611  9812 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:39.163246  9004 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 14:54:39.164378  9446 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.214272  9576 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:39.158234  9641 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.209899  9554 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:39.218019  9641 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.218293  9554 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:39.228170  9628 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:39.228756  9628 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:39.190441  9743 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.229701  9628 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.230433  2650 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
I20260709 14:54:39.214349  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.220245  2760 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:39.236004  9616 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.213321  9810 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:39.236337  9446 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.237468  9810 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.240530  9337 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.213020  9812 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:39.218932  9708 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.242734  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:39.214124  9745 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 14:54:39.243685  2609 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:39.243940  9744 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:39.244449  9744 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:39.245527  9744 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.245879  9708 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:39.249783  9770 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.047s	user 0.003s	sys 0.000s
W20260709 14:54:39.250584  9788 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.847s	user 0.000s	sys 0.002s
I20260709 14:54:39.254276  9744 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.255688  9784 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.257395  9784 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:39.257853  9628 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:39.259698  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:39.232318  9743 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:39.260169  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:39.262547  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50403.6us
I20260709 14:54:39.264017  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.265082  2609 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:39.265851  9616 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.266149  9620 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:54:39.268186  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.268033  9620 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:54:39.269203  8708 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:39.269701  8708 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:39.271580  8708 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.266431  9616 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.158994  9563 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 14:54:39.280725  3546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1838) took 2020 ms (client timeout 2142 ms). Trace:
W20260709 14:54:39.281272  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 83794.4us
I20260709 14:54:39.276470  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.271595  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:39.281344  3546 rpcz_store.cc:269] 0709 14:54:37.260556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.262148 (+  1592us) service_pool.cc:224] Handling call
0709 14:54:39.280428 (+2018280us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54002017d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:39.280637 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95925504}
I20260709 14:54:39.284883  2751 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 14:54:39.285331  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1293) took 3323 ms. Trace:
I20260709 14:54:39.285801  2604 rpcz_store.cc:276] 0709 14:54:35.962015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.138720 (+176705us) service_pool.cc:224] Handling call
0709 14:54:36.169509 (+ 30789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:37.613205 (+1443696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.284188 (+1670983us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54003c7654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:39.285220 (+  1032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3219555712}
I20260709 14:54:39.280908  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.235262  2753 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:39.297228  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.272046  9060 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.292938  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:39.295249  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:39.304003  2605 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:39.296589  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.285652  2751 pending_rounds.cc:85] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 12
I20260709 14:54:39.298215  2609 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:39.304857  2751 pending_rounds.cc:107] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 14:54:39.305029  2604 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 14:54:39.300275  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:39.305873  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.305354  2751 raft_consensus.cc:2889] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:39.306464  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.307704  2650 leader_election.cc:403] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 14:54:39.313158  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:39.314966  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.317834  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:39.319327  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89137.3us
I20260709 14:54:39.320062  2611 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:39.320968  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.323776  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:39.328344  2605 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:39.332418  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.332558  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.288697  9755 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 14:54:39.325655  9563 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:39.346743  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.349424  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.351249  3540 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
W20260709 14:54:39.357816  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.360396  2608 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:54:39.336066  9661 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:39.371814  9812 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.376734  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.377538  3540 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:39.377769  9563 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22.
W20260709 14:54:39.378286  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.380023  9337 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:39.380769  9337 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.387553  9812 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.389424  3540 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:39.392993  9586 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:39.392820  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:39.394593  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.395817  2603 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.396574  2609 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:39.394496  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.397579  2611 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:39.404147  2608 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 14:54:39.345120  9544 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:39.412045  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52255.6us
I20260709 14:54:39.415045  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.422087  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.423609  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.425757  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1626) took 2050 ms. Trace:
I20260709 14:54:39.428365  2609 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:39.428670  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.431856  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:39.426226  3543 rpcz_store.cc:276] 0709 14:54:37.375637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.375766 (+   129us) service_pool.cc:224] Handling call
0709 14:54:37.376146 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:37.691464 (+315318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.425463 (+1733999us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54003df9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:39.425681 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696940800}
I20260709 14:54:39.432533  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.433630  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:39.434696  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.435864  3537 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:39.435755  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.439199  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.439718  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.434252  3543 raft_consensus.cc:2376] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18. Re-sending same reply.
I20260709 14:54:39.439924  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.440944  3537 raft_consensus.cc:2368] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:54:39.440964  3540 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:39.442473  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.443284  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.446242  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.446128  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.452107  2608 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:39.435066  9750 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.445466  9810 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:39.456362  2726 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:39.477741  2726 leader_election.cc:403] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 14:54:39.450289  3543 raft_consensus.cc:2376] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16. Re-sending same reply.
I20260709 14:54:39.457410  9616 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.458384  9761 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:39.459465  9823 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 14:54:39.459658  3537 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 14:54:39.486120  3537 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 11
I20260709 14:54:39.486515  3537 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 14:54:39.486917  3537 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:39.463163  9637 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:39.467561  3411 leader_election.cc:400] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:39.471504  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.496562  3411 leader_election.cc:403] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 14:54:39.435171  9544 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:39.471793  2609 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:39.472081  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.472448  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.475528  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.507517  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:39.454733  9750 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.484750  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.493233  2603 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
W20260709 14:54:39.508153  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59371.5us
I20260709 14:54:39.511942  3543 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.519845  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:39.520854  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 64668.8us
I20260709 14:54:39.523119  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:39.524814  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.527354  2604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:39.528338  2604 raft_consensus.cc:606] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:39.535895  2604 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:39.425009  9554 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.570545  3543 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:39.540696  9810 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:39.544924  9544 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.556098  8708 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.564013  9617 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.539237  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.594002  3547 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.581048  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.517534  9734 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.517951  9549 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 14:54:39.584287  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 116895us
I20260709 14:54:39.587761  2678 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:39.503877  9761 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.531275  9755 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:39.590384  9750 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:39.599711  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:39.605278  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.632438  9733 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.618892  9617 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 14:54:39.603773  2604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "963a4155717b4274b0121341056b7cb8"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:39.603742  9810 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:39.609665  9784 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.611526  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.644850  2604 raft_consensus.cc:606] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Received request to transfer leadership to TS c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.645349  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.646962  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:39.647835  3543 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:39.610518  9586 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.617622  9563 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.597131  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.603953  9793 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:39.625998  9777 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.107s	user 0.002s	sys 0.004s
I20260709 14:54:39.627058  2686 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:39.619517  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:39.650010  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 50432.6us
I20260709 14:54:39.650444  2608 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:54:39.665093  3412 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Timed out: RequestConsensusVote RPC to 127.0.107.194:40091 timed out after 1.805s (SENT)
I20260709 14:54:39.697014  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:39.701407  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 104400us
I20260709 14:54:39.703220  2726 leader_election.cc:400] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:39.704118  2726 leader_election.cc:403] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:39.645444  2604 raft_consensus.cc:614] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 14:54:39.707726  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:39.713814  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48798.6us
I20260709 14:54:39.714051  2576 leader_election.cc:400] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Vote denied by peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831) with higher term. Message: Invalid argument: T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
W20260709 14:54:39.715283  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60301.2us
W20260709 14:54:39.716365  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 96643.7us
I20260709 14:54:39.699430  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.718997  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:39.723775  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.725579  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1300) took 3959 ms. Trace:
I20260709 14:54:39.725729  3547 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:39.735704  3543 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:39.738577  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.739356  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.741974  2576 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.742017  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:39.744576  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:39.744925  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45626.4us
I20260709 14:54:39.695297  9794 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:39.712664  9810 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:39.695938  9776 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.763545  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:39.764377  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:39.712885  9750 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.766521  3540 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:54:39.726245  2679 rpcz_store.cc:276] 0709 14:54:35.766270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.466624 (+700354us) service_pool.cc:224] Handling call
0709 14:54:36.499212 (+ 32588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:36.499568 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:36.666569 (+167001us) log.cc:844] Serialized 14 byte log entry
0709 14:54:36.834932 (+168363us) log.cc:844] Serialized 14 byte log entry
0709 14:54:37.201714 (+366782us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b1800474ee0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:54:37.528532 (+326818us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:37.992303 (+463771us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b1800474ee0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287
0709 14:54:39.297407 (+1305104us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b1800474ee0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17203e3c7 00007fa16e6509a7 00007fa16e6600bf
0709 14:54:39.303388 (+  5981us) log.cc:844] Serialized 14 byte log entry
0709 14:54:39.725092 (+421704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:39.725161 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:39.725238 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.725335 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:39.725510 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142110,"prepare.queue_time_us":394264,"prepare.run_cpu_time_us":1353,"prepare.run_wall_time_us":6104,"raft.queue_time_us":321672,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":12043,"spinlock_wait_cycles":303610624,"thread_start_us":1071633,"threads_started":5,"wal-append.queue_time_us":501781,"wal-append.run_cpu_time_us":5279,"wal-append.run_wall_time_us":1346876}
I20260709 14:54:39.773283  2680 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:39.731150  9785 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:39.729130  9628 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.729389  9755 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:39.779328  2684 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 17: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:39.729113  9561 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:39.778980  2681 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
W20260709 14:54:39.777482  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58665.8us
I20260709 14:54:39.787386  2682 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:39.647881  9784 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.749396  9494 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:39.800310  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:39.758574  9331 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:39.744387  9374 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.804549  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:39.692855  9586 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.817028  3540 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 14:54:39.814508  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 68553.5us
I20260709 14:54:39.842921  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:39.844604  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:39.815095  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:39.793182  9777 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 14:54:39.723420  9563 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.766000  9554 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.724767  9793 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.787997  9780 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:39.836313  9785 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:39.749886  9383 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 14:54:39.779458  9355 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 14:54:39.860919  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121451us
I20260709 14:54:39.744302  9446 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 14:54:39.847162  9810 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:39.865478  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:39.867012  9060 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:39.873045  9617 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.874364  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1286) took 1132 ms. Trace:
I20260709 14:54:39.915509  2760 rpcz_store.cc:276] 0709 14:54:38.742082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:38.767088 (+ 25006us) service_pool.cc:224] Handling call
0709 14:54:39.873960 (+1106872us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:39.874204 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036649728}
I20260709 14:54:39.878214  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1747) took 1016 ms. Trace:
I20260709 14:54:39.883718  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.917501  2686 rpcz_store.cc:276] 0709 14:54:38.861635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:38.861865 (+   230us) service_pool.cc:224] Handling call
0709 14:54:38.878053 (+ 16188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.700278 (+822225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.877734 (+177456us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54002b2d54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:39.878093 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722013440}
I20260709 14:54:39.773375  9768 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 14:54:39.815412  9753 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:39.881388  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.777638  9708 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 14:54:39.852602  9374 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:39.939815  9753 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:39.779314  9637 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
W20260709 14:54:39.862808  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 133079us
I20260709 14:54:39.911675  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:39.912230  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46847.2us
I20260709 14:54:39.859869  9794 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:39.885557  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1489) took 2187 ms. Trace:
I20260709 14:54:39.815744  9561 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.796535  9750 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.916103  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:39.834347  9784 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:39.803074  9776 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:39.864838  9780 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.904086  9554 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:39.953359  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:40.004572  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51332.9us
I20260709 14:54:40.004856  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.955986  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.006982  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.957875  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:39.958791  9831 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 14:54:39.965948  9761 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:39.967955  9835 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 14:54:39.934157  9337 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:39.940860  9617 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:39.969765  9374 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:39.932660  9586 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:39.972028  9785 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:39.920784  9355 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:39.975716  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.917275  9620 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:39.989777  3540 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
W20260709 14:54:40.042140  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:39.981992  3536 rpcz_store.cc:276] 0709 14:54:37.698162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.957651 (+259489us) service_pool.cc:224] Handling call
0709 14:54:37.958000 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:38.236025 (+278025us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:38.282665 (+ 46640us) log.cc:844] Serialized 14 byte log entry
0709 14:54:39.226964 (+944299us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b18003842e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:39.431353 (+204389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:39.431426 (+    73us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:39.431479 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:39.885114 (+453635us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:39.885200 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:39.885458 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44130,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":266,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7222,"spinlock_wait_cycles":655950336,"thread_start_us":204122,"threads_started":1,"wal-append.queue_time_us":1246990}
I20260709 14:54:39.962502  9734 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:40.006503  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:39.958021  9753 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:40.004441  9744 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.003s
I20260709 14:54:39.988344  9446 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.017609  2754 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:39.997651  9794 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:40.022133  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.023499  2758 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:40.024521  9617 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:40.030543  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.071283  2604 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 14:54:40.031569  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.005040  9780 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:40.035303  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:40.008268  9784 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:40.009895  9554 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.012969  9637 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:40.068521  2759 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.067095  2612 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:40.027338  9708 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 14:54:40.076684  3538 raft_consensus.cc:2410] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 14:54:40.009110  9745 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:40.015421  9803 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 14:54:40.039911  9843 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.074780  9734 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.084735  9784 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:40.156787  9794 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.141196  9745 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:40.088655  9620 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 14:54:40.154803  9563 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.159677  9734 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.169559  9784 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.171255  9784 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:40.171937  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:40.143116  8708 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.539s	user 0.006s	sys 0.007s
I20260709 14:54:40.091648  9636 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.173949  9636 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.061252  9355 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.179621  9750 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.154407  9637 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:40.180845  9355 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:40.164610  2678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1775) took 2453 ms (client timeout 2313 ms). Trace:
I20260709 14:54:40.180346  9554 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:40.182919  2678 rpcz_store.cc:269] 0709 14:54:37.697361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.259237 (+1561876us) service_pool.cc:224] Handling call
0709 14:54:40.150720 (+891483us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540024e5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:40.150884 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752060672}
I20260709 14:54:40.077504  9446 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.181761  9637 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.182269  9355 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:40.204306  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:40.204434  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.153324  9843 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.208884  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.215360  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.216259  3540 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:40.218137  2758 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:40.203608  9563 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.221606  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.198019  9554 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:40.223857  3545 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:40.213346  9736 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.240528  3541 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:40.253912  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.254930  2754 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:40.225769  9734 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.212154  9355 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.248781  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:40.252513  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.252806  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.261253  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:40.244879  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 172998us
I20260709 14:54:40.267210  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.227690  9563 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.229405  9554 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:40.286229  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.286398  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.275864  2683 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.277891  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.283367  3411 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.289453  2755 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 14:54:40.289986  3411 leader_election.cc:403] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:40.279379  2681 raft_consensus.cc:1240] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:40.292939  2680 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 14:54:40.292124  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.293503  2680 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 10
I20260709 14:54:40.293979  2680 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 14:54:40.294263  2680 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:40.294561  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.295516  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.297974  2678 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 14:54:40.302120  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.303404  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.306818  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.307881  2757 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:54:40.308454  2683 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:40.310858  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.317484  2681 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:40.317859  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.318039  9854 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:40.318656  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.319389  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.320014  9793 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.271404  8708 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 14:54:40.320524  9848 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:40.320838  3547 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:40.321338  3547 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:40.321168  9848 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.275457  9734 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:40.322945  2612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1295) took 2648 ms (client timeout 1805 ms). Trace:
I20260709 14:54:40.323167  9734 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:40.323509  2612 rpcz_store.cc:269] 0709 14:54:37.674736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:37.674981 (+   245us) service_pool.cc:224] Handling call
0709 14:54:40.322646 (+2647665us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:40.322797 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5414494464}
W20260709 14:54:40.325690  9628 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.001s
I20260709 14:54:40.325977  9734 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:40.326295  9628 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:40.326815  9636 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:40.321341  9850 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:40.329358  9850 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:40.329767  9850 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:40.330140  9850 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:40.330298  9554 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.309072  2684 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:54:40.271245  9849 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.332095  9849 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.332788  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.333668  2684 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.320236  2755 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:40.335762  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.336383  2755 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:40.336671  2677 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:40.337335  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.342657  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.343488  2678 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:40.345992  2677 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:40.346233  2759 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:40.352137  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.348956  2684 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:40.338570  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.308672  2754 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:40.356431  2759 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:40.363399  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.370096  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.370404  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.355605  2755 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:40.371582  2759 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:40.334482  3547 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.373014  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.373517  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.374505  2757 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 14:54:40.302253  9250 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.375133  2760 raft_consensus.cc:2368] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.373559  3547 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:40.374518  2678 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:40.373114  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:40.383764  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.334560  9849 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.326685  9734 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.405843  2683 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:40.406513  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.381538  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.389254  9563 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:40.390749  9847 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 14:54:40.391713  9857 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:40.335765  9855 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:40.411010  9857 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:40.411186  9855 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:40.411525  9857 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:40.318742  9854 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.412627  9843 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:40.414131  9854 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.399976  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.416011  2678 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.404220  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.421206  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.424069  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.432550  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.434495  2678 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:40.435844  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.435986  9446 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.436429  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1507) took 1003 ms. Trace:
I20260709 14:54:40.427738  9563 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:40.436862  3538 rpcz_store.cc:276] 0709 14:54:39.432883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.439455 (+  6572us) service_pool.cc:224] Handling call
0709 14:54:40.436196 (+996741us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400550ad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:40.436366 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457577344}
I20260709 14:54:40.443194  9561 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:40.407891  3547 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 14:54:40.409060  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.371100  2680 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:54:40.411602  9855 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:40.448304  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.449944  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.451002  2680 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.451735  9848 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:40.452448  2680 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:40.452589  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.453815  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.454457  2759 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:40.454017  2680 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:40.457542  9355 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:40.458531  9810 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.539s	user 0.003s	sys 0.011s
I20260709 14:54:40.482853  9810 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:40.491973  9745 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.494236  9745 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:40.495646  9331 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:40.496027  9060 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:40.496707  9060 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:40.496881  9331 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.498060  9060 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:40.483115  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 60336.1us
I20260709 14:54:40.423056  9734 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.463449  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:40.464184  9833 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.641s	user 0.005s	sys 0.000s
I20260709 14:54:40.457581  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.499977  9538 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 14:54:40.499972  9060 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:40.538602  2752 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:40.542577  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.543617  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.469856  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.500567  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1314) took 1031 ms. Trace:
I20260709 14:54:40.450270  9554 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:40.503013  9547 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.503325  2652 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.505162  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1817) took 1114 ms. Trace:
I20260709 14:54:40.483376  2754 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:40.508272  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.551975  2760 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:40.544720  2610 rpcz_store.cc:276] 0709 14:54:39.468988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.472926 (+  3938us) service_pool.cc:224] Handling call
0709 14:54:39.473276 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.473534 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:39.728865 (+255331us) log.cc:844] Serialized 14 byte log entry
0709 14:54:40.500160 (+771295us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:40.500225 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:40.500292 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.500368 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:40.500504 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580912,"prepare.queue_time_us":429936,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":61,"raft.queue_time_us":581867,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":444843,"threads_started":2,"wal-append.queue_time_us":360207}
I20260709 14:54:40.558089  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.509475  3547 pending_rounds.cc:85] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 8
I20260709 14:54:40.509867  3411 leader_election.cc:400] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.567272  3411 leader_election.cc:403] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 14:54:40.417389  9780 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.485394  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1302) took 1087 ms. Trace:
I20260709 14:54:40.572912  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.513201  9776 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.489835  9494 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:40.499660  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 595366us
I20260709 14:54:40.499964  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.514469  9628 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:40.515955  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.521751  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.591359  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.483853  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1505) took 1051 ms. Trace:
I20260709 14:54:40.592103  2752 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:40.592732  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.593786  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:40.592090  3541 rpcz_store.cc:276] 0709 14:54:39.432288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.432748 (+   460us) service_pool.cc:224] Handling call
0709 14:54:40.483575 (+1050827us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54004b64d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:40.483766 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1683646720}
I20260709 14:54:40.528941  2681 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.590654  2757 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:40.516984  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:40.535367  2651 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.543803  3543 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:40.502236  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1315) took 1032 ms. Trace:
I20260709 14:54:40.545626  2759 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:40.545320  3538 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:40.546866  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:40.550413  2758 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:40.568370  3547 pending_rounds.cc:107] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 14:54:40.607329  3547 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:40.572921  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.609563  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1856) took 1019 ms. Trace:
I20260709 14:54:40.551446  2605 rpcz_store.cc:276] 0709 14:54:39.390587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.390735 (+   148us) service_pool.cc:224] Handling call
0709 14:54:39.391227 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:39.391706 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:39.391742 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:39.392375 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 14:54:39.730016 (+337641us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:40.075163 (+345147us) log.cc:844] Serialized 14 byte log entry
0709 14:54:40.504782 (+429619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.504888 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:40.504928 (+    40us) raft_consensus.cc:1685] finished
0709 14:54:40.504940 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:40.505096 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":436274,"prepare.queue_time_us":246199,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":448805,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":12829,"thread_start_us":637337,"threads_started":2,"wal-append.queue_time_us":546423}
I20260709 14:54:40.585870  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.499392  9375 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.585337  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.574195  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
W20260709 14:54:40.607422  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107592us
I20260709 14:54:40.609153  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:40.588685  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78999.7us
W20260709 14:54:40.635660  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.639389  9250 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:40.640494  9375 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.641847  2610 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:40.639520  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:40.636045  2678 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:40.644533  9250 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:40.645220  9250 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:40.645792  9250 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:40.610232  3547 rpcz_store.cc:276] 0709 14:54:39.589995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.738969 (+148974us) service_pool.cc:224] Handling call
0709 14:54:39.765065 (+ 26096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:40.320709 (+555644us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002894a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5bddc0 00007fa16e6481c0 00007fa16e649231 00007fa16e645d8f 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c
0709 14:54:40.609184 (+288475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.609485 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486428032}
W20260709 14:54:40.657374  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.603113  2603 rpcz_store.cc:276] 0709 14:54:39.469371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.526613 (+ 57242us) service_pool.cc:224] Handling call
0709 14:54:39.527106 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.527450 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:39.749252 (+221802us) log.cc:844] Serialized 14 byte log entry
0709 14:54:40.501852 (+752600us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:40.501915 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:40.501977 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.502048 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:40.502175 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":490466,"prepare.queue_time_us":389541,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":55,"raft.queue_time_us":473649,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":500346,"threads_started":2,"wal-append.queue_time_us":402383}
I20260709 14:54:40.659423  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.573338  2606 rpcz_store.cc:276] 0709 14:54:39.398522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:39.398780 (+   258us) service_pool.cc:224] Handling call
0709 14:54:39.430006 (+ 31226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:39.430288 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:39.619166 (+188878us) log.cc:844] Serialized 14 byte log entry
0709 14:54:39.777161 (+157995us) log.cc:844] Serialized 14 byte log entry
0709 14:54:39.779935 (+  2774us) log.cc:844] Serialized 14 byte log entry
0709 14:54:40.484587 (+704652us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18004fd560. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:54:40.484796 (+   209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:40.484864 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:40.484938 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.485029 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:40.485313 (+   284us) inbound_call.cc:177] Queueing success response
0709 14:54:40.510961 (+ 25648us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18004fd560. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
Metrics: {"mutex_wait_us":705545,"prepare.queue_time_us":891027,"prepare.run_cpu_time_us":227,"prepare.run_wall_time_us":9024,"raft.queue_time_us":707675,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":13600,"spinlock_wait_cycles":62425088,"thread_start_us":311241,"threads_started":2,"wal-append.queue_time_us":288692}
W20260709 14:54:40.660179  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156960us
I20260709 14:54:40.660495  2612 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:54:40.661373  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 126125us
I20260709 14:54:40.661839  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.663154  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:40.665316  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:40.665727  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.666678  2608 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:54:40.667250  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.645394  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:40.670441  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123713us
I20260709 14:54:40.671548  2678 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 18: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.675197  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:40.486187  9857 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.676056  2681 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:40.678906  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.677197  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:40.596735  9755 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:40.690231  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.687464  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:40.631821  9776 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:40.659126  9750 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:40.699867  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 417793us
W20260709 14:54:40.700697  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 191943us
I20260709 14:54:40.576349  9547 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.733323  9734 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:40.733866  9547 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:40.665335  9768 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 14:54:40.702024  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.735147  2752 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:40.735028  9787 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:40.736865  9538 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:40.737336  9538 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:40.739246  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 151369us
I20260709 14:54:40.664520  9561 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:40.741001  9561 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:40.741544  9561 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:40.742038  9337 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:40.675200  9563 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:40.692245  9812 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:40.702205  9852 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:40.710472  9736 consensus_queue.cc:1059] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 14:54:40.710043  8885 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 14:54:40.754683  9563 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:40.770352  9563 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:40.773823  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.608404  9628 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.777386  9628 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:40.778604  9337 raft_consensus.cc:3055] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:40.779179  9337 raft_consensus.cc:740] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:40.780177  9337 consensus_queue.cc:260] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.702080  9843 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:40.724992  9780 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.727319  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:40.728114  9494 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:40.731112  9857 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.731838  3542 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:40.670367  9446 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:40.743659  2576 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:40.734421  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.761787  9812 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:40.750358  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:40.829115  2758 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:40.830125  2753 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 16.
I20260709 14:54:40.695617  9617 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:40.696048  9784 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:40.795917  9843 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:40.721807  9375 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:40.738229  9538 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:40.823163  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 111383us
I20260709 14:54:40.712381  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.703380  9793 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 14:54:40.850914  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:40.803742  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:40.840202  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96667.5us
I20260709 14:54:40.852449  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.854156  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.855263  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.742509  9561 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:40.800571  9383 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:40.844353  9609 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:40.755148  9750 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:40.795719  9858 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:40.883278  2757 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 14:54:40.883540  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.845180  2686 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.846520  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.846477  9617 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:40.886488  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.887758  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:40.888553  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.850246  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.856698  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.825085  9446 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:40.728157  2681 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:40.856019  2605 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:40.900071  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.901691  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.831921  9812 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:40.781737  9337 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:40.846928  9843 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:40.889222  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.890972  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:40.891004  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.874891  9852 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:40.877033  9637 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:54:40.874711  2723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.063s	sys 0.109s
W20260709 14:54:40.928309  2723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.064s	sys 0.109s
I20260709 14:54:40.903439  9446 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:40.902805  2604 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:40.875156  9733 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:40.916392  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.949420  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:40.957656  2681 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:40.875929  9383 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:40.856487  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52840.7us
I20260709 14:54:40.890336  9784 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:40.881480  9787 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:40.925078  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.925428  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:40.857512  9544 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 14:54:40.929064  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 634577us
I20260709 14:54:40.901677  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:40.908138  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:40.920120  9785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.009s	sys 0.023s
I20260709 14:54:40.936874  2606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5f95289d39644bead88ff8640742871"
dest_uuid: "56620e612f0549e6abf320aa80398e74"
mode: GRACEFUL
new_leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
 from {username='slave'} at 127.0.0.1:43306
I20260709 14:54:40.896339  2758 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:40.915531  9852 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:40.932639  9753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.028s	sys 0.006s
I20260709 14:54:40.911640  9616 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:40.909456  9547 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:40.906111  9858 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:40.930546  9637 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:40.924556  9609 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:40.948414  9816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.019s	sys 0.010s
I20260709 14:54:40.973203  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:40.993772  2606 raft_consensus.cc:606] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Received request to transfer leadership to TS 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:41.013986  2606 raft_consensus.cc:614] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 14:54:40.979422  3543 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
W20260709 14:54:40.981673  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:40.982402  2680 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:41.015709  3543 pending_rounds.cc:85] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 12
I20260709 14:54:41.016275  3543 pending_rounds.cc:107] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 14:54:41.016624  3543 raft_consensus.cc:2889] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:41.016978  2680 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:40.987660  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.019744  2681 raft_consensus.cc:2393] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:40.982589  2723 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:40.931624  9781 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 14:54:41.020293  3409 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.045s
I20260709 14:54:41.014137  3538 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:40.980207  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62847us
I20260709 14:54:40.992687  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:41.022409  9816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.020s	sys 0.010s
W20260709 14:54:40.919885  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.012s	sys 0.015s
W20260709 14:54:40.998584  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.033s	sys 0.068s
W20260709 14:54:41.153867  2685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.034s
I20260709 14:54:41.023535  2680 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 14:54:41.156420  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.013s	sys 0.015s
W20260709 14:54:41.013206  2684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.013s	sys 0.006s
I20260709 14:54:41.029429  2608 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: [17.8-18.9]
I20260709 14:54:41.033357  2610 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 14:54:41.011525  9850 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.005s
W20260709 14:54:40.993160  9785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.010s	sys 0.024s
W20260709 14:54:41.044972  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62543.4us
I20260709 14:54:41.163441  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:41.046209  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:41.003527  9753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.029s	sys 0.007s
W20260709 14:54:41.046801  3409 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.046s
W20260709 14:54:41.171233  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 670914us
W20260709 14:54:41.173444  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:40.961958  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.011s	sys 0.019s
W20260709 14:54:41.174583  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.011s	sys 0.020s
W20260709 14:54:41.175066  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 833677us
I20260709 14:54:41.176079  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:41.045576  9761 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:41.008862  9547 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:40.992063  9793 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:41.044324   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 14:54:41.026846  9637 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:41.003038  9616 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:40.997809  8885 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:54:40.999547  9852 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:41.030494  9355 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:40.967855  9733 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:41.054679  9863 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:40.976330  9617 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:41.079355  9864 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 14:54:41.081475  9060 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.002s
I20260709 14:54:41.029666  9383 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:41.084594  9586 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:40.991604  9787 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:41.104804  9609 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:41.106313  3547 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 14:54:41.109946  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:41.113565  9538 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:41.113783  2612 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:41.115537  9858 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:41.128829  9855 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.000s
W20260709 14:54:41.129392  9861 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 14:54:41.129806  9750 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:41.131040  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:41.103279  9563 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:41.090075  9843 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:41.154222  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.033s	sys 0.068s
I20260709 14:54:41.026094  2681 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:41.026576  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:41.157393  2684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.014s	sys 0.006s
I20260709 14:54:41.158455  2683 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
W20260709 14:54:41.158675  2685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.003s	sys 0.034s
W20260709 14:54:41.165259  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 119137us
I20260709 14:54:41.334482  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1785) took 1076 ms. Trace:
W20260709 14:54:41.177699  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:41.335083  2685 rpcz_store.cc:276] 0709 14:54:40.258385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.258540 (+   155us) service_pool.cc:224] Handling call
0709 14:54:40.259061 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:40.259360 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:40.597910 (+338550us) log.cc:844] Serialized 14 byte log entry
0709 14:54:41.334091 (+736181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:41.334154 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:41.334207 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.334281 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:41.334411 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":708595,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"thread_start_us":1045970,"threads_started":2}
I20260709 14:54:41.179643  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:41.189596  3542 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:41.184052   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:54:41.201193  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:41.335556  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:41.208109  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 462564us
I20260709 14:54:41.266237  9586 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.191314  9734 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:41.190761  9793 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.191084  9547 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:41.181010  9761 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:41.257045  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.038s	sys 0.032s
W20260709 14:54:41.354704  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.038s	sys 0.032s
I20260709 14:54:41.200109  9616 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:41.195938  9810 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:41.268952  2609 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:41.207599  9355 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:41.181969  9850 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:41.224808  9863 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:41.191246  9868 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 14:54:41.202644  9544 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
W20260709 14:54:41.309976  9331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.024s	sys 0.063s
I20260709 14:54:41.181665  9661 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:41.288767  9857 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:41.340600  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 161072us
W20260709 14:54:41.341935  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 392649us
I20260709 14:54:41.369984  9355 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:41.344810  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 143786us
I20260709 14:54:41.417459  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:41.348574  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:41.351356  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.010s	sys 0.009s
W20260709 14:54:41.425595  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.010s	sys 0.010s
W20260709 14:54:41.361363  2650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.036s	sys 0.076s
I20260709 14:54:41.362469  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:41.367558  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1333) took 1351 ms. Trace:
I20260709 14:54:41.427034  2756 rpcz_store.cc:276] 0709 14:54:40.016386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.019610 (+  3224us) service_pool.cc:224] Handling call
0709 14:54:40.020005 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.232918 (+212913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.232959 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.242256 (+  9297us) log.cc:844] Serialized 26 byte log entry
0709 14:54:40.540765 (+298509us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:40.547754 (+  6989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.548170 (+   416us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.367153 (+818983us) raft_consensus.cc:1685] finished
0709 14:54:41.367188 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:41.367405 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":161495,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":476312,"threads_started":2,"wal-append.queue_time_us":700778}
I20260709 14:54:41.339105  9743 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:41.337929  9849 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:41.379791  3546 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:41.353358  9586 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:41.351840  9768 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:41.354161  9793 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:41.398334  9850 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:41.315948  9855 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:41.369284  9795 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.035s
I20260709 14:54:41.358460  9760 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:41.411665  2651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.018s	sys 0.032s
W20260709 14:54:41.343883  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:41.470014  2651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.018s	sys 0.032s
I20260709 14:54:41.415472  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:41.418145  9870 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 14:54:41.417454  9857 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:41.391615  9331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.033s	sys 0.064s
I20260709 14:54:41.425024  2752 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:41.329181  9383 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:41.436507  2650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.036s	sys 0.076s
I20260709 14:54:41.430498  2756 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 14:54:41.490922  2758 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:41.377839  9863 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:41.401664  9661 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.414254  9810 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:41.450317  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 750591us
W20260709 14:54:41.437134  9250 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.791s	user 0.015s	sys 0.003s
W20260709 14:54:41.470916  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 580148us
W20260709 14:54:41.468314  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 277109us
W20260709 14:54:41.483299  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 67978.2us
I20260709 14:54:41.482620  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:41.507131  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:41.455538  9795 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.012s	sys 0.036s
W20260709 14:54:41.492816  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 803483us
I20260709 14:54:41.512974  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:41.438726  9781 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 14:54:41.517203  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:41.507197  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:41.509552  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:41.520953  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1320) took 1139 ms. Trace:
I20260709 14:54:41.487324  9850 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:41.504226  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:41.488377  9744 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:41.490547  9768 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:41.524354  2677 rpcz_store.cc:276] 0709 14:54:40.381840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.382032 (+   192us) service_pool.cc:224] Handling call
0709 14:54:40.391319 (+  9287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.406052 (+ 14733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.406108 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.406870 (+   762us) log.cc:844] Serialized 26 byte log entry
0709 14:54:40.906868 (+499998us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:40.906940 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.907055 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.520629 (+613574us) raft_consensus.cc:1685] finished
0709 14:54:41.520657 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:41.520875 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14139,"raft.queue_time_us":14379,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":17372,"thread_start_us":496145,"threads_started":1,"wal-append.queue_time_us":751825}
I20260709 14:54:41.502800  9661 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:41.502578  9544 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:41.504475  9750 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:41.529178  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.539835  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.508469  9733 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:41.508844  9810 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:41.450184  9761 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:41.522361  9887 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:41.531420  8885 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:41.567641  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.524245  9755 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:41.575929  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.576362  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.534763  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:41.548395  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:41.578487  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.514899  9561 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:41.544329  9785 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:41.521669  9060 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 14:54:41.515128  9250 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:41.580394  9848 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:41.569617  9787 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:41.585023  9544 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:41.546993  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:41.613428  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 274787us
I20260709 14:54:41.579879  9810 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:41.575085  9885 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:41.608574  9561 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:41.586274  9733 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 14:54:41.610227  2680 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:41.579356  9895 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:41.539865  9850 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:41.667675  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:41.628167  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 292682us
I20260709 14:54:41.545111  9768 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:41.651400  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:41.616639  9784 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 14:54:41.604667  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56426us
I20260709 14:54:41.610148  9563 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:41.620082  9761 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:41.539979  9744 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:41.614531  9785 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.670527  2575 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:41.592741  9887 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:41.653496  9812 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.738s	user 0.015s	sys 0.000s
I20260709 14:54:41.607592  8885 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:41.672211  9893 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 14:54:41.694981  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 183591us
W20260709 14:54:41.674202  9876 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
I20260709 14:54:41.667728  9561 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:41.701119  9563 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 14:54:41.673650  9885 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:41.710037  9887 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:41.675108  9895 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.716053  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:41.639045  9781 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 14:54:41.729945  9895 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:41.680766  9784 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.731235  9784 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:54:41.714433  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 207332us
I20260709 14:54:41.658306  9755 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 14:54:41.705214  9785 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:41.739482  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:41.741129  2677 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:41.822299  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:41.821337  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 155831us
I20260709 14:54:41.825179  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:41.826306  2611 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:41.827281  9860 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.782s	user 0.001s	sys 0.001s
I20260709 14:54:41.827661  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:41.829119  9874 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.579s	user 0.000s	sys 0.002s
I20260709 14:54:41.741595  9855 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:41.742070  9060 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:41.845074  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1320) took 1311 ms. Trace:
W20260709 14:54:41.847559  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 312848us
I20260709 14:54:41.769209  9795 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:41.769320  9857 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:41.677683  9896 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 14:54:41.852886  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:41.863719  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:41.864038  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:41.849639  2605 rpcz_store.cc:276] 0709 14:54:40.534091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.983350 (+449259us) service_pool.cc:224] Handling call
0709 14:54:40.984257 (+   907us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:40.987773 (+  3516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.987826 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:40.988638 (+   812us) log.cc:844] Serialized 52 byte log entry
0709 14:54:41.436552 (+447914us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:41.436606 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.436693 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.844667 (+407974us) raft_consensus.cc:1685] finished
0709 14:54:41.844700 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:41.844977 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3340,"raft.queue_time_us":4138,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":5941,"thread_start_us":407279,"threads_started":1,"wal-append.queue_time_us":608319}
I20260709 14:54:41.865600  2652 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:41.713678  9812 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:41.881028  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:41.884111  2760 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:54:41.889760  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:41.895634  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:41.896544  2756 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:41.898157  2756 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:41.890738  2760 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:41.893817  9563 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:41.906091  9563 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:41.906708  9563 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:41.903587  2756 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:41.853890  9750 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:41.912771  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:41.855463  9060 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:41.914348  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 243765us
W20260709 14:54:41.879220  9880 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.336s	user 0.002s	sys 0.000s
W20260709 14:54:41.893293  9881 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.377s	user 0.000s	sys 0.002s
I20260709 14:54:41.841390  9538 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 14:54:41.919832  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:41.931475  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:41.943677  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 204320us
W20260709 14:54:41.948948  9886 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
W20260709 14:54:41.953994  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:41.775972  9544 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.027261  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1319) took 1653 ms. Trace:
I20260709 14:54:42.058954  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1335) took 1703 ms. Trace:
I20260709 14:54:42.028985  2684 rpcz_store.cc:276] 0709 14:54:40.374176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.374329 (+   153us) service_pool.cc:224] Handling call
0709 14:54:40.374639 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.405187 (+ 30548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.405224 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.427934 (+ 22710us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.333268 (+905334us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:41.333335 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.333445 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.828304 (+494859us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:41.938275 (+109971us) raft_consensus.cc:1685] finished
0709 14:54:41.938303 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.027139 (+ 88836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30010,"raft.queue_time_us":30808,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":16828,"spinlock_wait_cycles":31878400,"thread_start_us":873709,"threads_started":1,"wal-append.queue_time_us":706484}
W20260709 14:54:41.937316  9337 consensus_meta.cc:287] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.025s	user 0.035s	sys 0.002s
I20260709 14:54:41.918550  9060 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:41.876073  9795 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.092221  2760 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:42.059662  2753 rpcz_store.cc:276] 0709 14:54:40.355908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.900928 (+545020us) service_pool.cc:224] Handling call
0709 14:54:40.931266 (+ 30338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.931826 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.931873 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.955063 (+ 23190us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.363222 (+408159us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:41.363281 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.364759 (+  1478us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.947960 (+583201us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:42.001624 (+ 53664us) raft_consensus.cc:1685] finished
0709 14:54:42.001667 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.058814 (+ 57147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1533,"raft.queue_time_us":68525,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7886,"spinlock_wait_cycles":55996800,"thread_start_us":396048,"threads_started":1,"wal-append.queue_time_us":551987}
I20260709 14:54:41.919849  9900 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:42.101859  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:41.939538  9909 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:42.056373  9860 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 14:54:42.114362  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 166541us
I20260709 14:54:42.056331  9905 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:42.132357  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 177083us
I20260709 14:54:42.134817  3542 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
W20260709 14:54:42.151197  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 327511us
I20260709 14:54:42.092437  9755 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:42.160269  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2000) took 1640 ms. Trace:
I20260709 14:54:42.152415  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:42.164997  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:42.166162  3547 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:42.163260  2607 rpcz_store.cc:276] 0709 14:54:40.519791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.520772 (+   981us) service_pool.cc:224] Handling call
0709 14:54:40.575482 (+ 54710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.739132 (+163650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.739168 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.739795 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.355289 (+615494us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:41.355359 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.355517 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:42.159885 (+804368us) raft_consensus.cc:1685] finished
0709 14:54:42.159920 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.160161 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163004,"raft.queue_time_us":165801,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":615034,"threads_started":1,"wal-append.queue_time_us":713168}
I20260709 14:54:42.099380  9857 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:42.174794  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:42.180694  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:42.108189  9900 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:42.182170  3541 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
W20260709 14:54:42.203233  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 337732us
W20260709 14:54:42.158306  9891 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.503s	user 0.001s	sys 0.001s
I20260709 14:54:42.203608  3541 pending_rounds.cc:85] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 12
I20260709 14:54:42.216889  3541 pending_rounds.cc:107] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 14:54:42.217375  3541 raft_consensus.cc:2889] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:42.178246  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:42.151824  9901 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:42.153427  9885 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:42.210364  2611 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 14:54:42.230798  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:42.141227  9879 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:54:42.223934  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 120604us
I20260709 14:54:42.231990  2753 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 14:54:42.231047  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78748.6us
I20260709 14:54:42.203341  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1331) took 2187 ms. Trace:
I20260709 14:54:42.231714  3542 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:42.233595  2751 rpcz_store.cc:276] 0709 14:54:40.015738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.018296 (+  2558us) service_pool.cc:224] Handling call
0709 14:54:40.046227 (+ 27931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.243249 (+197022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.243307 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.244179 (+   872us) log.cc:844] Serialized 26 byte log entry
0709 14:54:40.580964 (+336785us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:40.962672 (+381708us) log.cc:844] Serialized 14 byte log entry
0709 14:54:40.975286 (+ 12614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:40.975408 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:41.826396 (+850988us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:42.202850 (+376454us) raft_consensus.cc:1685] finished
0709 14:54:42.202992 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.203232 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12149,"prepare.queue_time_us":90172,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":206231,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":5665,"spinlock_wait_cycles":7077120,"thread_start_us":850124,"threads_started":3,"wal-append.queue_time_us":743798}
I20260709 14:54:42.172942  9896 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:42.179936  9857 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:42.229889  9446 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.262s	user 0.009s	sys 0.017s
I20260709 14:54:42.227402  9744 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:42.216987  9900 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:42.231820  9768 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:42.215818  9850 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:42.152218  9909 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.229643  9885 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:42.261070  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 545087us
I20260709 14:54:42.240219  9897 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:42.281863  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:42.232211  9901 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:42.168653  9750 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.251981  9857 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:42.167589  9905 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:42.250557  8885 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:42.310523  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 161048us
I20260709 14:54:42.288301  9901 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:42.316766  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 83471.7us
W20260709 14:54:42.319084  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85725.4us
I20260709 14:54:42.320880  2577 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:42.321655  2611 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:42.274462  9812 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:42.285908  9885 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:42.252125  9896 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.251682  9904 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 14:54:42.309957  9755 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:42.268527  9446 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.327533  8885 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:42.356951  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:42.352494  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:42.352494  3547 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:42.363142  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:42.340567  9915 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:42.353732  9616 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.976s	user 0.007s	sys 0.008s
I20260709 14:54:42.378434  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:42.381513  2753 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 14:54:42.406873  3547 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:42.337462  9928 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.532559  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:42.448338  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:42.545090  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1323) took 2112 ms. Trace:
W20260709 14:54:42.545931  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 97645.3us
W20260709 14:54:42.417575  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65025.3us
I20260709 14:54:42.354094  9930 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:42.338915  9879 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 14:54:42.548636  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:42.549387  9879 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.549149  9930 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.550014  9879 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.430949  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:42.375613  8885 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:42.381009  9586 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.932s	user 0.017s	sys 0.001s
I20260709 14:54:42.415800  9899 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 14:54:42.445995  9795 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:42.381035  9910 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:42.381405  9637 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.154s	user 0.011s	sys 0.006s
I20260709 14:54:42.331032  9331 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 14:54:42.374650  9929 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:42.380885  9916 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:42.436461  9755 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.484501  9915 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.369678  9934 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 14:54:42.504207  3542 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 14:54:42.513970  9563 consensus_meta.cc:287] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.002s
I20260709 14:54:42.430651  9616 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.519935  9905 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:42.522365  9561 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.800s	user 0.014s	sys 0.000s
W20260709 14:54:42.523985  9852 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.276s	user 0.002s	sys 0.012s
W20260709 14:54:42.524478  9355 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.082s	user 0.010s	sys 0.006s
W20260709 14:54:42.527753  9761 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.828s	user 0.014s	sys 0.001s
I20260709 14:54:42.542306  9928 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.450896  9850 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.355156  9926 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:42.545818  2683 rpcz_store.cc:276] 0709 14:54:40.432541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.446180 (+ 13639us) service_pool.cc:224] Handling call
0709 14:54:41.158246 (+712066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:41.333798 (+175552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:42.544738 (+1210940us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b540024efd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:42.544996 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632526080}
I20260709 14:54:42.550724  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:42.497892  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:42.552650  2757 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:42.554076  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:42.563055  9944 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:42.595281  3547 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:42.619443  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1336) took 2263 ms. Trace:
I20260709 14:54:42.619720  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2009) took 2086 ms. Trace:
W20260709 14:54:42.578626  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 296811us
I20260709 14:54:42.676126  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:42.609341  9922 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.159s	user 0.000s	sys 0.003s
I20260709 14:54:42.555572  9930 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:42.577648  9916 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:42.559718  9923 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:42.611893  9734 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.256s	user 0.017s	sys 0.000s
I20260709 14:54:42.562959  9913 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:42.564327  9938 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:42.677157  9944 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:42.554216  8885 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:42.614303  9899 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:42.598007  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 68081.5us
I20260709 14:54:42.747287  9899 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:42.747782  9919 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 14:54:42.613883  9547 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.244s	user 0.010s	sys 0.006s
I20260709 14:54:42.659436  2605 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:42.677053  2604 rpcz_store.cc:276] 0709 14:54:40.533341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.947495 (+414154us) service_pool.cc:224] Handling call
0709 14:54:40.948016 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:41.326921 (+378905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:41.326973 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:41.327040 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:41.327100 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:42.619440 (+1292340us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54002ff8d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:42.619500 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.619656 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870550784}
W20260709 14:54:42.564668  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:42.674571  2758 rpcz_store.cc:276] 0709 14:54:40.356189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.999643 (+643454us) service_pool.cc:224] Handling call
0709 14:54:41.000183 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:41.491502 (+491319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:42.619097 (+1127595us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:42.619367 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127770368}
W20260709 14:54:42.684067  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 519684us
W20260709 14:54:42.687443  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 136833us
W20260709 14:54:42.566165  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 251791us
I20260709 14:54:42.752468  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:42.548623  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:42.688417  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:42.590998  9637 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.753868  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:42.590719  9929 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.754814  2760 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:42.698690  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1646) took 1376 ms. Trace:
I20260709 14:54:42.665024  9858 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.700392  9909 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:42.591092  9910 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:42.683964  9383 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.185s	user 0.006s	sys 0.010s
I20260709 14:54:42.619968  9561 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.664580  9544 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:42.699326  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.002s	sys 0.006s
I20260709 14:54:42.623512  9355 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:42.736541  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:42.553830  9879 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:42.696007  9917 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.368s	user 0.001s	sys 0.001s
I20260709 14:54:42.690665  9904 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:42.740810  9446 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:42.652947  9850 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:42.623050  9852 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:42.745662  9661 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.175s	user 0.012s	sys 0.013s
W20260709 14:54:42.746050  9784 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.014s	user 0.005s	sys 0.016s
I20260709 14:54:42.747900  9899 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:42.910393  9949 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 14:54:42.755411  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 67155.1us
W20260709 14:54:42.555999  9883 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.012s	user 0.011s	sys 0.000s
I20260709 14:54:42.758481  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:42.759281  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 210955us
I20260709 14:54:42.930437  2753 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:42.614675  9563 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.700054  9916 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:42.731948  9734 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:42.554620  9586 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.755796  3538 rpcz_store.cc:276] 0709 14:54:41.322404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.323544 (+  1140us) service_pool.cc:224] Handling call
0709 14:54:41.324035 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:41.324376 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:41.624683 (+300307us) log.cc:844] Serialized 14 byte log entry
0709 14:54:42.100555 (+475872us) log.cc:844] Serialized 14 byte log entry
0709 14:54:42.560960 (+460405us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b180047b180. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa17251a58f 00007fa16b559571 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd
0709 14:54:42.681238 (+120278us) log.cc:844] Serialized 14 byte log entry
0709 14:54:42.681591 (+   353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:42.681673 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:42.681781 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:42.681897 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:42.698589 (+ 16692us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207812,"prepare.queue_time_us":433215,"prepare.run_cpu_time_us":188,"prepare.run_wall_time_us":19533,"raft.queue_time_us":499050,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":16444,"spinlock_wait_cycles":81240960,"thread_start_us":1117352,"threads_started":4,"wal-append.queue_time_us":963426}
W20260709 14:54:42.740921  9793 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.284s	user 0.024s	sys 0.010s
W20260709 14:54:42.771293  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.007s	sys 0.007s
W20260709 14:54:42.939333  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.01956e+06us
W20260709 14:54:42.895502  9947 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 14:54:42.921953  9919 log.cc:927] Time spent T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 14:54:42.788113  3541 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:42.765379  8885 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:42.783505  3546 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 14:54:42.749676  9944 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:42.859653  9936 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.658177  9933 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.144s
I20260709 14:54:42.683597  9924 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 14:54:42.900043  2757 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:42.896625  9850 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:42.798130  9896 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:42.894697  9901 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.002s
I20260709 14:54:42.771195  9930 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 14:54:42.492354   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:54:42.908269  9948 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 14:54:42.910070  9784 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.912971  9928 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:42.733613  9617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.477s	user 0.029s	sys 0.028s
W20260709 14:54:43.006783  9617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.751s	user 0.029s	sys 0.029s
I20260709 14:54:43.008225  9961 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.009109  9250 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.008962  9961 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.010262  9784 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:42.772132  9951 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:43.011078  9901 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:42.920674  9810 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.245s	user 0.016s	sys 0.000s
I20260709 14:54:43.012065  9935 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.012332  9810 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.012746  9935 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.013698  9901 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:42.765033  9854 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:43.015128  9854 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.017412  9854 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:42.927685  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:43.018896  3543 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:42.563234  9920 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.216s
I20260709 14:54:43.019527  2611 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 14:54:43.021247  9750 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 14:54:42.684167  9931 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 14:54:42.935042  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.023661  2753 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:43.023923  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1760) took 1490 ms. Trace:
I20260709 14:54:42.935740  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.025128  2758 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:43.024401  2677 rpcz_store.cc:276] 0709 14:54:41.533902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.534076 (+   174us) service_pool.cc:224] Handling call
0709 14:54:43.023721 (+1489645us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:43.023854 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346448768}
I20260709 14:54:42.938417  2759 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:42.771677  3545 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
W20260709 14:54:42.894153  9863 consensus_meta.cc:287] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.384s	user 0.012s	sys 0.002s
W20260709 14:54:42.943059  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 145290us
I20260709 14:54:42.944571  2685 raft_consensus.cc:2393] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:42.944819  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:43.030512  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:43.031351  3547 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:42.946101  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:42.748540  9547 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:42.702939  9923 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:42.890755  9929 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:42.765272  9858 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:42.937441  9937 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.064040  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.065289  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:42.970327  2686 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:43.036015  9785 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.258s	user 0.016s	sys 0.000s
I20260709 14:54:42.968842  9538 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:42.968480  9956 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.068809  2606 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
I20260709 14:54:42.942418  9849 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 14:54:42.979537  9925 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 14:54:42.946504  9793 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:42.982905  9936 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:42.984375  9933 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:42.988116  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:42.963037  9909 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:42.935961  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:43.084846  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 96875.9us
I20260709 14:54:43.060264  9760 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 14:54:43.034811  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 88869.3us
I20260709 14:54:43.048755  9938 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:43.039206  9563 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.041075  9915 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.043164  9586 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:43.006117  9867 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.642s	user 0.011s	sys 0.018s
I20260709 14:54:42.999675  8885 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:43.056666  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:43.057376  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:43.103204  9871 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 2.002s	user 0.004s	sys 0.004s
I20260709 14:54:43.104032  8885 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.105480  9561 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.076262  9956 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.060307  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:43.107142  3409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.597s (SENT) [suppressed 24 similar messages]
I20260709 14:54:43.107643  2758 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:43.108469  9936 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.108831  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.109067  9956 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.109479  2758 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.048631  9905 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:43.112579  9905 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.112694  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1779) took 2858 ms. Trace:
I20260709 14:54:43.113045  9905 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.113168  2682 rpcz_store.cc:276] 0709 14:54:40.254171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.266485 (+ 12314us) service_pool.cc:224] Handling call
0709 14:54:40.266961 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.267449 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.267480 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.268072 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.331488 (+1063416us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:41.331567 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.331727 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:42.983193 (+1651466us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:43.110063 (+126870us) raft_consensus.cc:1685] finished
0709 14:54:43.110089 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:43.112616 (+  2527us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":16182,"spinlock_wait_cycles":66144000,"thread_start_us":992204,"threads_started":1,"wal-append.queue_time_us":963274}
I20260709 14:54:43.102231  2756 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
W20260709 14:54:43.115103  9892 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.571s	user 0.000s	sys 0.012s
I20260709 14:54:43.115895  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.116664  2756 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:43.097644  9755 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 14:54:43.036041  9810 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.075292  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.120424  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.084347  9933 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:43.125528  9962 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:43.090342  9909 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.127068  2752 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:43.127318  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1816) took 2831 ms. Trace:
I20260709 14:54:43.090760  9879 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:43.082056  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.085683  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:42.931344  9661 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.074415  9785 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.037336  9904 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:43.077838  9885 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.729s	user 0.014s	sys 0.000s
I20260709 14:54:43.093870  2681 raft_consensus.cc:2368] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:43.152359  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:43.047199  9863 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.154377  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.156332  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:43.078153  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.027s
W20260709 14:54:43.081559  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 137718us
I20260709 14:54:43.194702  2758 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:43.131490  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.132834  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.134020  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.133831  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.087301  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:43.072310  9937 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.076000  9538 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:43.201987  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 114818us
I20260709 14:54:43.140628  2757 raft_consensus.cc:2376] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21. Re-sending same reply.
I20260709 14:54:43.127957  2755 rpcz_store.cc:276] 0709 14:54:40.295989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.853767 (+557778us) service_pool.cc:224] Handling call
0709 14:54:40.854165 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.897541 (+ 43376us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:40.913102 (+ 15561us) log.cc:844] Serialized 14 byte log entry
0709 14:54:41.099815 (+186713us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800361000. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:41.288999 (+189184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:41.289751 (+   752us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.289971 (+   220us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:41.308013 (+ 18042us) log.cc:844] Serialized 14 byte log entry
0709 14:54:41.618618 (+310605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.618720 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:42.984823 (+1366103us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:43.118759 (+133936us) raft_consensus.cc:1685] finished
0709 14:54:43.118792 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:43.127197 (+  8405us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5694,"prepare.queue_time_us":37578,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":143,"raft.queue_time_us":191550,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":27475,"spinlock_wait_cycles":15062528,"thread_start_us":498816,"threads_started":2,"wal-append.queue_time_us":554738}
W20260709 14:54:43.111740  9857 consensus_meta.cc:287] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.788s	user 0.017s	sys 0.000s
W20260709 14:54:43.207125  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:43.207331  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:43.171157  9885 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.172451  9921 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.174471  9661 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:43.196914  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.010s	sys 0.028s
I20260709 14:54:43.153858  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:43.178225  9944 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.187929  2752 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:54:43.196782  2612 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:43.101181  9935 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.140668  2760 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:43.148998  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.144006  9761 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:43.150339  9907 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.231s	user 0.000s	sys 0.016s
I20260709 14:54:43.212528  2756 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:43.231316  2611 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 14:54:43.236672  2756 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 14:54:43.235201  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81416.1us
W20260709 14:54:43.235503  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 128409us
I20260709 14:54:43.207440  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.245461  2756 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 14:54:43.246387  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:43.174796  9899 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:43.224962  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 91206.4us
I20260709 14:54:43.196382  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.159754  9879 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:43.127951  9926 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 14:54:43.204339  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:43.241566  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:43.247849  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.252655  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.144219  9903 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 14:54:43.239914  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:43.255569  2756 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:43.215395  9852 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.233479  9355 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.215448  9561 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:43.205552  9910 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.257880  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.276387  2682 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.258427  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.254552  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:43.259894  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.277962  2684 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:43.279362  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.279975  2684 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:43.242952  9736 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.330s	user 0.044s	sys 0.075s
I20260709 14:54:43.281289  2753 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:43.261090  2757 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:43.262661  2680 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 14:54:43.264117  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59856.8us
I20260709 14:54:43.285519  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.236763  9962 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:43.231033  9915 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.235294  9563 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.277197  2756 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:43.222118  9857 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.286425  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:43.241101  9931 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:43.246430  9661 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:43.289270  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.263667  9868 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.292523  9793 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.292768  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.274600  9908 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:43.293429  9547 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.257210  9929 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:43.299392  9966 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 14:54:43.300741  9785 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.260596  9760 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:43.305507  2684 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:43.407773  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 153312us
I20260709 14:54:43.408169  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.244521  9761 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.286999  9250 consensus_queue.cc:1059] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 14:54:43.308528  9953 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:43.310993  9873 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 2.180s	user 0.007s	sys 0.002s
I20260709 14:54:43.415181  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.416056  2756 raft_consensus.cc:3060] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.299504  9538 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.313539  9937 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.316080  9899 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.327287  9787 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:43.331107  9962 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.429992  9962 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.357620  9563 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.433599  9563 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.252732  9938 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.435089  9547 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:43.435739  9547 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:43.436831  9547 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.408957  9879 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:43.347427  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61136us
I20260709 14:54:43.352317  9915 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.439013  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:43.263377  9863 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.256268  9934 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.439958  2677 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:43.440433  9934 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.441402  9915 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.228225  9933 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.442220  9563 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 14:54:43.289240  9736 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.044s	sys 0.076s
I20260709 14:54:43.442706  9933 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.228874  9921 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:43.340719  9878 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.449s	user 0.015s	sys 0.042s
I20260709 14:54:43.271497  9561 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.286532  9586 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:43.390681  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.291004  9910 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:43.354234  9060 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.240s	user 0.015s	sys 0.003s
I20260709 14:54:43.407378  9760 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.351645  9755 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.408964  2684 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:43.464632  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.465651  2684 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:43.411000  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.468345  2678 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:43.469897  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.450258  9561 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.355607  9852 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:43.355693  9661 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.408847  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:43.394685  9908 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.400367  9929 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.407015  8885 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:43.389132  9931 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.423359  9953 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:43.428397  9250 raft_consensus.cc:3055] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 14:54:43.339897  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60018.5us
I20260709 14:54:43.504621  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.443089  9563 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:43.407554  9900 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.118s	user 0.008s	sys 0.011s
I20260709 14:54:43.397322  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:43.514846  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 117612us
I20260709 14:54:43.515563  3409 leader_election.cc:400] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:43.512091  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.518500  9920 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:43.519297  9920 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.504938  9250 raft_consensus.cc:740] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:43.519768  9920 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.520948  9761 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.520673  9250 consensus_queue.cc:260] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.510860  9793 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 14:54:43.473227  9760 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:43.504309  9944 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:43.449988  9878 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.622s	user 0.016s	sys 0.043s
I20260709 14:54:43.474987  9586 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:43.453665  9858 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.475816  9755 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.476908  9905 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.477473  9885 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.478533  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.504779  9953 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.480086  9934 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.489383  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.470742  2678 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:54:43.488610  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 188415us
I20260709 14:54:43.504333  9852 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:43.502031  2756 raft_consensus.cc:2468] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:43.560832  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.561846  2756 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 20
I20260709 14:54:43.521927  9250 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.506035  2612 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 14:54:43.426955  9895 consensus_meta.cc:287] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.696s	user 0.016s	sys 0.003s
I20260709 14:54:43.517644  2611 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:43.516721  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.494033  9661 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:43.494906  9929 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.472460  9060 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.492733  9908 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:43.472010  9910 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:43.496505  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 87695.9us
I20260709 14:54:43.523016  9563 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:43.617174  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 100558us
I20260709 14:54:43.618005  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.618542  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:43.551704  9951 consensus_meta.cc:287] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.002s
I20260709 14:54:43.588989  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.651175  9951 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:43.651350  2757 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:43.652700  9755 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.588989  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.656047  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.566015  9736 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:43.657485  9736 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:43.609570  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1328) took 1684 ms. Trace:
I20260709 14:54:43.659369  2605 rpcz_store.cc:276] 0709 14:54:41.925420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.925584 (+   164us) service_pool.cc:224] Handling call
0709 14:54:41.925986 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:42.749187 (+823201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:43.609289 (+860102us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:43.609502 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661060992}
I20260709 14:54:43.658672  9736 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.661190  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.661339  9857 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.536497  9900 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.504088  8885 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.663899  9900 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.665079  8885 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.610719  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.612928  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.607555  9921 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:43.627719  9916 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.002s
I20260709 14:54:43.601387  9538 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.636096  9962 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.637259  9060 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.638814  9931 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.639232  9964 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:43.698572  2756 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:43.642016  9905 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.643960  9852 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.644615  9910 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.556857  9976 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:43.662582  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:43.673194  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54824.8us
I20260709 14:54:43.699733  2678 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:54:43.717144  3409 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:43.717478  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.718395  2678 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:43.639792  9586 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.704877  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.724632  2756 raft_consensus.cc:2393] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:43.676522  9878 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:43.680552  9916 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.725198  9878 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:43.726373  9878 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.727357  9916 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.727651  9878 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:43.729063  9908 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.678319  9755 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:43.730993  9755 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:43.731734  9755 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.732957  9920 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.721155  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.735143  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.735872  2684 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:43.723067  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.735880  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.736994  9760 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.737521  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.738044  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.604815  9895 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.702984  9964 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.730800  9905 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:43.680392  9538 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.738811  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.751410  2678 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.739866  2756 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:43.741492  2611 raft_consensus.cc:2368] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:43.741520  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.762699  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.763461  2684 raft_consensus.cc:2393] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.745395  2608 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:43.768227  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:43.778008  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:43.779173  2757 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:43.779806  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.780339  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.780652  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.780757  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.781481  2756 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:43.781785  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.783105  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:43.748426  9760 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.745061  9868 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:43.784387  9857 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:43.785009  9804 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:43.785506  9804 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:43.739234  9964 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.742511  9895 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.743422  9908 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.766783  9905 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:43.741191  9920 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.753684  9755 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.795425  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.800189  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.801929  2680 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 14:54:43.800499  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.801718  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.805599  2757 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:43.748136  9929 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.747474  9661 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.808220  2753 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.723219  9976 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.815625  2678 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:43.800064  9905 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:43.802260  9868 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:43.808652  9986 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 14:54:43.804979  9060 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.786497  9804 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.821913  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.823777  2605 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:43.823832  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:43.827531  2757 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.792336  9980 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 14:54:43.814184  9250 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:43.820638  9905 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.833184  9921 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:43.828588  9060 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:43.860451  9953 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.861809  9060 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:43.862983  9987 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 14:54:43.864181  9982 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 14:54:43.867177  9250 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.837174  2758 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:43.824565  9868 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:43.875008  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51250.2us
I20260709 14:54:43.851317  9976 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.820454  9637 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.851372  9962 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.853469  9929 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.855158  9858 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.878000  9858 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.881201  9858 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.856426  9561 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:43.883092  9743 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:43.884022  9743 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:43.885496  9561 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:43.856992  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:43.857391  9804 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.887581  9929 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:43.876516  9868 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.888180  9743 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.890448  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:43.888499  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:43.849426  9755 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.893958  2758 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:43.869637  9964 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:43.896471  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:43.877813  9250 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:43.902413  9868 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:43.903986  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.904954  2679 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:43.906049  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.908631  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.910075  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.912361  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:43.913216  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1762) took 2366 ms (client timeout 1884 ms). Trace:
I20260709 14:54:43.913638  9853 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.913909  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.914386  9853 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:43.915769  2681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1757) took 2386 ms (client timeout 2123 ms). Trace:
W20260709 14:54:43.916424  2681 rpcz_store.cc:269] 0709 14:54:41.529028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.530044 (+  1016us) service_pool.cc:224] Handling call
0709 14:54:43.915434 (+2385390us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540012c854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:43.915667 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345962368}
I20260709 14:54:43.917482  9853 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:43.918010  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.906693  2679 raft_consensus.cc:3060] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:43.920248  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1330) took 1993 ms. Trace:
I20260709 14:54:43.920939  2606 rpcz_store.cc:276] 0709 14:54:41.926881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.927157 (+   276us) service_pool.cc:224] Handling call
0709 14:54:41.927453 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:43.071485 (+1144032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:43.919846 (+848361us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001ead54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:43.920142 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624117120}
I20260709 14:54:43.921797  2677 raft_consensus.cc:2393] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:43.925146  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:43.924140  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.910380  9953 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:43.929898  9961 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:43.932015  9894 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 2.304s	user 0.004s	sys 0.010s
I20260709 14:54:43.930303  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.935137  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.943395  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:43.897238  2758 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.944550  2680 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:43.945295  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.946589  2609 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
W20260709 14:54:43.913722  2686 rpcz_store.cc:269] 0709 14:54:41.546798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.619163 (+ 72365us) service_pool.cc:224] Handling call
0709 14:54:43.912843 (+2293680us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:43.913095 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973986048}
I20260709 14:54:43.948405  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.941223  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:43.950453  2677 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:43.948737  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:43.952584  2609 raft_consensus.cc:2410] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:43.954892  2681 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:43.897132  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:43.960526  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63490.2us
I20260709 14:54:43.961879  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:43.964558  2606 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:43.968319  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:43.971227  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:43.963202  9628 raft_consensus.cc:2764] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 14:54:43.963742  9929 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:43.965513  9852 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:43.975353  9753 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:54:43.975821  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:43.977159  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:43.979938  3546 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 14:54:43.979240  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:43.995858  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.008495  2606 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:43.986052  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1526) took 3020 ms (client timeout 2219 ms). Trace:
I20260709 14:54:44.011255  2758 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:43.979583  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1338) took 3622 ms. Trace:
I20260709 14:54:44.012578  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:43.979204  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.015050  2679 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:43.980960  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:44.004756  2723 leader_election.cc:400] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:44.018507  2723 leader_election.cc:403] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 14:54:44.006268  2609 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:44.012380  2754 rpcz_store.cc:276] 0709 14:54:40.356799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.115004 (+758205us) service_pool.cc:224] Handling call
0709 14:54:41.115455 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:41.115931 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:41.115962 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:41.116609 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.388864 (+272255us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:41.388918 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.388996 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:43.931157 (+2542161us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:43.945870 (+ 14713us) raft_consensus.cc:1685] finished
0709 14:54:43.945910 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:43.979480 (+ 33570us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":125,"spinlock_wait_cycles":29613952,"thread_start_us":271758,"threads_started":1,"wal-append.queue_time_us":424972}
I20260709 14:54:44.016412  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:44.009881  3545 rpcz_store.cc:269] 0709 14:54:40.965966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.966132 (+   166us) service_pool.cc:224] Handling call
0709 14:54:43.985771 (+3019639us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:43.985932 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3974338432}
I20260709 14:54:44.021344  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:44.022675  3541 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 14:54:44.023934  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:44.030807  3412 leader_election.cc:400] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:44.031421  3412 leader_election.cc:403] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 14:54:44.035627  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:44.001294  9994 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 14:54:44.037601  2723 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.964s (SENT)
I20260709 14:54:43.985312  9547 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:43.996327  9768 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 14:54:44.045903  2683 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 14:54:44.037627  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.047190  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.048329  2606 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:44.048640  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:44.020715  9743 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:44.052397  9929 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:44.052268  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.053431  2606 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:44.055029  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:44.037243  9753 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:44.056428  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.056573  2680 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:44.058301  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:44.058537  9561 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:44.058698  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.059118  9561 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:44.059576  9561 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:43.980634  9852 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:44.061576  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.062038  9852 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:44.061407  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.065531  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:44.067484  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.070489  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.071089  9852 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:44.027137  9868 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21.
I20260709 14:54:44.072099  9935 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:44.072378  9768 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:44.058307  9905 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.072726  9935 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.073184  9768 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.073722  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.074379  2612 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:44.075239  9935 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.076767  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.077333  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.037209  9961 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:44.079655  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:44.080782  2678 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 14:54:44.081768  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:44.082568  2678 raft_consensus.cc:2410] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 14:54:44.052287  9743 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.085088  9924 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:44.085637  9924 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.085816  9743 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:44.044098  9812 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:44.000804  9964 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:44.076870  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:43.996932  9858 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:44.087793  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.020351  9744 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:44.088279  2680 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:44.089396  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.049036  9942 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:44.092588  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:44.098279  2678 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:44.090795  2612 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:44.100064  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.093645  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.100358  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.103627  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.104323  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.089891  9909 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.071511  9561 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.108359  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.083319  9905 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:44.090787  9858 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:44.096601  9744 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:44.119652  9744 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:44.110531  2612 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:44.105175  9768 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:44.115496  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.123857  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.124374  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:44.116030  9888 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 2.335s	user 0.005s	sys 0.000s
I20260709 14:54:44.124742  2612 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:44.119004  9858 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:44.128762  3545 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:44.106141  9964 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:44.127175  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.125262  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.092286  9812 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.129738  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:44.132433  2612 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:44.131196  2680 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:54:44.133863  2678 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:44.130703  9743 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:44.135783  9743 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:44.135735  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.136466  9743 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:44.138911  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.144378  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.145783  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:44.151993  2612 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 14:54:44.152803  2612 pending_rounds.cc:85] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 8
I20260709 14:54:44.153108  2612 pending_rounds.cc:107] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:54:44.153522  2612 raft_consensus.cc:2889] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:44.156139  3545 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:44.144378  9964 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.150597  9935 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:44.181982  3536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1654) took 1741 ms (client timeout 1648 ms). Trace:
W20260709 14:54:44.183322  3536 rpcz_store.cc:269] 0709 14:54:42.441113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:42.497539 (+ 56426us) service_pool.cc:224] Handling call
0709 14:54:44.181634 (+1684095us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540009abd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:44.181863 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026577280}
I20260709 14:54:44.171443  9905 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:44.172232  9561 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.175247  2609 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:44.175580  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1528) took 3203 ms. Trace:
I20260709 14:54:44.187104  3540 rpcz_store.cc:276] 0709 14:54:40.972043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:40.972212 (+   169us) service_pool.cc:224] Handling call
0709 14:54:40.985202 (+ 12990us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:40.985702 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:40.985753 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:40.996897 (+ 11144us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.474307 (+477410us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:41.474380 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.474510 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.115357 (+2640847us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:44.163417 (+ 48060us) raft_consensus.cc:1685] finished
0709 14:54:44.163443 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.175465 (+ 12022us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1922,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":6195,"spinlock_wait_cycles":38217984,"thread_start_us":438816,"threads_started":1,"wal-append.queue_time_us":673546}
I20260709 14:54:44.177521  9812 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:44.104190  9924 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.191804  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.191550  9768 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:44.197504  9744 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.197789  9768 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:44.199314  9985 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.567s	user 0.008s	sys 0.005s
I20260709 14:54:44.201660  2754 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:44.186981  9935 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:44.211570  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.221450  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.222568  3546 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:44.227890  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.233579  2753 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:44.213725  9935 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:44.240782  9942 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:44.255440  9964 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.258570  3540 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:44.261044  9956 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:44.263025  3409 leader_election.cc:400] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25.
I20260709 14:54:44.292209  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.293463  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:44.263391  9858 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:44.189925  9905 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.219427  9753 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:44.272397  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:44.303871  2752 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:44.282519  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:44.284164  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.287475  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.294670  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.291574  9743 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.298638  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.301102  3538 raft_consensus.cc:1240] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:44.307920  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:44.346608  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.327067  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1341) took 1047 ms. Trace:
I20260709 14:54:44.403646  2678 rpcz_store.cc:276] 0709 14:54:43.279871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.147498 (+867627us) service_pool.cc:224] Handling call
0709 14:54:44.147923 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.148730 (+   807us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 14:54:44.155045 (+  6315us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.326274 (+171229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:44.326467 (+   193us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:44.326597 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.326722 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.326971 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7077,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":265,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":4931,"spinlock_wait_cycles":9624960,"thread_start_us":144723,"threads_started":1}
I20260709 14:54:44.310094  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.412371  3545 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:44.348917  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:44.362401  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63848.9us
I20260709 14:54:44.419731  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:44.420699  2678 raft_consensus.cc:1275] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 14:54:44.340934  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.317868  3541 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:54:44.339710  2650 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.337908  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.421685  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1350) took 1125 ms. Trace:
I20260709 14:54:44.329069  2755 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:44.307509  3546 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:54:44.425709  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 84937.6us
I20260709 14:54:44.425457  2678 rpcz_store.cc:276] 0709 14:54:43.296147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.419845 (+1123698us) service_pool.cc:224] Handling call
0709 14:54:44.420272 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.421373 (+  1101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.421611 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:44.426386  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 86805.9us
I20260709 14:54:44.291846  9956 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.450106  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.449774  9852 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:44.466492  9929 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:44.464442  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.467700  3536 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:44.461967  2752 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:44.417212  9961 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:44.338644  9812 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:44.307569  9942 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:54:44.474473 10003 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 14:54:44.416574  9563 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:44.422524  9937 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:44.351455  9561 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 14:54:44.487674  3545 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:44.322541  9768 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:44.491072  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63317.4us
I20260709 14:54:44.503165  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.316797  9753 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.474522  9852 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 14:54:44.508510  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.511576  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.512501  3545 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:44.519695  3546 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:44.532402  9743 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.494627  9812 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:44.502348  9563 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.537408  9563 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:44.539240  9858 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:44.543481 10005 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 14:54:44.548064  3545 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:44.551113  3538 raft_consensus.cc:2468] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:44.555539  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1846) took 1270 ms. Trace:
I20260709 14:54:44.554565  2724 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:44.559634  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 107193us
I20260709 14:54:44.560859  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.564857  3536 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:44.550655  2755 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:44.556208  2756 rpcz_store.cc:276] 0709 14:54:43.285172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.806650 (+521478us) service_pool.cc:224] Handling call
0709 14:54:43.818236 (+ 11586us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:43.822383 (+  4147us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:43.830386 (+  8003us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.168478 (+338092us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b1800208cc0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:44.283242 (+114764us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:44.292045 (+  8803us) log.cc:844] Serialized 52 byte log entry
0709 14:54:44.292255 (+   210us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:44.292291 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.292365 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.545711 (+253346us) raft_consensus.cc:1685] finished
0709 14:54:44.555192 (+  9481us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.555445 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20777,"prepare.queue_time_us":7816,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":5503,"raft.queue_time_us":30017,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":7479,"spinlock_wait_cycles":68266752,"thread_start_us":61293,"threads_started":1,"wal-append.queue_time_us":472098}
I20260709 14:54:44.560528  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.572916  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.544358  9937 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.570784  9563 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:54:44.580116  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.571312  9858 raft_consensus.cc:2804] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:44.582790  9563 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:44.584364  9753 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.592500  9852 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:44.594034  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.595319  9858 raft_consensus.cc:697] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:44.596448  9858 consensus_queue.cc:237] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.600664  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1346) took 1310 ms. Trace:
I20260709 14:54:44.577694  9929 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:44.601588  9929 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:44.601227  2684 rpcz_store.cc:276] 0709 14:54:43.290469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.258922 (+968453us) service_pool.cc:224] Handling call
0709 14:54:44.259434 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:44.260021 (+   587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:44.260057 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:44.260724 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 14:54:44.333108 (+ 72384us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:44.333178 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.333406 (+   228us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.600400 (+266994us) raft_consensus.cc:1685] finished
0709 14:54:44.600424 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.600594 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":71509,"threads_started":1,"wal-append.queue_time_us":246085}
I20260709 14:54:44.602741  9929 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.578425  9868 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:44.603539  9563 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:44.604308  9868 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:44.605402  9868 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:44.605683  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:44.552976  9956 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.607439  9956 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:44.610284 10009 raft_consensus.cc:2764] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 14:54:44.558820  9935 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.611253 10009 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:44.611763 10009 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:44.613098  9935 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:44.594530  9852 consensus_queue.cc:1059] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 14:54:44.615882  9963 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.446s	user 0.004s	sys 0.003s
I20260709 14:54:44.616492  9852 raft_consensus.cc:3055] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:44.616904  9852 raft_consensus.cc:740] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:44.617849  9852 consensus_queue.cc:260] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.612748 10009 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.619820  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:44.569857  9961 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.628365  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:44.581099  9905 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:44.632881  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.639925  9937 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.643183  2684 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:44.646638  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1756) took 3118 ms (client timeout 1964 ms). Trace:
I20260709 14:54:44.647269  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:44.647305  2685 rpcz_store.cc:269] 0709 14:54:41.528695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.528854 (+   159us) service_pool.cc:224] Handling call
0709 14:54:44.646312 (+3117458us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001c0554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:44.646533 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3136174848}
I20260709 14:54:44.649228  2684 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:44.649398  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:44.649870  2685 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 14:54:44.650313  2685 pending_rounds.cc:85] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 11
I20260709 14:54:44.651978  2685 pending_rounds.cc:107] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 14:54:44.652407  2685 raft_consensus.cc:2889] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:44.630318  9852 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:44.658630  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.664127  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1357) took 1144 ms. Trace:
I20260709 14:54:44.664990  2685 rpcz_store.cc:276] 0709 14:54:43.509402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.649095 (+1139693us) service_pool.cc:224] Handling call
0709 14:54:44.649453 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.653183 (+  3730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.653405 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:44.667188  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1353) took 1370 ms. Trace:
I20260709 14:54:44.667743  2678 rpcz_store.cc:276] 0709 14:54:43.296997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.451793 (+1154796us) service_pool.cc:224] Handling call
0709 14:54:44.452179 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:44.452618 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:44.452653 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:44.466768 (+ 14115us) log.cc:844] Serialized 26 byte log entry
0709 14:54:44.627921 (+161153us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:44.627988 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.628102 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.666916 (+ 38814us) raft_consensus.cc:1685] finished
0709 14:54:44.666938 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.667115 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":3729,"thread_start_us":127563,"threads_started":1,"wal-append.queue_time_us":181042}
I20260709 14:54:44.670940  2678 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
W20260709 14:54:44.671802  9973 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.016s	user 0.000s	sys 0.002s
I20260709 14:54:44.674064  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1349) took 1921 ms. Trace:
I20260709 14:54:44.671774  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1812) took 1116 ms. Trace:
I20260709 14:54:44.674505  2751 rpcz_store.cc:276] 0709 14:54:42.752638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:42.753692 (+  1054us) service_pool.cc:224] Handling call
0709 14:54:42.768366 (+ 14674us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:42.944974 (+176608us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:43.199399 (+254425us) log.cc:844] Serialized 14 byte log entry
0709 14:54:43.502811 (+303412us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:43.515838 (+ 13027us) log.cc:844] Serialized 52 byte log entry
0709 14:54:43.516107 (+   269us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:43.516161 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:43.516285 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.671102 (+1154817us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:44.673748 (+  2646us) raft_consensus.cc:1685] finished
0709 14:54:44.673860 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.674001 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":448682,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2669,"raft.queue_time_us":20923,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":19764,"spinlock_wait_cycles":25291136,"thread_start_us":605730,"threads_started":3,"wal-append.queue_time_us":375429}
I20260709 14:54:44.678645  2678 rpcz_store.cc:276] 0709 14:54:43.555800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.670263 (+1114463us) service_pool.cc:224] Handling call
0709 14:54:44.670682 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.671465 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.671702 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:44.680862  2678 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
W20260709 14:54:44.681387  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.681799  9961 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.682425  9661 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:44.662802  9923 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:44.727232  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1811) took 1171 ms. Trace:
I20260709 14:54:44.839879  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1360) took 1078 ms. Trace:
I20260709 14:54:44.840448  2685 rpcz_store.cc:276] 0709 14:54:43.555406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.666124 (+1110718us) service_pool.cc:224] Handling call
0709 14:54:44.666607 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.666898 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:44.667238 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.726575 (+ 59337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:44.726667 (+    92us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:44.726755 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.726894 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.727109 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":177,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":168,"thread_start_us":58769,"threads_started":1}
I20260709 14:54:44.840618  2678 rpcz_store.cc:276] 0709 14:54:43.761783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.680136 (+918353us) service_pool.cc:224] Handling call
0709 14:54:44.680636 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.692745 (+ 12109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.839782 (+147037us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:44.804543  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.843999  2678 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:44.845077  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1765) took 1009 ms. Trace:
I20260709 14:54:44.846017  2685 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:44.845649  2678 rpcz_store.cc:276] 0709 14:54:43.836141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.842789 (+1006648us) service_pool.cc:224] Handling call
0709 14:54:44.843175 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.844638 (+  1463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.844982 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:44.849220  2678 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: [17.10-20.11]
I20260709 14:54:44.851239  2678 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:44.853567  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1364) took 1088 ms. Trace:
I20260709 14:54:44.854192  2685 rpcz_store.cc:276] 0709 14:54:43.765250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.842586 (+1077336us) service_pool.cc:224] Handling call
0709 14:54:44.843787 (+  1201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.852886 (+  9099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.853471 (+   585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:44.856343  2685 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 14:54:44.867323  2678 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:44.889130  3536 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:44.894104  9898 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 3.159s	user 0.007s	sys 0.010s
I20260709 14:54:44.895936  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1326) took 4087 ms. Trace:
I20260709 14:54:44.896467  2603 rpcz_store.cc:276] 0709 14:54:40.808692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.150558 (+341866us) service_pool.cc:224] Handling call
0709 14:54:41.152490 (+  1932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:41.153147 (+   657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:41.153197 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:41.194353 (+ 41156us) log.cc:844] Serialized 26 byte log entry
0709 14:54:41.525450 (+331097us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:41.525502 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:41.525595 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.879059 (+3353464us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:44.895513 (+ 16454us) raft_consensus.cc:1685] finished
0709 14:54:44.895546 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.895862 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":14333,"spinlock_wait_cycles":45631232,"thread_start_us":262190,"threads_started":1,"wal-append.queue_time_us":444730}
I20260709 14:54:44.906245  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2012) took 1317 ms. Trace:
I20260709 14:54:44.907021  2608 rpcz_store.cc:276] 0709 14:54:43.589362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.677210 (+ 87848us) service_pool.cc:224] Handling call
0709 14:54:43.704393 (+ 27183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:43.765930 (+ 61537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.905782 (+1139852us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b54001d9054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:44.906123 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93640448}
I20260709 14:54:44.908964  2608 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:44.909487  2608 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:44.912420  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1342) took 1327 ms. Trace:
I20260709 14:54:44.913070  2611 rpcz_store.cc:276] 0709 14:54:43.584722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.655729 (+ 71007us) service_pool.cc:224] Handling call
0709 14:54:44.912085 (+1256356us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b5400135954. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:44.912326 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193962240}
I20260709 14:54:44.920123  9538 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:44.922799  2682 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:44.924219  3547 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:44.920361  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45644 (request call id 1766) took 1083 ms. Trace:
I20260709 14:54:44.925374  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.925632  2680 rpcz_store.cc:276] 0709 14:54:43.836538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.058003 (+221465us) service_pool.cc:224] Handling call
0709 14:54:44.920134 (+862131us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:44.920289 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67282816}
I20260709 14:54:44.927084  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:44.927549  9563 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:44.928012  9561 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:44.928400  9930 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:44.928884  9930 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:44.929529  2651 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:44.929991  9974 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.292s	user 0.002s	sys 0.000s
I20260709 14:54:44.930481  9561 raft_consensus.cc:2804] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:44.931092  9561 raft_consensus.cc:697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:44.932077  9561 consensus_queue.cc:237] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.910600  2608 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:44.933315  9908 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:44.933791  9908 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:44.935525  9921 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:44.935870  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:44.936712  9921 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:44.938125  9563 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:44.938797  9563 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:44.939529  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1354) took 1602 ms. Trace:
I20260709 14:54:44.941285  2651 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:44.942127  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 1.0517e+06us
I20260709 14:54:44.945905  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:44.952628  9849 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:44.953691  2608 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:44.940011  2757 rpcz_store.cc:276] 0709 14:54:43.337782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.831702 (+493920us) service_pool.cc:224] Handling call
0709 14:54:43.901419 (+ 69717us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:43.976791 (+ 75372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:43.976851 (+    60us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:43.977827 (+   976us) log.cc:844] Serialized 52 byte log entry
0709 14:54:44.128067 (+150240us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:44.130573 (+  2506us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.130919 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.131008 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.935877 (+804869us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800115540. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:44.939151 (+  3274us) raft_consensus.cc:1685] finished
0709 14:54:44.939264 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.939461 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79054,"prepare.queue_time_us":2131,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6087,"raft.queue_time_us":76755,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":10859,"spinlock_wait_cycles":4760832,"thread_start_us":79365,"threads_started":1,"wal-append.queue_time_us":128010}
I20260709 14:54:44.956873  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2015) took 1277 ms. Trace:
I20260709 14:54:44.957420  2606 rpcz_store.cc:276] 0709 14:54:43.679087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.151028 (+471941us) service_pool.cc:224] Handling call
0709 14:54:44.152953 (+  1925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.956419 (+803466us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:44.956454 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:44.956504 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:44.956548 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.956629 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.956800 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":802902}
I20260709 14:54:44.958570  9849 raft_consensus.cc:697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:44.959965  9849 consensus_queue.cc:237] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:44.962896  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1659) took 1756 ms. Trace:
I20260709 14:54:44.963407  3543 rpcz_store.cc:276] 0709 14:54:43.206218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.206406 (+   188us) service_pool.cc:224] Handling call
0709 14:54:43.206863 (+   457us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:43.220303 (+ 13440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:43.220356 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:43.239179 (+ 18823us) log.cc:844] Serialized 52 byte log entry
0709 14:54:43.458135 (+218956us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:43.458202 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:43.458316 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:44.929290 (+1470974us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:44.953574 (+ 24284us) raft_consensus.cc:1685] finished
0709 14:54:44.953599 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.962782 (+  9183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":360,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":15753,"spinlock_wait_cycles":50155904,"thread_start_us":190885,"threads_started":1,"wal-append.queue_time_us":389453}
I20260709 14:54:44.934703  9908 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:44.967868  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2027) took 1220 ms. Trace:
I20260709 14:54:44.968430  2612 rpcz_store.cc:276] 0709 14:54:43.747511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.155047 (+407536us) service_pool.cc:224] Handling call
0709 14:54:44.155294 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.155466 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:44.163955 (+  8489us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.967243 (+803288us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.967429 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:44.967475 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:44.967537 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.967610 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.967791 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745965,"prepare.queue_time_us":15565,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"spinlock_wait_cycles":61531520,"thread_start_us":56504,"threads_started":1,"wal-append.queue_time_us":169858}
I20260709 14:54:44.971549  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2031) took 1218 ms. Trace:
I20260709 14:54:44.972098  2610 rpcz_store.cc:276] 0709 14:54:43.753192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.463593 (+710401us) service_pool.cc:224] Handling call
0709 14:54:44.465930 (+  2337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.466274 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:44.493097 (+ 26823us) log.cc:844] Serialized 14 byte log entry
0709 14:54:44.971054 (+477957us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:44.971133 (+    79us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:44.971192 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:44.971284 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:44.971467 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":417439,"prepare.queue_time_us":26630,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2907,"thread_start_us":59970,"threads_started":1,"wal-append.queue_time_us":87724}
I20260709 14:54:44.954494  9953 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:54:44.985790  9953 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:45.027853  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:44.972726  9858 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:45.083318  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.120303  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.116233  9855 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:44.939807  9563 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:45.159844  9988 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 1.527s	user 0.018s	sys 0.010s
I20260709 14:54:45.135190  9855 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:45.209519  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 263387us
W20260709 14:54:45.211419  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 270030us
W20260709 14:54:45.213073  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.219105  9616 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.219918  9331 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.117503  9858 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:45.222798  9941 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.275377  9331 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.276649  9941 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.280205  9331 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.280576  9941 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:45.218596  2576 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.015s
I20260709 14:54:45.230643  9383 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:45.133699  9953 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.283981  9383 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:45.284750  9383 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:45.286273  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.289088  9544 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:45.258529  2603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1352) took 1213 ms. Trace:
W20260709 14:54:45.283617  2576 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.007s	sys 0.015s
W20260709 14:54:45.298883  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 606969us
W20260709 14:54:45.300951  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.304998 10028 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.306341  9761 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:45.297849  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:45.308816  9916 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:45.309551  9916 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:45.310596  2757 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:45.311578 10009 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:45.312207 10009 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:45.312652 10009 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:45.320075  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:45.324549  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1345) took 2034 ms. Trace:
I20260709 14:54:45.323470  9916 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.290784  9544 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:45.329069  2609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 2022) took 1577 ms (client timeout 1961 ms). Trace:
W20260709 14:54:45.329751  2609 rpcz_store.cc:269] 0709 14:54:43.733049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.035285 (+302236us) service_pool.cc:224] Handling call
0709 14:54:45.310398 (+1275113us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54001eb254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.310728 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317513088}
I20260709 14:54:45.291873  2603 rpcz_store.cc:276] 0709 14:54:44.045134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.900476 (+855342us) service_pool.cc:224] Handling call
0709 14:54:44.900842 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.901109 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:44.901867 (+   758us) log.cc:844] Serialized 14 byte log entry
0709 14:54:45.256682 (+354815us) log.cc:844] Serialized 14 byte log entry
0709 14:54:45.257143 (+   461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:45.257224 (+    81us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:45.257366 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.257546 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:45.258107 (+   561us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3099,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":156,"raft.queue_time_us":16596,"spinlock_wait_cycles":32085120,"thread_start_us":340084,"threads_started":1,"wal-append.queue_time_us":360771}
I20260709 14:54:45.310361  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:45.325230  2682 rpcz_store.cc:276] 0709 14:54:43.290157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.257638 (+967481us) service_pool.cc:224] Handling call
0709 14:54:44.922420 (+664782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:44.923393 (+   973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.323764 (+400371us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400444650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.324157 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527772928}
I20260709 14:54:45.338917  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1356) took 1829 ms. Trace:
I20260709 14:54:45.339555  2684 rpcz_store.cc:276] 0709 14:54:43.509243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.644825 (+1135582us) service_pool.cc:224] Handling call
0709 14:54:44.648915 (+  4090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:44.650075 (+  1160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.338292 (+688217us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b54001c0550. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.338660 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8128896}
I20260709 14:54:45.342518  9855 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:45.347074  9616 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:45.347884  9616 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:45.348518  9616 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:45.354130  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:45.357738  9383 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.306422 10028 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.363300  9383 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.369392  9843 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:45.370285  9744 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.373395  9331 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:45.377481  9331 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:45.378981  9858 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:45.378118  9544 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:45.372999  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:45.389375  9933 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:45.390583  9933 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:45.391157  9933 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:45.394253  9544 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:45.395920  9991 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:45.396440  9383 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:45.398448  9991 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:45.399242  9991 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:45.389931 10028 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:45.400796 10001 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.111s	user 0.003s	sys 0.010s
I20260709 14:54:45.403162  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1341) took 4491 ms. Trace:
I20260709 14:54:45.404273  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.406517  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.408267  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.412571  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:45.414090  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 99466.6us
W20260709 14:54:45.416059  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.415843  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105899us
I20260709 14:54:45.417346  9895 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:45.418387  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:45.406561  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:45.419481  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:45.423911  9916 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:45.426870  9858 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 14:54:45.431628  2607 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 14:54:45.428369  9901 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:45.417910  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1892) took 1140 ms. Trace:
I20260709 14:54:45.432372  9901 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:45.432478  3547 rpcz_store.cc:276] 0709 14:54:44.277413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.277597 (+   184us) service_pool.cc:224] Handling call
0709 14:54:45.417268 (+1139671us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54002251d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.417506 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451007744}
I20260709 14:54:45.430346  9812 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:45.403824  2759 rpcz_store.cc:276] 0709 14:54:40.911225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:41.124233 (+213008us) service_pool.cc:224] Handling call
0709 14:54:41.124540 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:43.029414 (+1904874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.402279 (+2372865us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:45.402865 (+   586us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4170935552}
I20260709 14:54:45.436576  9858 consensus_queue.cc:1243] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer c1b0e692e4294ee3b8694bbada174ba1 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 14:54:45.438478  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.440362  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.441430 10002 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.023s	user 0.007s	sys 0.006s
I20260709 14:54:45.434228  9901 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.444038  9913 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:45.444444  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:45.445188  9913 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:45.446688  9942 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:45.447158  2608 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:54:45.448665  9913 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:45.451409  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2034) took 1335 ms. Trace:
I20260709 14:54:45.430621  9991 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.445622  3547 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 14:54:45.452476  2608 rpcz_store.cc:276] 0709 14:54:44.116086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.908249 (+792163us) service_pool.cc:224] Handling call
0709 14:54:44.908772 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:45.447639 (+538867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.451286 (+  3647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:45.328603  9941 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:45.395089  9544 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:45.444129  9901 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:45.460386  9843 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:45.474547  9921 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:45.475893  9921 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:45.464351  9544 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:45.483091  3543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:45.484669  3543 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:45.460672  9913 raft_consensus.cc:2764] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 14:54:45.493148  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 74994.6us
I20260709 14:54:45.498373  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.466779 10028 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:45.507777  2609 raft_consensus.cc:3060] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:45.513657  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.502734  9843 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 14:54:45.519374  9331 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:45.519666  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:45.521124  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.526130  2608 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:45.553462  9743 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:45.553879  9937 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:45.554966  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53725.7us
I20260709 14:54:45.555243  9937 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.558652  9936 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:45.559993  9936 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:45.560051  9945 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:45.563788  9945 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:45.563341  9930 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.568221  9930 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:45.572608  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1801) took 2320 ms. Trace:
I20260709 14:54:45.573675  2683 rpcz_store.cc:276] 0709 14:54:43.252060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:43.253547 (+  1487us) service_pool.cc:224] Handling call
0709 14:54:43.915104 (+661557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:44.046969 (+131865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.507727 (+1460758us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:45.571854 (+ 64127us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540012cd50. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.572266 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812141184}
I20260709 14:54:45.554396  9743 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:45.576126  9743 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:45.578189  9743 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:45.580235  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:45.581844  9609 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:45.582979  9936 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:45.583920  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51822.7us
I20260709 14:54:45.584399  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37160 (request call id 1877) took 1509 ms. Trace:
I20260709 14:54:45.584996  3541 rpcz_store.cc:276] 0709 14:54:44.075105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.076501 (+  1396us) service_pool.cc:224] Handling call
0709 14:54:45.583877 (+1507376us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:45.584159 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471810176}
I20260709 14:54:45.591485  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:45.592491  2608 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:45.520676  9383 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 14:54:45.565261  9945 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:45.548274  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2030) took 1704 ms. Trace:
I20260709 14:54:45.603425  9383 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:45.538692  9843 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 14:54:45.611047  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:45.612054  2608 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:45.603376  2603 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:45.525313  9941 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:45.632253  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89112.6us
I20260709 14:54:45.611105  9383 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:45.642613  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45736.9us
I20260709 14:54:45.635774  9991 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:45.558648  9937 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.619959  9964 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.622017  9921 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:45.600059  2605 rpcz_store.cc:276] 0709 14:54:43.752617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.354739 (+602122us) service_pool.cc:224] Handling call
0709 14:54:44.355304 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:44.971289 (+615985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:44.971330 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:44.971959 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 14:54:45.340980 (+369021us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:45.364725 (+ 23745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:45.367965 (+  3240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:45.368009 (+    44us) raft_consensus.cc:1685] finished
0709 14:54:45.368021 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:45.452852 (+ 84831us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":615425,"raft.queue_time_us":629174,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"thread_start_us":363177,"threads_started":1,"wal-append.queue_time_us":380037}
I20260709 14:54:45.664389  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:45.680356  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46577.1us
I20260709 14:54:45.691407  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:45.692611  3545 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:45.700585  9761 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 14:54:45.707163  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:45.712085  9383 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:45.714092  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.716152  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.717013  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.728760  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.716864  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:45.737890  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 98716.2us
W20260709 14:54:45.739238  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.724561  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:45.714758  9383 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:45.743948  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 184863us
I20260709 14:54:45.744297  2608 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:45.749413  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:45.675277  9843 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 14:54:45.767102  3547 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 14:54:45.772307  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.690129 10028 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 14:54:45.717231  9926 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 14:54:45.777741  9561 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 14:54:45.778661  9868 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:45.779443  9849 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:45.782987  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.783416  9853 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:45.784080 10034 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 14:54:45.831487  3545 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 14:54:45.832695  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:45.788285  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 99539.8us
I20260709 14:54:45.789230  2650 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:45.776229  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103407us
I20260709 14:54:45.790298  9761 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:45.790475  9900 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:45.814616 10035 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 14:54:45.816709  9852 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.150s	user 0.018s	sys 0.005s
I20260709 14:54:45.725306  9964 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:45.829800  2609 raft_consensus.cc:2468] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:45.817996  9951 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.818799  9945 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:45.824352  9849 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:45.861145  9849 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:45.858134  9761 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:45.863446  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:45.859928  9868 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:45.876772  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 120200us
I20260709 14:54:45.819882 10009 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.831393  9616 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.866302  9853 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:45.864178  9761 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:45.899768  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:45.893064  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 122005us
I20260709 14:54:45.901222  2609 raft_consensus.cc:2368] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19.
I20260709 14:54:45.901917  3412 leader_election.cc:400] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:45.902411  3412 leader_election.cc:403] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:54:45.903553  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:45.897055  3537 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
W20260709 14:54:45.908812  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 84132.5us
I20260709 14:54:45.936986  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:45.947288  9628 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.947613  9561 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:54:45.948541  9561 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:45.948508  9628 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:45.885955  9951 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.950457  9561 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:45.953547  9951 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:45.949173  9929 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:45.975746  2609 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:54:45.971539  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:45.982800  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:45.943318  9855 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:45.975961  9933 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.584s	user 0.019s	sys 0.000s
I20260709 14:54:45.947340  9858 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:46.002481  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.000439  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 66264us
I20260709 14:54:45.998531  9941 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:46.007092  2759 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:54:45.985077  9843 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:45.983232  9901 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:45.960633  9936 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 14:54:46.004333  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91177.8us
I20260709 14:54:45.961447  9900 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.011093  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.018640  2757 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 22. Current term is 23. Ops: []
W20260709 14:54:46.021013 10042 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 14:54:46.022537 10044 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 14:54:46.023972  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 160948us
I20260709 14:54:46.002468  9933 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.027114  9628 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:46.028083  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 133153us
I20260709 14:54:46.029248  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:46.035548 10036 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 14:54:46.038309  9563 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:46.039057  9858 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.039386  9586 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:46.029426  9933 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:46.043067  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.062541  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.009130  9544 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.003s
I20260709 14:54:46.076012 10043 raft_consensus.cc:3060] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 14:54:46.077291  9900 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.079875  9930 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 14:54:46.083262  3547 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:46.054384  9941 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:46.084101  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 307367us
W20260709 14:54:46.092289  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.093474  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.089319  2652 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.082103  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:46.087937  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.008239  9929 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.079964  9868 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:46.107949  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78923.4us
I20260709 14:54:46.054569  9853 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.108693  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 15.
I20260709 14:54:46.110929  9924 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:46.111491  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.114012  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.084098 10009 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:46.056933  9855 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.071694  9901 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.123466 10060 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:46.124521  9916 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.700s	user 0.014s	sys 0.003s
I20260709 14:54:46.125761  2759 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 14:54:46.122761 10055 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.080861  9544 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.069909  9849 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:46.149433  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.159029  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.166934  9945 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.168198  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.167900  9945 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.169025  9916 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:46.171022  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:46.171461 10051 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.206s	user 0.004s	sys 0.000s
I20260709 14:54:46.171264  9945 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.174240  9929 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.174989  9616 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.175814  9616 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:46.176733  9616 consensus_queue.cc:1243] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:46.177497  9616 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:46.177889  9616 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:46.183466  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 98684.9us
W20260709 14:54:46.159435  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2004) took 1928 ms (client timeout 2307 ms). Trace:
I20260709 14:54:46.172564  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:46.187105  2755 rpcz_store.cc:269] 0709 14:54:44.231137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:44.281923 (+ 50786us) service_pool.cc:224] Handling call
0709 14:54:46.158929 (+1877006us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b54002058d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:46.159184 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45788544}
I20260709 14:54:46.196033  3538 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:46.198170  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116183us
I20260709 14:54:46.206178  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.207540  2609 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:46.203994  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.209208  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:46.199482  9563 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:46.216905  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105752us
W20260709 14:54:46.222877  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 93669.5us
I20260709 14:54:46.199951  9868 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:46.242776  2608 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:54:46.220264  9936 raft_consensus.cc:2764] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 14:54:46.198215 10055 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.227945 10009 raft_consensus.cc:2804] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:46.237561  9964 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:46.232918  9937 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:46.250912  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.252673  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.240293  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.260155  9904 consensus_queue.cc:1048] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 14:54:46.262579  3540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.263586  3540 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:46.263901  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:46.223822 10060 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.265009  2684 raft_consensus.cc:2376] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22. Re-sending same reply.
I20260709 14:54:46.267378  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.269268  2683 raft_consensus.cc:1275] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:46.271349  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.271351  9858 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.243773  9945 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.274981  9868 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.275529  9849 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.256699  9937 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:46.277544  9937 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:46.277827  9868 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.275344  9945 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.257936  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.272781  3547 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 18: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:46.272784  9855 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:46.314318  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64196.2us
W20260709 14:54:46.286510  9743 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.009s
I20260709 14:54:46.280313  9544 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:46.325922  2759 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 14:54:46.240799  9951 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 14:54:46.324545  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 125009us
I20260709 14:54:46.333984  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.334195  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.335315  2684 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:46.335903  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.336704  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:46.325085  2686 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:54:46.330313  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:46.332237  2609 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:46.332463  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.241298  9616 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.244135  9563 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.339506  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:46.351346  2679 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:46.336877  9991 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.004s
I20260709 14:54:46.354511 10069 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:46.355852 10059 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:46.356657 10071 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 14:54:46.357637 10073 consensus_queue.cc:1059] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:46.323338 10009 raft_consensus.cc:697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:46.321624 10055 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.360642  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:46.334213  9901 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.359988  2605 pending_rounds.cc:85] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 10
I20260709 14:54:46.367513  3540 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:46.329296 10060 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.372859 10069 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:46.373822  2604 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:46.374423  2683 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:46.374043  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:46.381877  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 143127us
W20260709 14:54:46.378851  9383 consensus_meta.cc:287] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.621s	user 0.016s	sys 0.000s
I20260709 14:54:46.383442  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.383363  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.384608  2677 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:54:46.384956  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.385095  2684 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:54:46.332749  9904 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 14:54:46.375613  2605 pending_rounds.cc:107] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 14:54:46.414175  2605 raft_consensus.cc:2889] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:46.417901  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:46.421032  2679 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:46.421957 10009 consensus_queue.cc:237] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.427558  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:46.428702  2604 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:46.432116 10064 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 14:54:46.435954  9945 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:46.438205  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.440747 10065 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
I20260709 14:54:46.380198 10069 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:46.444592 10069 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.418609  9616 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:46.446336 10069 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:46.421955 10070 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 14:54:46.466396  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.468030  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:46.465553  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.472456  2609 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 14:54:46.463579  9964 raft_consensus.cc:3055] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:46.489423  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.503798  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54999.7us
I20260709 14:54:46.514269  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:46.484632  9904 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.519441  9743 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:46.527972 10055 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 14:54:46.528244  9853 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 14:54:46.529259 10055 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:46.529690 10055 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:46.523473  9849 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:46.530340  9849 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:46.530812  9849 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:46.530654 10055 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.532539 10055 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:46.534155 10043 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:46.543443  9895 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:46.544598  9895 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.546948  9904 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.547807  9563 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:46.549824  9904 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:46.549892  9895 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:46.552277  9849 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.499042  9964 raft_consensus.cc:740] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:46.559953 10062 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:46.560987 10062 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:46.577173  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 240529us
I20260709 14:54:46.590060  2604 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:46.593987  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.597848  9849 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:46.613080  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.567092  9563 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 14:54:46.619886  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.622767 10061 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.472s	user 0.003s	sys 0.003s
I20260709 14:54:46.627504 10050 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:46.628150 10050 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:46.629153 10066 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:46.629372 10050 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.630074 10066 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.634043  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.634341  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:46.635574  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 318247us
I20260709 14:54:46.636662  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.637364  9936 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.638253  2723 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.640415  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:46.641386  9628 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.643991  9628 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.616209 10028 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:46.644819  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.645330 10028 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:46.642081  9941 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:46.642752  9855 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:46.646566 10028 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.648037 10028 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:46.650729  9964 consensus_queue.cc:260] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:46.651878  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.652480  9964 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:46.638113  9936 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.655771  9936 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:46.658154  9945 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:46.620898  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.660669  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.662796  9855 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:46.663926 10066 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:46.662874  9628 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.665757 10060 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:46.663271 10050 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:46.592501  9937 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:46.671360  9937 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:46.671917  9908 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:46.579151 10062 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.664539 10066 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 14:54:46.666071  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:46.665987  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:46.679229  3541 raft_consensus.cc:2468] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:46.681344  2726 leader_election.cc:400] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 18. Current term is 19.
I20260709 14:54:46.685704  3542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.683645  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:46.688226  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:46.704726 10054 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.706481  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:46.708386  9812 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.709455  9812 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:46.712066  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.713922  9930 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:46.714617  9930 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:46.715111  9930 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:46.712069  9849 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:46.717162  9849 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:54:46.721992  3547 raft_consensus.cc:1240] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:46.723510  3538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:46.723938 10067 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.485s	user 0.003s	sys 0.000s
I20260709 14:54:46.705925 10054 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:46.687357  9936 raft_consensus.cc:2764] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 14:54:46.729905  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:46.743309  3547 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:46.746978  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.757611 10054 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:46.761746  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.781198  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.785342  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:46.782392  2612 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:46.775002  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:46.809911  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:46.814631  3537 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:46.834867  2608 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:46.835477  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:46.743156 10060 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 14:54:46.835543  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:46.824934 10058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.001s	sys 0.013s
I20260709 14:54:46.845559  2612 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:54:46.848016  2612 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:54:46.837040  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:46.856568  9743 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:46.875458  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:46.876922  3547 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:46.880431  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.881678  2755 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:46.886299  2759 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 14:54:46.888356  2752 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:54:46.881278  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.897729  3536 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:46.900934  2611 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:46.775173  9929 consensus_queue.cc:1048] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 14:54:46.905231  3541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:46.907186  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65766.2us
I20260709 14:54:46.906467  3541 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
W20260709 14:54:46.909843  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.910470  9921 consensus_queue.cc:1059] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:46.915436  9913 consensus_queue.cc:1059] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:46.917240  9921 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:46.918911  9913 raft_consensus.cc:3055] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:46.919378  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:46.920313  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.920580  2606 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:46.926705  2650 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:46.928221  9812 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:46.929069  9849 raft_consensus.cc:2804] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:46.929668  9849 raft_consensus.cc:697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:46.932068  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:46.919521  9921 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:46.813429  9937 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:46.930819  9849 consensus_queue.cc:237] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:46.948721  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:46.919875  9913 raft_consensus.cc:740] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:46.960830  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 292256us
I20260709 14:54:46.967597  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:46.972767  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:46.970083 10058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.002s	sys 0.013s
I20260709 14:54:47.012791  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.011375  2754 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:47.012087  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:47.018994  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:47.021044  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 250866us
I20260709 14:54:47.022270  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:47.023954  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:47.025535  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:47.017918  2611 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:47.022702  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:47.028966  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:47.030108  2752 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
W20260709 14:54:47.032701  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:47.033288  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:47.013070  9761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.011s	sys 0.007s
I20260709 14:54:47.014771 10052 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:47.016251  9858 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:47.040258  3545 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:54:47.011137  9929 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:47.048519  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.052668  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:47.046710  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:46.976027  9937 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.021970 10068 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 14:54:47.040851  9761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.011s	sys 0.007s
I20260709 14:54:47.075292  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.034516  9855 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:47.040041  9936 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:47.103454  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:47.104884  3545 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:47.039690 10052 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:47.126397  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:47.072564  9945 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:47.079747  9929 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:47.089201  9855 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:47.076972 10068 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:47.048019 10083 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:47.031111 10081 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 14:54:47.171602  9921 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:47.173617  9913 consensus_queue.cc:260] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:47.099936  9941 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:54:47.210079  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.210992  2603 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:47.214358  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.047824 10073 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:47.237598  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:47.106422 10060 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:47.107388 10028 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:47.227941  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.050756 10009 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.157s
I20260709 14:54:47.247104  3538 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 23. Ops: []
W20260709 14:54:47.252139  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 226650us
I20260709 14:54:47.226075 10052 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:47.047602 10066 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:47.236770 10068 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:54:47.237003 10069 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.772s	user 0.009s	sys 0.008s
W20260709 14:54:47.261109  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.238513  9855 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:47.275354  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 196784us
I20260709 14:54:47.262312  3536 raft_consensus.cc:2393] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:47.265457  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:47.237015 10083 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:47.256489  9901 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.241107  9853 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.292065  3542 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 14:54:47.284065  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:47.261904 10073 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:47.310348  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.259742  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
I20260709 14:54:47.308722  3545 raft_consensus.cc:2376] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25. Re-sending same reply.
I20260709 14:54:47.263552 10091 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:47.270957 10066 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:47.299520  9930 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.006s
I20260709 14:54:47.274487  9761 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:47.368547  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.324465  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.274063 10055 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:47.375025  2683 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:47.330442  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.282598  9812 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.297256 10083 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:47.283442 10081 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:47.388762  3545 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:47.333045  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 358868us
I20260709 14:54:47.403009  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:47.333703  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:47.335595  2684 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:47.337296  9929 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:47.339612  9964 consensus_meta.cc:287] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.687s	user 0.017s	sys 0.000s
I20260709 14:54:47.341248  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.268388 10028 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:47.343954  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78618us
I20260709 14:54:47.411446  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:47.344861  9908 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.002s
W20260709 14:54:47.345293 10093 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:54:47.351135  9921 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:47.351703 10087 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 14:54:47.355813 10086 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 14:54:47.358389 10073 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.329288 10066 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:47.310158  9901 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:47.322324  9853 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.307152 10060 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 14:54:47.378079  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.383179  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.408418  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.417150  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:47.419729  3540 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 14:54:47.478719  3540 pending_rounds.cc:85] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 9
I20260709 14:54:47.479107  3540 pending_rounds.cc:107] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 19.10
W20260709 14:54:47.479188  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62196.2us
W20260709 14:54:47.447777  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:47.480330  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.453620  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.413499  9628 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:47.373670  9930 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.379912 10055 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.475550  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:47.475239  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.479472  3540 raft_consensus.cc:2889] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:47.421464  3545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.484457  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.386515  9761 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:47.498515  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.404933  9812 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.542172  2757 raft_consensus.cc:1275] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:47.464792  9858 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:47.413856 10081 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.490757  9628 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:47.547498  9937 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.427263 10099 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:47.619231  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
W20260709 14:54:47.600241  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.600229  2751 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:47.464401  9383 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:47.490062 10098 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:47.604027  2682 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:47.606292  2759 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 14:54:47.608879  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.625366  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.603430 10108 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.629496  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.632018 10028 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:47.635248  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.640759  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:47.638932 10060 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:47.639758  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.641100  2683 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:47.648365  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:47.614737  9628 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:47.683761  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.630086 10073 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:47.621755  9857 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:47.684998  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:47.691126  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.622627 10099 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:47.697280  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.645037 10098 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:47.733877  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.732604  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.676654  9858 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.711735  3537 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
W20260709 14:54:47.741423  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51518.4us
I20260709 14:54:47.689591  9929 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.789013  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1564) took 1033 ms. Trace:
I20260709 14:54:47.778755  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.796952  3537 rpcz_store.cc:276] 0709 14:54:46.755764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.650194 (+894430us) service_pool.cc:224] Handling call
0709 14:54:47.788925 (+138731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:47.804711  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:47.677657  9383 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.804765  3547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.678192 10060 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:47.668640 10108 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.711931 10099 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:47.732353 10054 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:47.777443  9908 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:47.853530  9908 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.854985  9929 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:47.856980  9908 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.860828  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.862414  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.847816  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1563) took 1092 ms. Trace:
I20260709 14:54:47.711675  9857 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.849375  3537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1"
W20260709 14:54:47.849527 10097 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 14:54:47.789700  9858 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.850098  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.851766  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.739845  9744 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:47.863605  3536 rpcz_store.cc:276] 0709 14:54:46.755418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.706820 (+951402us) service_pool.cc:224] Handling call
0709 14:54:47.847698 (+140878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:47.884388  3546 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 14:54:47.869438  2755 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:47.794714 10091 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:47.776254 10098 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:47.735227 10102 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:47.891237  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1555) took 1140 ms. Trace:
W20260709 14:54:47.893594  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 224196us
I20260709 14:54:47.892376  3546 rpcz_store.cc:276] 0709 14:54:46.751247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.623003 (+871756us) service_pool.cc:224] Handling call
0709 14:54:47.891149 (+268146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:47.858596  9929 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.797469 10062 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:47.909121  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.858675  9901 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:47.905431  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:47.916518  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.880666  9908 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:47.920205  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.912302  9964 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.930857  9964 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:47.932200  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:47.933893  9964 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.892589  9930 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:47.922478  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:47.921690  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:47.936299  3546 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:47.935956  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.884346  9744 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:47.939105  2758 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:47.948242  3546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.944518  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:47.951110  2611 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:47.958601 10118 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 14:54:47.959287 10118 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:47.959729 10118 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:47.950996 10050 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.284s	user 0.014s	sys 0.004s
W20260709 14:54:47.964130  9855 consensus_meta.cc:287] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.006s
I20260709 14:54:47.964797  9858 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:47.964689  9855 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:47.967046  9937 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.967769  9937 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.973805  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:47.975178  2758 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 14:54:47.902572 10102 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.976245  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:47.976411  9857 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.977394 10102 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.978385  3537 raft_consensus.cc:2393] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:47.939837 10062 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:47.979589  9383 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:47.980849 10094 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
I20260709 14:54:47.982198  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.982997 10062 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:47.983455  9761 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:47.983721  3536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "c1b0e692e4294ee3b8694bbada174ba1" is_pre_election: true
I20260709 14:54:47.984488  3536 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:54:47.984566 10101 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.938781  9853 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:47.985273 10101 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.986059  3536 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:47.986294  9853 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:47.986959 10097 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:54:47.986990  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:47.987681 10095 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 14:54:47.989079  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.934798  9964 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:47.991096  9855 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.991325  9964 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:47.992384  9921 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 14:54:47.993192 10091 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:47.993978  9964 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:47.993299  9744 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:47.994658  9930 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:47.995330  9930 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:47.997565  9930 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:47.950841  9908 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:48.005350  2681 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:48.005474  9908 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:47.992303  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 97568.5us
I20260709 14:54:48.009447  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.010713 10114 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:47.990506  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:47.984740  3546 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:48.021039  3541 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:48.022033  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1559) took 1268 ms. Trace:
I20260709 14:54:48.022198  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:48.022495  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1554) took 1271 ms. Trace:
I20260709 14:54:48.022486  3541 rpcz_store.cc:276] 0709 14:54:46.754099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.011636 (+1257537us) service_pool.cc:224] Handling call
0709 14:54:48.020804 (+  9168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.021546 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.021966 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.024950  3546 rpcz_store.cc:276] 0709 14:54:46.750904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.983815 (+1232911us) service_pool.cc:224] Handling call
0709 14:54:47.984492 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.022150 (+ 37658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.022433 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:48.022243  2679 consensus_meta.cc:287] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.000s
I20260709 14:54:48.033565  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1556) took 1282 ms. Trace:
I20260709 14:54:48.033838  2679 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:48.034140  3536 rpcz_store.cc:276] 0709 14:54:46.751595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.985204 (+1233609us) service_pool.cc:224] Handling call
0709 14:54:47.985584 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:47.986609 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.033451 (+ 46842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:47.979662 10081 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:48.036589  2609 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:54:48.037360  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:48.038476  2684 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:48.041357  2683 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:48.042223  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:48.040659  3546 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 14:54:48.042649  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:48.035260  9563 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.047043  9563 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:48.048969 10092 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.726s	user 0.006s	sys 0.000s
I20260709 14:54:48.051301  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1543) took 1349 ms. Trace:
I20260709 14:54:48.051865  3543 rpcz_store.cc:276] 0709 14:54:46.702232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.702536 (+   304us) service_pool.cc:224] Handling call
0709 14:54:46.703186 (+   650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:46.703811 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:46.703860 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:46.704870 (+  1010us) log.cc:844] Serialized 52 byte log entry
0709 14:54:47.020881 (+316011us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:47.020942 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:47.021020 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.047977 (+1026957us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:48.050726 (+  2749us) raft_consensus.cc:1685] finished
0709 14:54:48.050755 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.051080 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2082,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":172,"spinlock_wait_cycles":74256256,"thread_start_us":271465,"threads_started":1,"wal-append.queue_time_us":542981}
I20260709 14:54:48.051301  9812 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:48.020821 10028 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:340: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 14:54:48.040345  9852 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 14:54:48.056792   431 tablet_server.cc:179] TabletServer@127.0.107.193:37831 shutting down...
W20260709 14:54:48.057528 10089 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.765s	user 0.003s	sys 0.003s
W20260709 14:54:48.060209 10083 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.001s
I20260709 14:54:48.060873 10083 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.032687  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:48.062397  3547 raft_consensus.cc:2368] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20.
I20260709 14:54:48.064486  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1570) took 1306 ms. Trace:
W20260709 14:54:48.062587  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.053715  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44397.5us
I20260709 14:54:48.069775  3546 rpcz_store.cc:276] 0709 14:54:46.758439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.027438 (+1268999us) service_pool.cc:224] Handling call
0709 14:54:48.029318 (+  1880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.043579 (+ 14261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.064418 (+ 20839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.021589  9855 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:48.079644  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.079677  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:48.080950  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.080971  2686 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:54:48.080618  2751 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:48.076182  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.083666  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.069422  2758 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:48.087543  2683 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:48.069242  2681 raft_consensus.cc:2393] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:48.072068  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:48.095561  2679 raft_consensus.cc:2376] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23. Re-sending same reply.
I20260709 14:54:48.097235  2679 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:48.072935  2678 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 14:54:48.026173  9744 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:48.101817  2683 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 14:54:48.093977  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.099725  3536 raft_consensus.cc:1240] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:48.050269 10099 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:48.109592  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1574) took 1349 ms. Trace:
I20260709 14:54:48.110431  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.110328  3536 rpcz_store.cc:276] 0709 14:54:46.759640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.075944 (+1316304us) service_pool.cc:224] Handling call
0709 14:54:48.099519 (+ 23575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.109128 (+  9609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.109497 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.111564  2760 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:48.113802  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.105026  2751 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:48.105310  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.119462  2683 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:48.119999  2683 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:48.120460  2604 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 14:54:48.120985  2683 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:48.122210  2683 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:48.122354  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 46289.8us
I20260709 14:54:48.123564  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.124426  2609 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:48.124982  2723 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.127161  2603 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:48.127588  2686 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:48.127805  2603 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:48.130175  2678 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:54:48.083518  9761 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.133921  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.136232  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.089875  9383 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.101631 10009 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:48.139983  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:48.140463  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.136085  9930 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:48.139381  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:48.147902  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.080169  9855 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:48.137830 10052 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.837s	user 0.015s	sys 0.000s
I20260709 14:54:48.155527  2758 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:48.152657  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2053) took 1572 ms. Trace:
I20260709 14:54:48.156056  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:48.140727  3411 connection.cc:256] RPC call timeout handler was delayed by -1.0613659110000526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3027s for a response
I20260709 14:54:48.144518  9858 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:48.153925  2603 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.144672 10009 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:48.140748 10114 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.160099  2610 rpcz_store.cc:276] 0709 14:54:46.579901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.580529 (+   628us) service_pool.cc:224] Handling call
0709 14:54:46.595507 (+ 14978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:46.666659 (+ 71152us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 14:54:46.667144 (+   485us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.018320 (+351176us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:47.036265 (+ 17945us) log.cc:844] Serialized 26 byte log entry
0709 14:54:47.036454 (+   189us) raft_consensus.cc:1645] Marking committed up to 15
0709 14:54:47.036489 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:47.036573 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.056753 (+1020180us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:48.144305 (+ 87552us) raft_consensus.cc:1685] finished
0709 14:54:48.152306 (+  8001us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.152546 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34422,"prepare.queue_time_us":2396,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":40136,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":7102,"spinlock_wait_cycles":49066752,"thread_start_us":337713,"threads_started":1,"wal-append.queue_time_us":524540}
I20260709 14:54:48.179951  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.181218  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.189582  2603 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:48.157043 10101 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.151046 10120 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:48.181870  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1553) took 1431 ms. Trace:
I20260709 14:54:48.194242  3547 rpcz_store.cc:276] 0709 14:54:46.750478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.785234 (+1034756us) service_pool.cc:224] Handling call
0709 14:54:48.181535 (+396301us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400202e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.181778 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551965568}
I20260709 14:54:48.172251 10062 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.184890 10009 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:48.199743  2606 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 14:54:48.200048  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:48.145141  9761 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.200358  2606 pending_rounds.cc:85] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 10
I20260709 14:54:48.202697  2606 pending_rounds.cc:107] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 14:54:48.203159  2606 raft_consensus.cc:2889] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:48.148916  9383 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.198318  2751 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:48.207273  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.176949  9930 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:54:48.216208  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.208479  2604 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:48.164373 10052 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.214072  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:48.213125  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:48.222429  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 76641.7us
I20260709 14:54:48.219691  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2057) took 1289 ms. Trace:
I20260709 14:54:48.228173  2683 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
W20260709 14:54:48.196174 10122 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 14:54:48.228428  2751 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:48.196398 10120 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:48.227828  2606 rpcz_store.cc:276] 0709 14:54:46.930218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.971391 (+ 41173us) service_pool.cc:224] Handling call
0709 14:54:48.179686 (+1208295us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b5800010b98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:48.188441 (+  8755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.214879 (+ 26438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.219608 (+  4729us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643840}
I20260709 14:54:48.201452 10062 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:48.214707 10086 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.737s	user 0.005s	sys 0.000s
W20260709 14:54:48.239593  3411 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.590s (SENT)
I20260709 14:54:48.237084  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:48.219453  2575 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.553s (SENT) [suppressed 40 similar messages]
I20260709 14:54:48.251802  9941 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:48.237643 10120 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:48.252633  9941 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:48.256523  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:48.257293  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.021s	sys 0.016s
I20260709 14:54:48.209223 10055 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:48.260345  2751 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:48.236433  9936 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.064s	user 0.016s	sys 0.001s
I20260709 14:54:48.264204  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:48.260154  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.021s	sys 0.017s
W20260709 14:54:48.269302  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 628603us
I20260709 14:54:48.224234 10108 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.231274  9930 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:48.241032 10099 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.192768 10114 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:48.291482  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 63375.4us
I20260709 14:54:48.292951  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:48.303654  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.304870  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.268934  9936 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:48.273653  9743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.009s	sys 0.029s
I20260709 14:54:48.316027  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.273475  9858 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:48.297848 10101 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:48.295198  9563 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.301023  9616 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.327811  2757 raft_consensus.cc:1240] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
W20260709 14:54:48.320212  9743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.010s	sys 0.029s
W20260709 14:54:48.335000  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.334013  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 229593us
W20260709 14:54:48.343257  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 103737us
I20260709 14:54:48.339027  2751 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:48.344167  3411 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.658s (SENT)
I20260709 14:54:48.349308  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1382) took 1006 ms. Trace:
I20260709 14:54:48.350163  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:48.353345  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.353677  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.355371  2751 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 14:54:48.349905  2684 rpcz_store.cc:276] 0709 14:54:47.342464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.044669 (+702205us) service_pool.cc:224] Handling call
0709 14:54:48.045085 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.080001 (+ 34916us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:48.115201 (+ 35200us) log.cc:844] Serialized 14 byte log entry
0709 14:54:48.348880 (+233679us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.348945 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:48.348997 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.349074 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.349232 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32862,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":4946,"thread_start_us":196086,"threads_started":1}
I20260709 14:54:48.360018  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.336761  9616 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:48.341279 10118 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:48.372946  2725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.044s	sys 0.049s
I20260709 14:54:48.363703 10101 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 14:54:48.424775  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.436211  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.400660  9616 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:48.425853  2725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.052s	sys 0.051s
I20260709 14:54:48.375411  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.430055  2760 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:48.388784 10123 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 14:54:48.387895  2753 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:48.407277  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63162.7us
W20260709 14:54:48.414459  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.431039  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.469573  2610 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 14:54:48.472582  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37174 (request call id 1548) took 1746 ms. Trace:
I20260709 14:54:48.473150  3538 rpcz_store.cc:276] 0709 14:54:46.726099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.735300 (+  9201us) service_pool.cc:224] Handling call
0709 14:54:46.755202 (+ 19902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:47.270815 (+515613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.472256 (+1201441us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54004b23d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:48.472494 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116351232}
I20260709 14:54:48.475368  9761 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.475931  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1870) took 1160 ms. Trace:
I20260709 14:54:48.476358  2755 rpcz_store.cc:276] 0709 14:54:47.315832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.316019 (+   187us) service_pool.cc:224] Handling call
0709 14:54:48.475713 (+1159694us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400120054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.475860 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501596544}
W20260709 14:54:48.478016  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.401368 10114 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.384969  9855 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:48.479900  3411 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.862s (SENT)
I20260709 14:54:48.480611  9744 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.481566  9744 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:48.483619  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1869) took 1220 ms. Trace:
I20260709 14:54:48.480389  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:48.485271  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1376) took 1145 ms. Trace:
W20260709 14:54:48.485906  3411 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.943s (SENT)
I20260709 14:54:48.432799  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.486799  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:48.487308  2604 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:48.431854  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.487869 10101 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:54:48.415936  2575 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.527s (SENT)
I20260709 14:54:48.489308  3546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1694) took 1571 ms. Trace:
W20260709 14:54:48.411158 10117 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.448s	user 0.003s	sys 0.001s
W20260709 14:54:48.491822  9628 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.793s	user 0.007s	sys 0.006s
I20260709 14:54:48.492329  9628 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:48.495568  3411 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 1.946s (SENT)
I20260709 14:54:48.496202  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:48.497191 10101 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 14:54:48.497774  3542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37148 (request call id 1692) took 1821 ms (client timeout 1633 ms). Trace:
W20260709 14:54:48.440285  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Timed out: RequestConsensusVote RPC to 127.0.107.193:37831 timed out after 2.219s (SENT)
W20260709 14:54:48.437795 10130 log.cc:927] Time spent T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 14:54:48.417982  3537 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 14:54:48.503230  3537 pending_rounds.cc:85] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting all ops after (but not including) 9
I20260709 14:54:48.503511  3537 pending_rounds.cc:107] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 14:54:48.503775  3537 raft_consensus.cc:2889] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:48.477836  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.485771  2682 rpcz_store.cc:276] 0709 14:54:47.340250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.380216 (+ 39966us) service_pool.cc:224] Handling call
0709 14:54:48.485019 (+1104803us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.485166 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62379904}
I20260709 14:54:48.489334  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.443244 10083 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:48.506810  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 60035.3us
W20260709 14:54:48.480365 10068 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.169s	user 0.017s	sys 0.000s
I20260709 14:54:48.484117  2759 rpcz_store.cc:276] 0709 14:54:47.264891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.265122 (+   231us) service_pool.cc:224] Handling call
0709 14:54:47.309792 (+ 44670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:47.629345 (+319553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.483219 (+853874us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:48.483510 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539015424}
I20260709 14:54:48.381320  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2017) took 2192 ms. Trace:
I20260709 14:54:48.475440  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:48.501672  3541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1695) took 1583 ms. Trace:
W20260709 14:54:48.498252  3542 rpcz_store.cc:269] 0709 14:54:46.676325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.676493 (+   168us) service_pool.cc:224] Handling call
0709 14:54:48.497517 (+1821024us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540009abd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.497674 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370696064}
W20260709 14:54:48.505412  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 89510.5us
W20260709 14:54:48.517995  2575 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.545s (SENT)
I20260709 14:54:48.415587  3540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1912) took 1884 ms. Trace:
I20260709 14:54:48.497109  3543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1917) took 1546 ms. Trace:
I20260709 14:54:48.490201  3546 rpcz_store.cc:276] 0709 14:54:46.917487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.076138 (+1158651us) service_pool.cc:224] Handling call
0709 14:54:48.076578 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:48.083744 (+  7166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.083784 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:48.134443 (+ 50659us) log.cc:844] Serialized 52 byte log entry
0709 14:54:48.359009 (+224566us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:48.359782 (+   773us) log.cc:844] Serialized 14 byte log entry
0709 14:54:48.366502 (+  6720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.366610 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.488846 (+122236us) raft_consensus.cc:1685] finished
0709 14:54:48.488897 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.489180 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2383,"prepare.queue_time_us":189,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":19778,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":12334,"thread_start_us":179946,"threads_started":1,"wal-append.queue_time_us":226578}
I20260709 14:54:48.407240 10052 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:48.506220  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.511022  9951 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:48.511744  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1914) took 1593 ms. Trace:
I20260709 14:54:48.531785  3537 rpcz_store.cc:276] 0709 14:54:46.918273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.417099 (+1498826us) service_pool.cc:224] Handling call
0709 14:54:48.417513 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.507253 (+ 89740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.511667 (+  4414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.533706  3537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37148 (request call id 1696) took 1282 ms. Trace:
W20260709 14:54:48.534155 10098 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.004s
I20260709 14:54:48.535177 10114 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 14:54:48.416304  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:48.535713 10114 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:48.535912  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 119670us
I20260709 14:54:48.536202 10114 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:54:48.536738  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:48.537465  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:48.538096  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:48.389001 10066 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.898s	user 0.014s	sys 0.001s
I20260709 14:54:48.524984  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1919) took 1424 ms. Trace:
I20260709 14:54:48.539173 10066 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:48.539525  3542 rpcz_store.cc:276] 0709 14:54:47.100967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.523930 (+1422963us) service_pool.cc:224] Handling call
0709 14:54:48.524325 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.524563 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.524587 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.524635 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:48.524681 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.524747 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.524887 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.540596  2754 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25.
I20260709 14:54:48.541985  3542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1920) took 1206 ms. Trace:
W20260709 14:54:48.512560 10113 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.541s	user 0.003s	sys 0.000s
I20260709 14:54:48.542672 10066 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.544122 10060 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.545017  9901 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:48.545758  9901 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.548759  9901 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.465657  2677 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:48.551126  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1821) took 1205 ms. Trace:
I20260709 14:54:48.517201  3541 rpcz_store.cc:276] 0709 14:54:46.917760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.115747 (+1197987us) service_pool.cc:224] Handling call
0709 14:54:48.144199 (+ 28452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:48.159922 (+ 15723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.159975 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:48.165712 (+  5737us) log.cc:844] Serialized 26 byte log entry
0709 14:54:48.361546 (+195834us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:48.361608 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.361741 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.501319 (+139578us) raft_consensus.cc:1685] finished
0709 14:54:48.501354 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.501584 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8354,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":9572,"thread_start_us":158360,"threads_started":1,"wal-append.queue_time_us":211256}
I20260709 14:54:48.520764  3543 rpcz_store.cc:276] 0709 14:54:46.950694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.496028 (+1545334us) service_pool.cc:224] Handling call
0709 14:54:48.496465 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.496712 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.496740 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.496791 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:48.496833 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.496900 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.497036 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.523058  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.524133 10062 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:48.516647  2756 rpcz_store.cc:276] 0709 14:54:46.164848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.167045 (+  2197us) service_pool.cc:224] Handling call
0709 14:54:46.167644 (+   599us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:46.198383 (+ 30739us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:46.198786 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 14:54:46.603753 (+404967us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b1800461800. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240
0709 14:54:46.604612 (+   859us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b1800461800. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9
0709 14:54:46.605548 (+   936us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b1800461800. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:47.018971 (+413423us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:47.088859 (+ 69888us) log.cc:844] Serialized 52 byte log entry
0709 14:54:47.089196 (+   337us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:47.089258 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:47.334503 (+245245us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b5400443754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:47.334589 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.191850 (+857261us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b24001ccf88. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:54:48.356594 (+164744us) raft_consensus.cc:1685] finished
0709 14:54:48.356646 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.356900 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37701,"prepare.queue_time_us":1556,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":27288,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":16921,"spinlock_wait_cycles":51386496,"thread_start_us":371055,"threads_started":1,"wal-append.queue_time_us":1025676}
I20260709 14:54:48.552672  2759 raft_consensus.cc:2376] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18. Re-sending same reply.
I20260709 14:54:48.553521  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.553787  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.554960  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.524930 10068 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.513147  9761 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:48.555315  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.557152  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.558073  2759 raft_consensus.cc:2393] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:48.559159  3545 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 25. Current term is 26.
W20260709 14:54:48.539245  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63871.2us
W20260709 14:54:48.551283 10136 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 14:54:48.561492  3545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37174 (request call id 1557) took 1809 ms (client timeout 2219 ms). Trace:
W20260709 14:54:48.561991  3545 rpcz_store.cc:269] 0709 14:54:46.751971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.408281 (+656310us) service_pool.cc:224] Handling call
0709 14:54:48.561191 (+1152910us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b540009b854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.561412 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201900800}
W20260709 14:54:48.552167  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 111922us
W20260709 14:54:48.563491  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.565362  3547 raft_consensus.cc:1240] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26. Ops: []
I20260709 14:54:48.567623  9937 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:48.568373  3547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1913) took 1650 ms. Trace:
I20260709 14:54:48.569172  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.569574  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.570315  2755 raft_consensus.cc:2376] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22. Re-sending same reply.
I20260709 14:54:48.567667  9913 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:48.571390  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.559075  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.577266 10009 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:48.577701  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 82181.1us
I20260709 14:54:48.574266  9743 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 14:54:48.437795 10119 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.472s	user 0.000s	sys 0.002s
I20260709 14:54:48.509052  9628 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.508522  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:48.534737 10098 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:48.542501  3542 rpcz_store.cc:276] 0709 14:54:47.336083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.540941 (+1204858us) service_pool.cc:224] Handling call
0709 14:54:48.541380 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.541597 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.541628 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.541705 (+    77us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:48.541742 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.541798 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.541916 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.367892  9930 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.568804  3547 rpcz_store.cc:276] 0709 14:54:46.918030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.195401 (+1277371us) service_pool.cc:224] Handling call
0709 14:54:48.565127 (+369726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.565923 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.568144 (+  2221us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540009b850. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.568305 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835454336}
I20260709 14:54:48.578781  9616 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:54:48.534164  3537 rpcz_store.cc:276] 0709 14:54:47.250956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.532727 (+1281771us) service_pool.cc:224] Handling call
0709 14:54:48.533121 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.533349 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.533375 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.533422 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:48.533454 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.533513 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.533638 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.589864 10118 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.575788  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:48.698060 10113 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.096s	user 0.002s	sys 0.004s
I20260709 14:54:48.592754 10120 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:48.594146  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:48.598443  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.602399  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:48.584055  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42502 (request call id 1867) took 1513 ms (client timeout 2004 ms). Trace:
I20260709 14:54:48.586390  2611 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:48.460973  2603 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
W20260709 14:54:48.710883  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.711486  2603 pending_rounds.cc:85] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 10
I20260709 14:54:48.711848  2603 pending_rounds.cc:107] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 14:54:48.514488  2751 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:48.712127  2603 raft_consensus.cc:2889] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:48.581825  3540 rpcz_store.cc:276] 0709 14:54:46.531150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:46.534975 (+  3825us) service_pool.cc:224] Handling call
0709 14:54:46.535469 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:47.619278 (+1083809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.415274 (+795996us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400318154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:48.415510 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020715392}
W20260709 14:54:48.538722  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:48.713097  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 174373us
W20260709 14:54:48.714049  2724 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:48.628209  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 119791us
I20260709 14:54:48.606475  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.559182 10068 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:48.678664  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1356) took 1213 ms. Trace:
W20260709 14:54:48.679958  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.634428  9941 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.581728  9858 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:48.604468  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.684051 10081 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:48.636883 10114 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:48.699461  2758 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:48.627281 10066 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:48.734802  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 140720us
I20260709 14:54:48.690446 10028 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:48.705435  2759 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:48.592134  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:48.699168 10119 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.070s	user 0.004s	sys 0.000s
I20260709 14:54:48.739588  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.585342  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.716058  2753 rpcz_store.cc:276] 0709 14:54:47.465125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.543733 (+ 78608us) service_pool.cc:224] Handling call
0709 14:54:47.610525 (+ 66792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.503292 (+892767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.634271 (+130979us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400570654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:48.678566 (+ 44295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634800512}
W20260709 14:54:48.710697  2754 rpcz_store.cc:269] 0709 14:54:47.070536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.079692 (+  9156us) service_pool.cc:224] Handling call
0709 14:54:48.583757 (+1504065us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:48.583964 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3045561600}
I20260709 14:54:48.628173  2677 rpcz_store.cc:276] 0709 14:54:47.346031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.021887 (+675856us) service_pool.cc:224] Handling call
0709 14:54:48.551050 (+529163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.728906  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:48.729655  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 443973us
I20260709 14:54:48.732339  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:48.732985  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.728096  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.743717  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.737582  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:48.745695  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:48.707525  9744 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:48.724740  9901 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:48.735399  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:48.749989  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.750783  2753 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:48.751873 10108 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:48.752377 10108 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:48.752513  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1868) took 1625 ms. Trace:
I20260709 14:54:48.752853 10108 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:48.752918  2752 rpcz_store.cc:276] 0709 14:54:47.127342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.223062 (+ 95720us) service_pool.cc:224] Handling call
0709 14:54:47.258981 (+ 35919us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:47.322181 (+ 63200us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:47.599391 (+277210us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.881175 (+281784us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:47.924651 (+ 43476us) log.cc:844] Serialized 52 byte log entry
0709 14:54:47.924912 (+   261us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:47.925260 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.925654 (+   394us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.925829 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:47.925936 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.430383 (+504447us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b1800376840. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:48.436949 (+  6566us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:48.681314 (+244365us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:54:48.752265 (+ 70951us) raft_consensus.cc:1685] finished
0709 14:54:48.752290 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.752451 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59969,"prepare.queue_time_us":607323,"prepare.run_cpu_time_us":189,"prepare.run_wall_time_us":15220,"raft.queue_time_us":59472,"raft.run_cpu_time_us":4211,"raft.run_wall_time_us":772959,"spinlock_wait_cycles":142387456,"thread_start_us":445539,"threads_started":2,"wal-append.queue_time_us":345991}
I20260709 14:54:48.747529 10066 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.756666 10098 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.758291 10066 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:48.759040  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1379) took 1397 ms. Trace:
I20260709 14:54:48.759490  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:48.734939 10081 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.760329  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.799881  2759 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:48.761387  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.762283  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.748207 10028 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:48.803814 10028 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:48.804669 10081 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:48.771014  9761 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.774379  2605 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:48.805858  9761 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.746547 10068 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 14:54:48.724243  9383 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:48.784070  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49645.9us
I20260709 14:54:48.827307  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.747818  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:48.787082  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.727890 10062 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 14:54:48.788792 10083 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:48.791180  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:48.844421  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96674.8us
I20260709 14:54:48.845409  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.846280  2758 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:48.784901  9855 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:48.792766  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47192.9us
I20260709 14:54:48.847090 10128 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:48.784901  9913 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.848618  9913 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.849052 10009 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.795647  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.849215  9901 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 14:54:48.852154  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.852900  9901 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:48.853343  9901 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:48.853673  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.728169  9937 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 14:54:48.796988  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:48.796588  2680 rpcz_store.cc:276] 0709 14:54:47.341331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.341562 (+   231us) service_pool.cc:224] Handling call
0709 14:54:47.341998 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:47.396314 (+ 54316us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:47.479560 (+ 83246us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.777527 (+297967us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:47.800479 (+ 22952us) log.cc:844] Serialized 26 byte log entry
0709 14:54:47.804537 (+  4058us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:47.804913 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 14:54:47.805147 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:47.805258 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.511783 (+706525us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:48.586426 (+ 74643us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b240021b9d8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b597fb0 00007fa16e20e488 00007fa16e20ebe8 00007fa16e1e6398 00007fa16e1e176a 00007fa16e1e0bb6
0709 14:54:48.738492 (+152066us) raft_consensus.cc:1685] finished
0709 14:54:48.738518 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.738713 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41752,"prepare.queue_time_us":243150,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":68903,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":28917,"spinlock_wait_cycles":99544320,"thread_start_us":237256,"threads_started":2,"wal-append.queue_time_us":391435}
I20260709 14:54:48.802994  9868 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:48.779423  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:48.816704  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57351us
I20260709 14:54:48.875303  2760 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:48.820417  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.801084  2756 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:48.821471  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.824496  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1357) took 1300 ms. Trace:
I20260709 14:54:48.749238  9628 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:48.827979  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:48.801793  9744 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:48.856575  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.856673  3538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1916) took 1907 ms. Trace:
I20260709 14:54:48.860617  2607 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 20
I20260709 14:54:48.788408 10058 consensus_queue.cc:1048] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 14:54:48.878104  2757 rpcz_store.cc:276] 0709 14:54:47.523704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:47.625119 (+101415us) service_pool.cc:224] Handling call
0709 14:54:47.668272 (+ 43153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.329799 (+661527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.824172 (+494373us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b5400184cd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:48.824406 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510672768}
I20260709 14:54:48.892573  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.893786  2754 raft_consensus.cc:2376] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21. Re-sending same reply.
I20260709 14:54:48.896799  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:48.897209  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 95367.8us
W20260709 14:54:48.897826  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69910.2us
I20260709 14:54:48.896920  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.842283 10052 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:48.907440  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:48.844085 10083 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:48.912678  2755 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:48.913717  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.916527  2757 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:48.917531  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:48.919322  2757 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:48.920202  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1874) took 1298 ms. Trace:
I20260709 14:54:48.920615  2757 rpcz_store.cc:276] 0709 14:54:47.621705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.918470 (+1296765us) service_pool.cc:224] Handling call
0709 14:54:48.918927 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.919791 (+   864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.920093 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.922269  2757 raft_consensus.cc:1240] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:48.923274  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1359) took 1228 ms. Trace:
W20260709 14:54:48.924209  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.924041 10143 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:48.925065  2757 rpcz_store.cc:276] 0709 14:54:47.695408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.921524 (+1226116us) service_pool.cc:224] Handling call
0709 14:54:48.922049 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.922778 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.923203 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:48.927928  3536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37160 (request call id 1915) took 1995 ms. Trace:
I20260709 14:54:48.928392  3536 rpcz_store.cc:276] 0709 14:54:46.933129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.426329 (+1493200us) service_pool.cc:224] Handling call
0709 14:54:48.426794 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:48.447822 (+ 21028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:48.447862 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:48.508147 (+ 60285us) log.cc:844] Serialized 26 byte log entry
0709 14:54:48.895406 (+387259us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:48.895482 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.895596 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:48.927670 (+ 32074us) raft_consensus.cc:1685] finished
0709 14:54:48.927695 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.927872 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28576,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":299780,"threads_started":1,"wal-append.queue_time_us":338660}
I20260709 14:54:48.854341  9901 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:48.885710  9858 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:48.884245  9868 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:48.930629  2751 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:48.882638 10098 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:48.938328  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 189167us
I20260709 14:54:48.940670  2758 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:48.971467  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58056.1us
I20260709 14:54:48.973394  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:48.985307  2760 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16.
I20260709 14:54:48.992420  2757 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 15. Current term is 16. Ops: []
I20260709 14:54:48.994478  2758 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:48.931010 10101 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:48.930848 10083 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:49.011269  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:49.029961  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:48.954802  9868 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:49.034005  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 187488us
I20260709 14:54:48.930555  9383 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 14:54:48.912680 10055 consensus_queue.cc:1059] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 14:54:49.046720  2754 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:48.916256 10052 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.020143 10101 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:48.983052 10098 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.029119  9913 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:48.940186  9951 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 14:54:49.035512 10068 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:48.974608  9858 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:49.062937  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:49.075424  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 176819us
I20260709 14:54:49.063320  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.064270  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.031841  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 147161us
I20260709 14:54:49.077823  2679 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:49.080380  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.082540  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.082571  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.083969  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.084652  2683 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:49.084926  2684 raft_consensus.cc:2376] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21. Re-sending same reply.
I20260709 14:54:49.089424  2603 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 14:54:49.086664  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.115448  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 208773us
I20260709 14:54:49.161715 10124 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:54:49.164906 10124 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:54:49.147292  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104939us
W20260709 14:54:49.174751  2577 leader_election.cc:336] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.307s (SENT)
I20260709 14:54:49.095865  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.176124  2677 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:49.103017  2605 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 14:54:49.056922  9937 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:49.144372 10145 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:49.071252 10142 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:54:49.176834  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:49.049788  9901 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:49.101315  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.125665  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.103530  2756 raft_consensus.cc:2376] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:49.124611  9849 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 14:54:49.131832  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 158537us
I20260709 14:54:49.100764 10120 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:49.124635  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.094678  9913 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 14:54:49.118028  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.117216  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.102260  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.033257  9563 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 14:54:49.099527  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.086229  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:49.193568  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 107619us
I20260709 14:54:49.194619  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.054479 10101 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:49.183491  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.185122  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.187222  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:49.188401  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.202479  2677 raft_consensus.cc:2393] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:49.184520  2686 raft_consensus.cc:2376] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:49.203580  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.204730 10147 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:49.205513 10147 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:49.205667 10145 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:49.206120 10145 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:49.206771 10147 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.207226 10145 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.208712 10145 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:49.210906  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.211460  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.212594  2685 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:49.212728  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.211856 10149 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 14:54:49.215569 10149 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 14:54:49.216034 10149 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:49.211788  9858 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:49.192660  2680 raft_consensus.cc:2376] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19. Re-sending same reply.
I20260709 14:54:49.236909  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.187390  9849 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:49.253855  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.255443  2681 raft_consensus.cc:2376] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23. Re-sending same reply.
I20260709 14:54:49.262672  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.268347  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.270264  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.279232  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 116664us
I20260709 14:54:49.279925  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.281102  2609 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:54:49.210911 10052 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:49.237533 10058 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:49.288964  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.183777  9937 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.261402  9849 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:49.295050  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.294725  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.291042  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.301962  2678 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:49.267619 10124 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:49.187865 10142 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.322108  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.322381  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.315665  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:49.330359  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 156685us
W20260709 14:54:49.346721  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.354163  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.361327  2681 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 14:54:49.361130  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.363751  2605 raft_consensus.cc:2393] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 16: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:49.365018  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.327668  9913 consensus_queue.cc:1243] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:49.372679 10168 consensus_queue.cc:1059] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:54:49.385694  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 154480us
I20260709 14:54:49.386373  2608 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 14:54:49.387087  2608 pending_rounds.cc:85] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 11
I20260709 14:54:49.387477  2608 pending_rounds.cc:107] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 14:54:49.387946  2608 raft_consensus.cc:2889] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:49.387238  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:49.403034  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.403378  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.404251  2605 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
W20260709 14:54:49.404124  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 55469.4us
I20260709 14:54:49.405325  2680 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:54:49.408538  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:54:49.412258  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:49.416498  2607 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.002s
I20260709 14:54:49.418679  2607 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:49.420805  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.421667  2605 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:54:49.239037 10149 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:49.426162  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.374632 10098 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:49.430748  2607 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:49.433341  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.336460 10124 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:49.438602  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.440307  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1370) took 1214 ms. Trace:
I20260709 14:54:49.415762 10052 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:49.444931  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.440871  2751 rpcz_store.cc:276] 0709 14:54:48.225561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.943952 (+718391us) service_pool.cc:224] Handling call
0709 14:54:48.995694 (+ 51742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.005707 (+ 10013us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:49.199379 (+193672us) log.cc:844] Serialized 14 byte log entry
0709 14:54:49.439736 (+240357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:49.439840 (+   104us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:49.439954 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.440074 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:49.440236 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1378,"prepare.queue_time_us":203944,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":43,"thread_start_us":303473,"threads_started":2}
I20260709 14:54:49.448845  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.457357  2575 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.431887 10145 raft_consensus.cc:3055] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:49.459834  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:49.465749  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.445677 10052 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:49.471580  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:49.475047  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:49.476471 10163 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 14:54:49.476773  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.477041  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.477592  2608 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 20.
I20260709 14:54:49.443707 10166 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 14:54:49.481164 10166 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:49.481849 10166 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.483245 10175 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 14:54:49.483433  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54747.2us
I20260709 14:54:49.484539 10175 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:49.484758  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:49.485188 10175 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:49.485558 10159 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 14:54:49.485882 10058 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:49.486429 10058 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:49.486866 10058 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:49.473625 10052 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:49.487697  9937 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:49.497776 10028 consensus_meta.cc:287] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.002s
I20260709 14:54:49.474857  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:49.508100  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.463995  9913 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:49.464288 10149 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 14:54:49.528959  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 191656us
W20260709 14:54:49.534121  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.533706  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.541260  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:49.461328 10145 raft_consensus.cc:740] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:49.547713  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:49.549866  2611 raft_consensus.cc:2410] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 14:54:49.518915 10108 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.765s	user 0.012s	sys 0.004s
I20260709 14:54:49.552084  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.553195  2607 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:54:49.484887 10142 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:49.562458  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:49.570760  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.464622 10143 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 14:54:49.544870 10128 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.697s	user 0.016s	sys 0.000s
I20260709 14:54:49.590191  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:49.583256  9937 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:49.526454 10028 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.583904 10164 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:49.589946 10124 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:49.618466  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:49.599252 10128 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:49.580819 10108 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.486690 10175 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:49.655318  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52315.2us
W20260709 14:54:49.661856  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.620857 10164 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:49.669965  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:49.673029  2607 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:49.605490 10147 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 14:54:49.675186  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.683818 10147 raft_consensus.cc:3055] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:49.684078 10142 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:49.675772  2679 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:49.684875 10142 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:49.666512 10124 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:49.687407 10124 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:49.688688 10124 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.665288 10171 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:49.694603 10171 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:49.696066 10171 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.684397 10147 raft_consensus.cc:740] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:49.702414  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84395.1us
I20260709 14:54:49.704362  2607 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:54:49.703519  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:49.705524  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 164385us
I20260709 14:54:49.709661  2575 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.709694  2611 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:49.610045 10145 consensus_queue.cc:260] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.713441  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:49.713858  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1385) took 1214 ms. Trace:
I20260709 14:54:49.714555  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.677300 10164 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:49.723636  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.725451  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:49.714859  9868 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.003s
I20260709 14:54:49.728951  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.727008  2575 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:49.686337 10142 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.716006 10143 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:49.713251  9616 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:49.735282  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.738435  2681 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:49.711392  9913 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:49.723506 10128 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:49.725761 10181 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:49.743806  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.742223  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.748186  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.749830  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.751662  2680 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:49.737277 10083 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.698s	user 0.006s	sys 0.009s
I20260709 14:54:49.730639  9744 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:49.755101 10083 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:49.755759 10179 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 14:54:49.758014 10083 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:49.731148  9868 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:49.761251  9868 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:49.762140  2759 raft_consensus.cc:2368] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22.
I20260709 14:54:49.733846  9937 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:49.722059 10180 consensus_queue.cc:1059] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 14:54:49.765657  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:49.768155  2576 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.766566 10083 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:49.772915  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:49.775617 10098 raft_consensus.cc:2764] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 14:54:49.858090  2753 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:49.778254  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:49.766563 10181 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:49.779855 10108 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:49.783160  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1358) took 2153 ms. Trace:
I20260709 14:54:49.783758  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:49.784751  9901 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.005s
I20260709 14:54:49.789950  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:49.871739  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103643us
I20260709 14:54:49.740130 10186 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:49.798799  9913 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:49.799893  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1380) took 1303 ms. Trace:
I20260709 14:54:49.786185 10120 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 14:54:49.793006  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.812175  2603 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 14:54:49.795446  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1360) took 1996 ms. Trace:
I20260709 14:54:49.812155  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.908919  2757 rpcz_store.cc:276] 0709 14:54:47.799459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.926863 (+1127404us) service_pool.cc:224] Handling call
0709 14:54:48.992201 (+ 65338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.992953 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.795193 (+802240us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b54000e6350. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:49.795372 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149262080}
I20260709 14:54:49.910892  2757 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: [18.12-21.13]
I20260709 14:54:49.911491  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2019) took 1391 ms. Trace:
I20260709 14:54:49.911708  2757 rpcz_store.cc:276] 0709 14:54:48.520184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.910193 (+1390009us) service_pool.cc:224] Handling call
0709 14:54:49.910556 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:49.911247 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.911437 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:49.912726  2757 raft_consensus.cc:1240] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:49.913390  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2020) took 1392 ms. Trace:
I20260709 14:54:49.913846  2757 rpcz_store.cc:276] 0709 14:54:48.521168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.912267 (+1391099us) service_pool.cc:224] Handling call
0709 14:54:49.912580 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.913174 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.913343 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:49.912545  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 134388us
I20260709 14:54:49.731953  9849 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.915266  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:49.915823  2684 raft_consensus.cc:2368] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21.
I20260709 14:54:49.806456  2680 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:49.869567  2754 rpcz_store.cc:276] 0709 14:54:47.649685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.919200 (+1269515us) service_pool.cc:224] Handling call
0709 14:54:48.919714 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:49.047562 (+127848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.782772 (+735210us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002454d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:49.783078 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291185408}
I20260709 14:54:49.917033  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:49.785742  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:49.915912  2681 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
W20260709 14:54:49.873812  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.917853  2723 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:49.876514  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:49.876955  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:49.770527  9868 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 14:54:49.904817  2755 rpcz_store.cc:276] 0709 14:54:48.496704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.746092 (+249388us) service_pool.cc:224] Handling call
0709 14:54:49.799589 (+1053497us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54003f11d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:49.799801 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252530560}
W20260709 14:54:49.919274  2724 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:49.801043 10172 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.208s	user 0.001s	sys 0.001s
I20260709 14:54:49.919386  2723 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.928812  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.929922  2680 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
I20260709 14:54:49.918416  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.921749  2612 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:49.770787 10145 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:49.924294  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:49.913295 10200 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:54:49.927495  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:49.770380 10182 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:49.919312  2753 raft_consensus.cc:2468] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:49.795398 10128 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:49.931883  2757 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:54:49.947706  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1387) took 1415 ms. Trace:
I20260709 14:54:49.948211  2752 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:49.948282  2757 rpcz_store.cc:276] 0709 14:54:48.532661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.914541 (+1381880us) service_pool.cc:224] Handling call
0709 14:54:49.928097 (+ 13556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.947272 (+ 19175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.947625 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:49.949790  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.800990  9937 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:49.933254  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:49.941606  2754 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:49.768283  9744 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.944844  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:49.931965  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58346.6us
W20260709 14:54:49.961475  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.866103 10058 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:49.963735  2753 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:49.863685 10083 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:49.961982  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.964644  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2023) took 1281 ms. Trace:
I20260709 14:54:49.968735  2753 rpcz_store.cc:276] 0709 14:54:48.683504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.946872 (+1263368us) service_pool.cc:224] Handling call
0709 14:54:49.963587 (+ 16715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.964363 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.964586 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:49.966388  2757 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 17. Current term is 18. Ops: []
W20260709 14:54:49.969497  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.970032  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:49.971246  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2024) took 1287 ms. Trace:
I20260709 14:54:49.944209 10200 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:49.971738  2757 rpcz_store.cc:276] 0709 14:54:48.684071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.950123 (+1266052us) service_pool.cc:224] Handling call
0709 14:54:49.965170 (+ 15047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.969878 (+  4708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.971166 (+  1288us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:49.973261  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:49.939951 10198 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:49.974803 10181 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.004158  2753 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:50.005257  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2026) took 1305 ms. Trace:
I20260709 14:54:50.005638  2753 rpcz_store.cc:276] 0709 14:54:48.700104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.969635 (+1269531us) service_pool.cc:224] Handling call
0709 14:54:50.003890 (+ 34255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.004782 (+   892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.005179 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:49.983255 10194 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:49.957679 10182 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.007470 10194 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:49.983844  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.008841  9744 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.009002  2681 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.010938  2604 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:50.011057  2752 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:50.010236 10182 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.012001  9868 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:50.012157  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2025) took 1327 ms. Trace:
I20260709 14:54:50.012655  9868 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:50.012774  2752 rpcz_store.cc:276] 0709 14:54:48.684603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.957536 (+1272933us) service_pool.cc:224] Handling call
0709 14:54:49.976173 (+ 18637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.011711 (+ 35538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.012058 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:49.958719 10128 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.014019  9868 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:50.024609 10192 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.229s	user 0.003s	sys 0.001s
W20260709 14:54:49.992419 10101 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.753s	user 0.015s	sys 0.000s
I20260709 14:54:49.992659  9937 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.035390  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2034) took 1297 ms. Trace:
I20260709 14:54:50.036259  2753 rpcz_store.cc:276] 0709 14:54:48.737650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.006507 (+1268857us) service_pool.cc:224] Handling call
0709 14:54:50.031801 (+ 25294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.032011 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:50.032036 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.032083 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:50.032141 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.032220 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:50.032363 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:49.998965 10200 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:49.999269 10199 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:54:49.999481 10197 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:50.000943  9849 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:50.016706 10128 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:49.975349 10124 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:50.000991 10052 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.513s	user 0.014s	sys 0.000s
I20260709 14:54:50.022078  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2027) took 1317 ms. Trace:
I20260709 14:54:49.985886 10083 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:49.992661 10058 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:49.987176 10120 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:50.031703  2678 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:54:50.064735  2753 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:50.068209  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:49.997505 10164 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:50.042096  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 256471us
I20260709 14:54:50.058382  2681 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
W20260709 14:54:50.060470  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.026567 10186 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:50.065955  2757 rpcz_store.cc:276] 0709 14:54:48.704798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.976309 (+1271511us) service_pool.cc:224] Handling call
0709 14:54:50.021200 (+ 44891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.021626 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:50.021661 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.021740 (+    79us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:50.021777 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.021846 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:50.022001 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5655,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":14544}
I20260709 14:54:50.070247  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:50.050558 10199 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:50.041214 10142 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:50.077863  2678 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 14:54:50.070956 10149 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.000s
I20260709 14:54:50.050323  9937 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:50.098009  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 72845.3us
I20260709 14:54:50.096400  2605 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:50.048233 10101 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.075321  9616 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:50.079196 10083 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:50.115525  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 196241us
I20260709 14:54:50.078513 10098 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.078960 10120 raft_consensus.cc:697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:50.121891  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:50.092669 10199 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:54:50.126463  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 336631us
W20260709 14:54:50.128237  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.072511 10052 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.115767 10185 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:50.133265  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:50.099695 10142 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:50.141829  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 224960us
I20260709 14:54:50.143777  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.143302 10185 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.129683 10098 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.107705  9937 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:50.156518  2754 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 14:54:50.157778  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1388) took 1624 ms. Trace:
I20260709 14:54:50.147287  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:50.158286  2754 rpcz_store.cc:276] 0709 14:54:48.533070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.917304 (+1384234us) service_pool.cc:224] Handling call
0709 14:54:49.941381 (+ 24077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.157335 (+215954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.157685 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:54:50.161883 10195 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:54:50.171526 10164 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.184230 10114 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.188306 10142 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.191187 10101 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.192341 10098 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.167784 10083 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:50.205909  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62309.9us
I20260709 14:54:50.208310  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:50.209794  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 62596.2us
I20260709 14:54:50.206446 10098 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:50.190433 10200 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.212991 10098 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:50.213505 10098 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:50.196235 10194 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:50.216523 10200 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.212232  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:50.221428  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.225203  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.225636 10127 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:50.228533  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.236313  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.236012 10194 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:50.196276 10120 consensus_queue.cc:237] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.241556 10204 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:54:50.223167 10175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.009s
I20260709 14:54:50.247607  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.258420 10214 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:50.233153 10213 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:50.260627  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:50.243873 10175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.002s	sys 0.009s
I20260709 14:54:50.236637  9628 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:50.263235  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141631us
I20260709 14:54:50.266593  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.258474 10052 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.262450 10214 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:50.258876 10083 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.272135 10215 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:50.272495  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.272835 10215 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:50.273265 10215 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:50.273380  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1390) took 1199 ms. Trace:
I20260709 14:54:50.273841  2677 rpcz_store.cc:276] 0709 14:54:49.073689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.213782 (+140093us) service_pool.cc:224] Handling call
0709 14:54:50.272365 (+1058583us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521eee 000000000037fce9 00007fa1735951e0 00007fa17365e0c4 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:50.273311 (+   946us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13446528}
I20260709 14:54:50.281355  9628 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.282436  9628 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:50.283900  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55522.7us
I20260709 14:54:50.284672  2754 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 14:54:50.303133  2755 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:50.291023 10217 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:54:50.309327  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.294064  9761 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:50.298375  2759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1396) took 1715 ms (client timeout 2134 ms). Trace:
I20260709 14:54:50.288678 10213 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:50.298945 10204 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.304553  2677 raft_consensus.cc:1240] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 25. Current term is 26. Ops: [25.12-25.12]
I20260709 14:54:50.323702  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:50.314311  3411 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277): Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 2.134s (SENT)
I20260709 14:54:50.315146  2678 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 14:54:50.315989  2759 rpcz_store.cc:269] 0709 14:54:48.582807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.808162 (+225355us) service_pool.cc:224] Handling call
0709 14:54:50.298121 (+1489959us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b540040ecd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:50.298299 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032950528}
I20260709 14:54:50.319772  2752 raft_consensus.cc:1275] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:50.329219  2677 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:50.332494  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.334793  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.336369  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.337942  2677 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:50.341086  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.337772  2678 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:50.349200  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.353091 10028 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.357236  9628 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:50.359102  2679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1400) took 1177 ms (client timeout 1562 ms). Trace:
W20260709 14:54:50.359639  2679 rpcz_store.cc:269] 0709 14:54:49.181547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.186502 (+  4955us) service_pool.cc:224] Handling call
0709 14:54:50.358791 (+1172289us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:50.359005 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083405568}
I20260709 14:54:50.361335  2679 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: [18.12-21.13]
I20260709 14:54:50.361533 10215 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.327705 10142 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 14:54:50.321458 10052 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:50.371145  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:50.328694 10197 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:50.327137 10091 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:54:50.391856  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 88840.6us
I20260709 14:54:50.379522  9761 consensus_queue.cc:1059] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 14:54:50.364715 10181 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 14:54:50.309465 10199 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.387224 10171 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:50.315366 10214 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 14:54:50.413141  2677 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:50.441183  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.375401  9563 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:50.420132  3409 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.523s (SENT)
I20260709 14:54:50.459476  2575 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:50.394908 10181 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:50.394407  9761 raft_consensus.cc:3055] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:54:50.459731  2677 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:50.335207 10213 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.413697 10171 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:50.393322 10215 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.408231 10083 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.378114 10175 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 14:54:50.387388 10145 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
W20260709 14:54:50.463282  3409 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.882s (SENT)
I20260709 14:54:50.407296 10091 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.483523  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.456393 10191 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.331908 10204 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.485234 10191 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.486734 10188 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.487025 10204 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.487793  2754 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:50.484558 10052 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 14:54:50.492480   431 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 3523)
I20260709 14:54:50.487504 10188 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:50.493319 10145 consensus_queue.cc:1243] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:50.495239 10188 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.495903 10169 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.496603 10169 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:50.479247 10171 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.500505 10213 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:50.501158 10199 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:50.500586  2756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2028) took 1791 ms (client timeout 2307 ms). Trace:
W20260709 14:54:50.502156  2756 rpcz_store.cc:269] 0709 14:54:48.708938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.877609 (+168671us) service_pool.cc:224] Handling call
0709 14:54:50.500360 (+1622751us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002058d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:50.500506 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448952704}
I20260709 14:54:50.503708  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:50.504554 10091 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.505492  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1395) took 1391 ms. Trace:
I20260709 14:54:50.505308 10098 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.505798  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.507375  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.508389  2759 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
W20260709 14:54:50.510428  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.483750 10083 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:50.473431  9761 raft_consensus.cc:740] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:50.507787 10169 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:50.505908  2684 rpcz_store.cc:276] 0709 14:54:49.114619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.210559 (+ 95940us) service_pool.cc:224] Handling call
0709 14:54:50.505261 (+1294702us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540022c254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:50.505423 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618908416}
W20260709 14:54:50.513856 10189 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.744s	user 0.000s	sys 0.002s
I20260709 14:54:50.514645  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.517609 10191 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.510290  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.472851 10181 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:50.516119  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.519325 10126 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:50.515769  2575 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Timed out: RequestConsensusVote RPC to 127.0.107.195:41829 timed out after 1.926s (SENT)
I20260709 14:54:50.520404  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:50.521534 10098 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.504614  2756 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25.
I20260709 14:54:50.519999  2680 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:50.524850  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.525365  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.526240  2680 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.519963 10181 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.526767 10150 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:50.517457  2684 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:50.519361  2759 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:50.535151  2751 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:50.521396  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.473661 10164 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:50.537339  2753 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
W20260709 14:54:50.539944  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.534001  2677 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:54:50.547119  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.547850  2611 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:50.527602 10181 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:50.547768  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:50.553928  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.549896  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.551656  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.554898  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.555572  2677 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:50.554901  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.556720  2604 raft_consensus.cc:2393] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:50.557649  2607 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:50.556805  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:50.558988  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 256420us
I20260709 14:54:50.559352  2759 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25.
I20260709 14:54:50.560066  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.556766 10188 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:50.559221  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.561587  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.519982 10126 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:50.564934  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.566885  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.567585  2612 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:50.569823  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:48.886093  3538 rpcz_store.cc:276] 0709 14:54:46.949654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:48.474311 (+1524657us) service_pool.cc:224] Handling call
0709 14:54:48.474767 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:48.500449 (+ 25682us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:48.546695 (+ 46246us) log.cc:844] Serialized 14 byte log entry
0709 14:54:48.856290 (+309595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:48.856354 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:48.856412 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:48.856484 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:48.856610 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21850,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":641,"raft.run_cpu_time_us":965,"raft.run_wall_time_us":26353,"thread_start_us":308481,"threads_started":1}
I20260709 14:54:50.566423 10127 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:50.575492 10126 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:50.576447  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.577198  2686 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.578686  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.581337  2684 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:50.581990  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:50.556978 10164 raft_consensus.cc:697] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:50.583139  3409 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:50.583243  2609 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:50.585021  2609 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:50.587250  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:54:50.589740  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.565554 10098 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:50.591967  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.592605  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:50.577198 10188 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:50.580225 10213 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.512807  9761 consensus_queue.cc:260] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.587360 10068 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 14:54:50.593381 10098 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:50.593549  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:50.624084  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2074) took 1554 ms. Trace:
I20260709 14:54:50.602661  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.599504  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:50.581302 10127 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:50.596745  2609 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.591602 10126 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:50.627578  2759 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:50.595567  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.470901  9563 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.576270 10171 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.594343 10204 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:54:50.619537  2753 raft_consensus.cc:1275] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 14:54:50.619333 10164 consensus_queue.cc:237] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.594659 10052 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:54:50.621783 10188 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.591974 10124 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:50.620512  2680 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:50.699211 10124 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.700321  2680 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:50.701846 10124 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.702163  2754 raft_consensus.cc:1240] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:50.609568 10213 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.702924  2680 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:50.704492  2680 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: [17.10-21.12]
I20260709 14:54:50.704572 10213 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:50.706048  2679 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:54:50.604323 10120 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:50.707087  9563 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.589951  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:50.708348 10052 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:50.708447  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 118644us
I20260709 14:54:50.709051 10068 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:54:50.709671 10068 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:50.709945  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1406) took 1421 ms. Trace:
I20260709 14:54:50.709609 10052 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:50.710304  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:50.640599  2609 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.644611  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:50.651783  2677 raft_consensus.cc:1240] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:50.627054  2606 rpcz_store.cc:276] 0709 14:54:49.069772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.071794 (+  2022us) service_pool.cc:224] Handling call
0709 14:54:49.099306 (+ 27512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.099589 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:49.287420 (+187831us) log.cc:844] Serialized 14 byte log entry
0709 14:54:50.623494 (+1336074us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b1800420460. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287
0709 14:54:50.623612 (+   118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.623674 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:50.623735 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.623838 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:50.623989 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17107,"prepare.queue_time_us":33719,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":73277,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":11403,"spinlock_wait_cycles":10822272,"thread_start_us":311357,"threads_started":3,"wal-append.queue_time_us":173111}
I20260709 14:54:50.625869 10062 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:50.669528  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.733505  2753 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 14:54:50.690418  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:50.733757  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 89270.8us
I20260709 14:54:50.693605  9941 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.695212  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.627895 10098 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.593683 10083 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.713415  2677 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 14:54:50.765208  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.767907  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.770260  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.731905  2611 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:50.788671  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:50.753501 10055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.639s	user 0.017s	sys 0.017s
I20260709 14:54:50.675225  9937 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.734740  2686 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
W20260709 14:54:50.805089  2686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1853) took 1738 ms (client timeout 1926 ms). Trace:
I20260709 14:54:50.735376  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:50.741290  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.749078  2757 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:50.761255  2607 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:50.725165  2612 raft_consensus.cc:2468] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:50.773146 10166 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:50.774469  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 64234.4us
I20260709 14:54:50.809046  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:50.777989 10083 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:50.780857  2678 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 14:54:50.795303  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:50.812449  2758 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.812744  2678 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:50.628232 10167 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:54:50.790202  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:50.711052 10081 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.649520 10127 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.649415 10230 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:54:50.826227  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.859133  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:50.860733  2753 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:50.810187  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:50.805809  2686 rpcz_store.cc:269] 0709 14:54:49.066803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.218644 (+151841us) service_pool.cc:224] Handling call
0709 14:54:50.576221 (+1357577us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:50.804978 (+228757us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45073664}
I20260709 14:54:50.816296  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:50.799731 10055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.018s	sys 0.017s
I20260709 14:54:50.865003  2612 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:50.760834 10120 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:50.649981 10197 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.148s
I20260709 14:54:50.710443  2682 rpcz_store.cc:276] 0709 14:54:49.289140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.311230 (+ 22090us) service_pool.cc:224] Handling call
0709 14:54:49.347104 (+ 35874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.347466 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:49.604823 (+257357us) log.cc:844] Serialized 14 byte log entry
0709 14:54:50.513183 (+908360us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:50.709356 (+196173us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b180017f1a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287
0709 14:54:50.709563 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.709624 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:50.709685 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.709763 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:50.709883 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2528,"prepare.queue_time_us":152916,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":57008,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":141,"spinlock_wait_cycles":79867008,"thread_start_us":498521,"threads_started":3,"wal-append.queue_time_us":126145}
I20260709 14:54:50.784775 10213 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:50.817584 10214 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:50.710785 10068 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:50.834235  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 99055.1us
I20260709 14:54:50.858191  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.890595  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:50.764381 10062 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.795935 10175 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 14:54:50.913889 10062 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:50.879408 10214 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:50.896811 10127 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:50.862996  9913 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 14:54:50.871498  9937 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 14:54:50.915966  9937 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:50.916675  9937 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:50.865485 10201 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 14:54:50.919193 10201 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:50.919282 10171 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:50.919917 10201 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:50.879453 10197 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:50.879997 10213 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.871166 10210 consensus_queue.cc:1059] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 14:54:50.887857 10230 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:50.946350  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.947232 10230 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:50.855600 10124 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 14:54:50.905963  2754 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:50.855211 10149 consensus_queue.cc:1059] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
W20260709 14:54:50.962456  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.915477 10214 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:50.879884 10083 consensus_queue.cc:1059] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 14:54:50.894060  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:50.940410  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.972715  2757 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:50.974499  2757 raft_consensus.cc:1240] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:50.964150  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:50.976011  2612 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:50.976480  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:50.825139  9941 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 14:54:50.966332  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.967536  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.980749  2758 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:50.981703 10118 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:50.982280 10118 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:50.968855  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.987541  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:50.980847  2605 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:49.714265  2760 rpcz_store.cc:276] 0709 14:54:48.499740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.469381 (+969641us) service_pool.cc:224] Handling call
0709 14:54:49.513260 (+ 43879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:49.513624 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 14:54:49.540437 (+ 26813us) log.cc:844] Serialized 14 byte log entry
0709 14:54:49.712345 (+171908us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:49.712584 (+   239us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:49.712856 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:49.713647 (+   791us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:49.713797 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26499,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":4041,"raft.queue_time_us":2233,"thread_start_us":122093,"threads_started":1}
I20260709 14:54:50.991031  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1389) took 2457 ms. Trace:
I20260709 14:54:50.993319  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1810) took 1261 ms. Trace:
I20260709 14:54:50.995810 10154 raft_consensus.cc:3055] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:50.877770 10081 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:50.997025  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 488930us
I20260709 14:54:50.998652  2651 leader_election.cc:400] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25.
I20260709 14:54:50.999208 10081 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:50.999719  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:51.002254  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:51.002589  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 470248us
W20260709 14:54:51.004451  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 269666us
I20260709 14:54:51.005507  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.006183  9852 raft_consensus.cc:3055] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:54:51.002780 10193 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:51.008255 10166 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:51.008693 10166 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:51.006772  9852 raft_consensus.cc:740] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:51.009606 10166 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:51.011514  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:51.011826 10166 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:50.991694  2755 rpcz_store.cc:276] 0709 14:54:48.533406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.924219 (+1390813us) service_pool.cc:224] Handling call
0709 14:54:50.302934 (+378715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.303658 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.984490 (+680832us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540040ecd0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:50.990939 (+  6449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824636672}
W20260709 14:54:50.961222  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.005609  2650 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.002370 10191 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:51.013535  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52366.4us
I20260709 14:54:50.996457 10154 raft_consensus.cc:740] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:51.014406  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.013536  2650 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.014542  9852 consensus_queue.cc:260] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:51.015998  2753 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:51.014873  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.016987 10154 consensus_queue.cc:260] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:51.017805  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.018299 10154 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:50.990392  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1402) took 1789 ms (client timeout 1523 ms). Trace:
I20260709 14:54:51.019964  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:51.020646  2651 leader_election.cc:400] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25.
W20260709 14:54:51.019608  2683 rpcz_store.cc:269] 0709 14:54:49.200551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:49.235995 (+ 35444us) service_pool.cc:224] Handling call
0709 14:54:50.987278 (+1751283us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17360c0a0 00007fa17360b984 00007fa16cb866d4 00007fa16cb86662
0709 14:54:50.990278 (+  3000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5730560}
I20260709 14:54:50.993824  2681 rpcz_store.cc:276] 0709 14:54:49.732152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.071879 (+339727us) service_pool.cc:224] Handling call
0709 14:54:50.072980 (+  1101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.074642 (+  1662us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:50.116005 (+ 41363us) log.cc:844] Serialized 14 byte log entry
0709 14:54:50.472964 (+356959us) log.cc:844] Serialized 14 byte log entry
0709 14:54:50.992692 (+519728us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b18004476a0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:54:50.992912 (+   220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.992973 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:50.993024 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:50.993101 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:50.993248 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2505,"prepare.queue_time_us":56666,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":156,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5017,"spinlock_wait_cycles":30653824,"thread_start_us":356210,"threads_started":1,"wal-append.queue_time_us":461299}
I20260709 14:54:51.026173  2612 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:51.027257  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:51.031404  2605 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 26
I20260709 14:54:50.970315  2609 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
W20260709 14:54:51.034983  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.033144  2609 pending_rounds.cc:85] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 12
I20260709 14:54:51.036031  2609 pending_rounds.cc:107] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 14:54:51.035852  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.040428  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:51.046512  2612 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:51.036895  2609 raft_consensus.cc:2889] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:51.053315  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:50.939322 10197 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:51.056310  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 162359us
W20260709 14:54:51.056191  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45879.9us
W20260709 14:54:51.057106  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.099066  2609 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:51.099975  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.057724  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50903.7us
W20260709 14:54:51.101024  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.059069  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.107501  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48490.5us
I20260709 14:54:51.107872  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.108693  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:51.062456  2753 raft_consensus.cc:1240] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:51.109925  2612 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:51.111102  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:50.918193  9937 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.069854  2758 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:51.117213  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.033128  9913 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:51.089295  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 87141us
W20260709 14:54:51.058869  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 200202us
I20260709 14:54:51.026710  9852 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:51.112141  2753 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:51.146178  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.114933  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.115247  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.117977  2758 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:51.072887  2757 raft_consensus.cc:1240] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:51.147951  2760 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:51.121621  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.122362  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.120404  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.124703  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.087106 10210 raft_consensus.cc:3055] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:51.085600  9849 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:51.107739 10217 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:51.063535  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:51.204733  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141342us
W20260709 14:54:51.205063  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 85957.2us
I20260709 14:54:51.206094  2723 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.149983  2683 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:51.150542  2753 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:51.105372 10062 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:51.203999  2678 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:51.209774 10062 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:51.104048 10157 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
W20260709 14:54:51.092548 10180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.008s	sys 0.032s
W20260709 14:54:51.211055 10180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.008s	sys 0.033s
I20260709 14:54:51.212697  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:51.214100  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.215237  2685 raft_consensus.cc:2393] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:51.219843  2683 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:51.219743  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.223757  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.225425  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:51.105681 10175 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.113s
I20260709 14:54:51.229970  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.230809  2755 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:51.231891  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:51.232542  2755 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:51.168058  9913 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:51.210530 10062 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.235349 10232 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:51.235805  9913 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:51.244540  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.249171  2755 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:51.237208  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:51.253794  2609 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:51.237841  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.255257  2683 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:51.256438  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:51.239887 10126 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.002s
I20260709 14:54:51.167923 10210 raft_consensus.cc:740] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:51.235880 10062 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:51.239925  2605 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:54:51.236150 10232 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.120203 10165 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:54:51.257248  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.250416  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.113338 10091 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 14:54:51.103678  9563 consensus_queue.cc:1048] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.110s
W20260709 14:54:51.265479  9855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.030s	sys 0.036s
I20260709 14:54:51.275731  2652 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.262246  2723 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.118391 10055 consensus_queue.cc:1048] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 14:54:51.251403 10204 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:51.218564  9858 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:54:51.295161  2680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.024s	sys 0.055s
I20260709 14:54:51.279181 10142 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:51.265846  9616 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:54:51.298128  2680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.024s	sys 0.055s
I20260709 14:54:51.289986  2609 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 14:54:51.277029  9937 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:51.303501  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.282544 10126 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:51.303592 10142 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:51.286772  9563 consensus_queue.cc:1243] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 14:54:51.266577 10180 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 14:54:51.283399  9913 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:51.266206  9941 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 14:54:51.289825  9855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.031s	sys 0.037s
W20260709 14:54:51.271939  2610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.025s	sys 0.025s
I20260709 14:54:51.294971 10149 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:51.302493 10166 raft_consensus.cc:2804] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:54:51.333436 10237 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 14:54:51.306916 10204 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:51.339389 10213 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:51.301386 10099 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:51.348882 10185 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:51.349503  2677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.024s	sys 0.062s
W20260709 14:54:51.397801  2677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.024s	sys 0.062s
I20260709 14:54:51.292573 10244 raft_consensus.cc:2804] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:51.342458 10171 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 14:54:51.361129  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.311072 10126 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:51.294591 10062 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 14:54:51.312913 10181 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:51.312094  9858 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:51.369379  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 246531us
I20260709 14:54:51.414851  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.280210 10197 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:51.303753 10210 consensus_queue.cc:260] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.355979 10150 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:51.363615 10199 raft_consensus.cc:3055] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:51.346050  9913 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:51.350328 10165 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 14:54:51.421820  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:51.395993 10166 raft_consensus.cc:697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:51.400903 10244 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:51.409796 10171 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:51.413386 10126 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:51.464694  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 356351us
I20260709 14:54:51.427196 10213 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:51.467638  2650 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.467993  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.431149 10214 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 14:54:51.471947  2650 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.455950  9913 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:51.433179 10199 raft_consensus.cc:740] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:51.462258 10210 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:51.457224 10165 consensus_queue.cc:1243] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:54:51.457922 10185 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:51.455848 10244 raft_consensus.cc:3060] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:51.491460  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:51.523142  9941 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:51.477810 10188 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 14:54:51.532325  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 406172us
W20260709 14:54:51.553946  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 62594.5us
W20260709 14:54:51.564100  2650 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.565260  2650 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:51.566545  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 446931us
W20260709 14:54:51.592348  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.478382 10213 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.456367 10142 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:51.569622  9941 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:51.595538  2650 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.606055  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:51.595067  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 341727us
W20260709 14:54:51.596097  2576 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.607928  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:51.608768  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:51.610498  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 321663us
W20260709 14:54:51.614897  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:51.607903  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.590116  9563 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 14:54:51.618479  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 492520us
I20260709 14:54:51.555017 10171 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:51.616863  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:51.626936  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:51.514928 10204 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.524139  9858 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.563336 10185 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:51.616869 10240 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
W20260709 14:54:51.628780 10238 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
W20260709 14:54:51.631064 10098 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.007s
I20260709 14:54:51.619730 10232 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:51.640271 10154 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.001s
I20260709 14:54:51.569291 10166 consensus_queue.cc:237] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:51.570026 10188 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:51.669390  2759 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: []
I20260709 14:54:51.645418 10124 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:51.647745 10167 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:51.659732 10243 log.cc:927] Time spent T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 14:54:51.608234 10199 consensus_queue.cc:260] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:51.676618 10193 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.010s	sys 0.010s
I20260709 14:54:51.668772 10188 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:51.712823  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 450704us
I20260709 14:54:51.724212  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.697284 10081 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:51.660614 10098 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:51.716645 10193 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.011s	sys 0.010s
I20260709 14:54:51.731218  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.748327  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:51.695518 10124 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:51.728289 10199 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:51.698995 10167 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:51.734189 10218 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:51.783216  2755 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 14:54:51.772969 10247 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 14:54:51.709892 10258 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
W20260709 14:54:51.780962 10246 log.cc:927] Time spent T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
W20260709 14:54:51.795514  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47300.4us
I20260709 14:54:51.736354  9855 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 14:54:51.779402 10230 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:51.780375  9849 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:51.780824 10263 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:51.793330 10241 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
W20260709 14:54:51.814097  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:51.815770  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 183150us
W20260709 14:54:51.831892 10252 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
W20260709 14:54:51.833565 10262 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:54:51.799587 10214 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:51.839843 10214 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:51.843268 10253 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 14:54:51.844324 10250 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 14:54:51.844759 10167 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.845870 10098 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:51.828913  9849 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.849138 10185 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.850752  9849 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.855640  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1416) took 1089 ms. Trace:
I20260709 14:54:51.860733  2754 rpcz_store.cc:276] 0709 14:54:50.766581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.769865 (+  3284us) service_pool.cc:224] Handling call
0709 14:54:51.855337 (+1085472us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400571554. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:51.855551 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268638976}
I20260709 14:54:51.856801  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:51.864223  2757 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 18.
I20260709 14:54:51.865160  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1891) took 1571 ms. Trace:
I20260709 14:54:51.865942  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1422) took 1330 ms. Trace:
I20260709 14:54:51.866187 10263 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.866554  2680 rpcz_store.cc:276] 0709 14:54:50.535702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.705351 (+169649us) service_pool.cc:224] Handling call
0709 14:54:50.705698 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:50.705946 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:50.705973 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:50.706608 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 14:54:51.305226 (+598618us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:51.305296 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:51.305560 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:51.865670 (+560110us) raft_consensus.cc:1685] finished
0709 14:54:51.865697 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:51.865870 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":593444,"threads_started":1,"wal-append.queue_time_us":782954}
I20260709 14:54:51.813493 10213 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.803949 10218 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:54:51.813122 10248 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
I20260709 14:54:51.865635  2759 rpcz_store.cc:276] 0709 14:54:50.294063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.663159 (+369096us) service_pool.cc:224] Handling call
0709 14:54:50.663566 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:51.670039 (+1006473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:51.864886 (+194847us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001851d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:51.865093 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312439552}
W20260709 14:54:51.828172 10251 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
W20260709 14:54:51.886021  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73502us
I20260709 14:54:51.821998  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1414) took 1056 ms. Trace:
I20260709 14:54:51.895644  2757 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 14:54:51.894728  2752 rpcz_store.cc:276] 0709 14:54:50.765861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.776849 (+ 10988us) service_pool.cc:224] Handling call
0709 14:54:50.777411 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:50.893825 (+116414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:50.893860 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:50.893909 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:50.893957 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:51.821600 (+927643us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54002765d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:51.821683 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:51.821870 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267067648}
I20260709 14:54:51.902679  2684 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 14:54:51.906010  2683 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 14:54:51.907480 10230 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:51.908381 10230 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:51.910166 10263 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.915359  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.822418  9941 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:51.906029  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.816805  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.919657  2686 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:51.920770  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.921573  2677 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:54:51.924072  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:51.925078  2686 raft_consensus.cc:2410] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 14:54:51.925547  2576 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.927129  2576 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.927732  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:51.903890 10124 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:51.929585  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.932304  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:51.930932  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.933692  2576 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.936143  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.940521  9616 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:51.941087  9616 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:51.941540  9616 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:51.947904  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.950235 10169 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:54:51.955226  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 138513us
I20260709 14:54:51.950294  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:51.932881 10149 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.568s	user 0.015s	sys 0.000s
I20260709 14:54:51.961239  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:54:51.966686  2576 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.972595  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:51.973724  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:51.953025 10230 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:51.979784  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:51.986159  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:51.959378 10258 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:51.958756 10180 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 14:54:51.991412  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.975135 10167 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:51.978909 10145 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:51.992264 10263 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:54:51.998101  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:51.980985 10149 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:51.990265 10142 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:51.979189 10052 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 14:54:51.994447 10270 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.002432 10180 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:52.043043 10256 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.043346 10271 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.043898 10256 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.044173 10271 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:51.994110 10081 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 14:54:52.024750 10145 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.053285 10145 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.043025 10277 consensus_queue.cc:1059] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 14:54:51.990960 10272 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 14:54:52.057644  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:52.033988  9937 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.721s	user 0.014s	sys 0.003s
I20260709 14:54:52.019572 10263 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:52.054816 10244 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.085446  9937 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.085588 10188 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.086222  9937 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.087195 10147 consensus_queue.cc:260] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.088508 10147 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.089141  9937 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:52.089304  9913 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.002s
I20260709 14:54:52.089769  9937 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:52.089852  9913 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.090327  9937 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.005704 10258 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.038425 10270 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.065399 10180 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:54:52.070727 10217 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.830s	user 0.019s	sys 0.000s
I20260709 14:54:52.093607 10217 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:54:52.093498 10258 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.094944 10271 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.095293  9937 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.084795 10244 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:52.096128 10126 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.003s
I20260709 14:54:52.096603 10126 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.076468 10263 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:52.067772 10201 consensus_meta.cc:287] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.147s	user 0.023s	sys 0.004s
W20260709 14:54:52.110986  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.111692  2757 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 14:54:52.071714 10124 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.114485  2759 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: []
W20260709 14:54:52.111603  2577 connection.cc:256] RPC call timeout handler was delayed by -1.905919322999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 14:54:52.060868 10256 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.118551  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:52.120882  2577 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.121794  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.123112  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:52.128960  2760 raft_consensus.cc:2376] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 16. Re-sending same reply.
I20260709 14:54:52.130285  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.123386  2751 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:54:52.075461  9616 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.103550 10244 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.131549 10244 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.131843  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.132733  2751 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:52.132917  9616 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:52.133455 10249 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.657s	user 0.002s	sys 0.000s
I20260709 14:54:52.124622  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.133917 10244 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.134429  2752 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:52.125473  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.135712  2753 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:52.136441 10210 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.008s
I20260709 14:54:52.126179  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.126047 10124 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.103552 10126 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.105690 10201 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.139194  9761 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:52.139341 10124 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.126780 10145 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.141196 10201 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.140923 10145 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:52.125360 10275 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 14:54:52.117404 10118 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.138185 10188 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.145246 10118 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:52.147962  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.148886  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.149317  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.127292  2577 connection.cc:256] RPC call timeout handler was delayed by -1.7414918989999251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
W20260709 14:54:52.153247  2577 connection.cc:256] RPC call timeout handler was delayed by -1.51251506899996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 14:54:52.154500  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.156318  2754 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:52.157428  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:52.166088  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.167896  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.172438  2606 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 14:54:52.170562  2605 pending_rounds.cc:85] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 11
I20260709 14:54:52.177273  2605 pending_rounds.cc:107] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 14:54:52.177683  2605 raft_consensus.cc:2889] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:52.167960  2683 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:52.179116  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.144241 10145 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:52.181461  2577 connection.cc:256] RPC call timeout handler was delayed by -1.2649303519999648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 14:54:52.187745  2577 connection.cc:256] RPC call timeout handler was delayed by -1.060639900999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 14:54:52.189530  2608 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
I20260709 14:54:52.191663  2606 raft_consensus.cc:1240] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 14:54:52.196715  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.194600  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.204895  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:52.205348  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 219325us
I20260709 14:54:52.206650  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:52.213860  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.180922 10124 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.215543  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.216295  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.192875 10167 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.223634  2608 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 14:54:52.224123  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:52.213500 10009 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:52.225996 10257 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.226665 10274 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.215006 10052 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.233515  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:52.238380  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.246119  2686 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.255451  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:52.228122 10145 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:52.230849 10259 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:52.267988  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 73516.5us
I20260709 14:54:52.269464  2680 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:54:52.250900  9852 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:54:52.271548  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.256891 10091 raft_consensus.cc:3055] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:52.227916 10124 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.233354 10167 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.278507  2754 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 14:54:52.287770  2686 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:52.277380  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.260113 10055 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:52.290038 10055 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:52.290139  9929 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:52.274101  2680 raft_consensus.cc:2410] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 14:54:52.290679 10055 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.292624  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.293579  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.289399  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.266803 10168 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.251087 10257 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.252766 10274 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.282124 10009 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 14:54:52.271375  9849 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:52.254088 10052 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.290800  9929 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.296247  2680 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:52.272621  9913 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.272548 10273 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 14:54:52.279997 10091 raft_consensus.cc:740] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:52.314851  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:52.298784 10286 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 14:54:52.303635 10289 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 14:54:52.301805  2612 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:52.271940 10180 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.304945 10009 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:52.318405  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:52.295647  2686 raft_consensus.cc:2410] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 14:54:52.308072  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.299353 10204 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:52.353729  2680 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:52.353863 10204 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.349267 10185 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:52.356755  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:52.309111  9849 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:52.359108  9849 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:52.287452 10145 consensus_queue.cc:1059] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 14:54:52.326040  9852 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 14:54:52.315207 10168 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.363701 10145 raft_consensus.cc:3055] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:52.364216 10145 raft_consensus.cc:740] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:52.365190 10145 consensus_queue.cc:260] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.326206 10009 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:52.367761 10009 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.369344 10009 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.349720 10091 consensus_queue.cc:260] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.372032 10091 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:52.352510 10180 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.353055  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.358256 10185 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:52.375516 10185 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.376485 10270 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.305732 10274 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.385231  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:52.335198  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:52.307240 10263 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:52.387871  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52805.1us
I20260709 14:54:52.388981  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.304994 10257 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.307016 10167 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.378814  2680 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 14:54:52.392638  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:52.401753  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.398675  2680 pending_rounds.cc:85] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 13
I20260709 14:54:52.402868  2680 pending_rounds.cc:107] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 14:54:52.403226  2680 raft_consensus.cc:2889] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:52.405275  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:52.326637 10124 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.382306 10204 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.415355 10293 consensus_queue.cc:1059] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:52.415577  9616 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:52.418176  9616 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:52.399154  9913 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 14:54:52.420331 10280 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.341s	user 0.004s	sys 0.000s
I20260709 14:54:52.422674  9913 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:52.428057  9913 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:52.420156  9616 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.431139 10142 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:52.433494 10142 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:52.433856 10142 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:52.431691 10181 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:52.435014 10181 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.423815  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1898) took 1347 ms. Trace:
I20260709 14:54:52.441605  2755 rpcz_store.cc:276] 0709 14:54:51.076757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:51.254270 (+177513us) service_pool.cc:224] Handling call
0709 14:54:51.256342 (+  2072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:51.797746 (+541404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:52.423295 (+625549us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:52.423681 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151154560}
W20260709 14:54:52.444599  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.445803  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.448298  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.448612  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.449757  2605 raft_consensus.cc:2468] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:52.452064  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.452100  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.453044  2610 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.453152  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:52.457533  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.456036  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.435688 10257 raft_consensus.cc:2764] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 14:54:52.463392 10058 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.463954 10147 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:52.464448 10147 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:52.464327 10058 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.464906 10147 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.469978  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.461793  3412 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.393258 10298 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:52.483294  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.486331  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.490957  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.463589 10260 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.498481 10188 raft_consensus.cc:2804] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:52.494297 10152 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.499339 10256 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:52.454782 10270 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:52.505360  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.455021 10124 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.506342  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.531849  2605 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:52.511758  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.508761 10188 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.519341 10152 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:52.520233 10290 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 14:54:52.523234 10058 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.520233 10256 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.520318 10244 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.464322 10257 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 14:54:52.481943 10214 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:52.488512 10263 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:52.558226  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.504114  9929 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.486627 10201 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 14:54:52.537065 10188 raft_consensus.cc:3060] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:54:52.571584  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 160633us
I20260709 14:54:52.553900 10256 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.564069 10214 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:52.597121  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.601397  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.603660  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.546574 10270 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.619733  2605 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:52.622323  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:52.569919 10126 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:52.529181 10260 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.552619 10124 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.563004 10244 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.597070  9937 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.606487 10214 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:52.626765 10169 raft_consensus.cc:2764] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 14:54:52.621649 10258 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.639791  9937 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:52.651966 10295 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 14:54:52.654098 10270 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:52.655260  9761 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.006s
I20260709 14:54:52.665632  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:52.611194 10293 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 14:54:52.674912 10259 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:52.685171 10244 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.669545 10303 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:52.694161 10175 raft_consensus.cc:3055] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:52.620028 10298 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
W20260709 14:54:52.696789  2427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 162) took 44367 ms (client timeout 15000 ms). Trace:
I20260709 14:54:52.698939 10169 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:54:52.707332 10279 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.698665 10204 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:52.708165 10279 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.632803 10126 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.710678 10279 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:52.711828  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:52.712028 10126 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.712349  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:52.713608 10145 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:52.713951  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:52.714691 10147 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.715770 10168 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.716900 10147 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.717543 10279 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:52.717646 10292 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.433s	user 0.001s	sys 0.001s
W20260709 14:54:52.674364 10118 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.005s
I20260709 14:54:52.675032 10302 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:52.719662 10302 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:52.720042 10302 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.718173 10279 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.723486 10099 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:52.717737  9937 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.724562 10279 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.725118 10185 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.728406 10256 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.730692 10204 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.730859 10256 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.731556  2603 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 26 because replica has last-logged OpId of term: 25 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 11.
I20260709 14:54:52.731290 10204 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.734254 10157 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.701538  2680 raft_consensus.cc:1275] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:54:52.734879 10157 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.735688 10175 raft_consensus.cc:740] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:52.736119 10299 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.169s	user 0.000s	sys 0.003s
I20260709 14:54:52.736678 10175 consensus_queue.cc:260] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.737911 10175 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.738255 10157 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:52.714661  2724 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:52.739270  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:52.739899  2759 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:52.704516  2427 rpcz_store.cc:269] 0709 14:54:08.328915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:08.329265 (+   350us) service_pool.cc:224] Handling call
0709 14:54:52.696446 (+44367181us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:54:36.971409 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:54:36.972397 (+   988us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:54:36.972456 (+    59us) write_op.cc:435] Acquired schema lock
0709 14:54:36.972492 (+    36us) tablet.cc:662] Decoding operations
0709 14:54:36.974134 (+  1642us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:54:36.974243 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 14:54:36.974272 (+    29us) tablet.cc:685] Acquiring locks for 16 operations
0709 14:54:37.198370 (+224098us) tablet.cc:701] Row locks acquired
0709 14:54:37.198431 (+    61us) write_op.cc:260] PREPARE: finished
0709 14:54:37.198635 (+   204us) write_op.cc:270] Start()
0709 14:54:37.198865 (+   230us) write_op.cc:276] Timestamp: P: 1783608877198555 usec, L: 0
0709 14:54:37.198918 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 14:54:37.243226 (+ 44308us) log.cc:844] Serialized 6361 byte log entry
0709 14:54:38.818943 (+1575717us) op_driver.cc:464] REPLICATION: finished
0709 14:54:39.076545 (+257602us) write_op.cc:301] APPLY: starting
0709 14:54:39.076727 (+   182us) tablet.cc:1367] starting BulkCheckPresence
0709 14:54:39.330525 (+253798us) tablet.cc:1370] finished BulkCheckPresence
0709 14:54:39.345526 (+ 15001us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:54:39.962551 (+617025us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18003fb6e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e3850ec 00007fa16e3d1c0a 00007fa16e3d14f7 00007fa16e3d1490 00007fa16e388f48 00007fa17209d1bb 00007fa1720e45d4 00007fa171f908da 00007fa171f933cb
0709 14:54:40.415383 (+452832us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:54:40.415802 (+   419us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 14:54:40.415872 (+    70us) write_op.cc:312] APPLY: finished
0709 14:54:40.416369 (+   497us) log.cc:844] Serialized 147 byte log entry
0709 14:54:41.200818 (+784449us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:41.244086 (+ 43268us) write_op.cc:454] Released schema lock
0709 14:54:41.282089 (+ 38003us) write_op.cc:341] FINISH: Updating metrics
0709 14:54:41.369045 (+ 86956us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:54:41.371335 (+  2290us) write_op.cc:454] Released schema lock
0709 14:54:43.199321 (+1827986us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b18003fb6e0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db
Metrics: {"mutex_wait_us":1112231,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":232266,"apply.run_cpu_time_us":71474,"apply.run_wall_time_us":2253905,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70975,"num_ops":16,"prepare.queue_time_us":333961,"prepare.run_cpu_time_us":106425,"prepare.run_wall_time_us":698404,"raft.queue_time_us":329482,"raft.run_cpu_time_us":104001,"raft.run_wall_time_us":153307,"replication_time_us":1619795,"spinlock_wait_cycles":44803456,"thread_start_us":1619936,"threads_started":5,"wal-append.queue_time_us":1211725,"wal-append.run_cpu_time_us":22176,"wal-append.run_wall_time_us":5267140}]]}
I20260709 14:54:52.743453 10276 consensus_queue.cc:1048] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:54:52.744609 10304 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 14:54:52.746371 10272 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:52.746622  2755 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 14:54:52.747793 10300 log.cc:927] Time spent T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 14:54:52.748164  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.748955  2752 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
W20260709 14:54:52.749150  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.750526 10204 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:52.747002 10272 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:52.753486 10272 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.698818  9849 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.755041  9761 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.755702  9761 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:52.713788 10152 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.758208  9761 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:52.667389 10258 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.762472 10272 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:52.763038 10270 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.763634 10270 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:52.764919 10058 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.785828  2756 pending_rounds.cc:85] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 8
I20260709 14:54:52.786499  2756 pending_rounds.cc:107] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 14:54:52.786875  2756 raft_consensus.cc:2889] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:54:52.791374  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.794118  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:52.671895  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76397.9us
I20260709 14:54:52.804250  2610 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:52.821738  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.765698 10058 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:52.825028  2612 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:52.825917  2753 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:52.826376  2753 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:52.828110  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:52.828796  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.830999  2611 raft_consensus.cc:2393] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:52.827273  2753 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.832695  2753 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:52.833315  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.834688  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.837220  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.842435  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.844645  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:52.802124 10147 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.849046  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:52.852455  2612 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:52.857105  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.849903  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:52.884701  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.821247 10276 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.824128 10203 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:52.894014  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.901630  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:52.903038  2611 raft_consensus.cc:2393] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:52.896481  2751 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 14:54:52.896385  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46621.8us
I20260709 14:54:52.915637  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:52.916888  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:52.851006 10271 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:52.958364  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:52.972038  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45848.2us
I20260709 14:54:52.850915 10256 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:52.970216  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.890954 10126 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:52.895783 10083 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:53.012463  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.013628  2755 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:54:52.968420  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.015923  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:52.984771  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:52.970687  2753 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 14:54:53.017434  2753 pending_rounds.cc:85] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 12
I20260709 14:54:53.017710  2753 pending_rounds.cc:107] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 14:54:53.017947  2753 raft_consensus.cc:2889] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:52.927935  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:52.965804  9913 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.001s
I20260709 14:54:52.926348  9937 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:53.027894  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.047411  2760 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 16.
I20260709 14:54:53.048012  2686 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:52.927063 10274 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:53.030910  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 408693us
I20260709 14:54:52.955044 10217 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:54:53.049690  2577 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.040807  2612 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:52.956405 10185 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.976179 10271 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.974237  9852 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:53.055016  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.009990 10210 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:52.850868 10154 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
I20260709 14:54:53.073967  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.073978 10150 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:54:53.050655 10055 consensus_meta.cc:287] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.742s	user 0.014s	sys 0.007s
I20260709 14:54:53.024206  9849 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:52.892742 10147 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:52.926807 10244 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.124s
I20260709 14:54:53.103260  2759 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:52.965278 10009 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 14:54:53.090328  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.194587  2755 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:53.010494 10083 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:52.918354 10276 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.076371 10274 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:53.113605  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.114735  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:53.202378  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.202486  2686 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:53.107630 10258 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:52.929023 10203 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:53.127424 10154 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:53.055621 10091 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:54:53.134088  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 2088) took 2580 ms (client timeout 1712 ms). Trace:
W20260709 14:54:53.217713  2603 rpcz_store.cc:269] 0709 14:54:50.553504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:50.553645 (+   141us) service_pool.cc:224] Handling call
0709 14:54:53.133651 (+2580006us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54001ead54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:53.133926 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5005715840}
I20260709 14:54:53.137746 10197 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:53.136690  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:53.235586  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 98968us
I20260709 14:54:53.139751 10145 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:53.144605  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.145120 10058 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:53.052946  9913 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.174484 10055 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:53.149580  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.153311 10124 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:53.289701  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:53.290174  2576 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.017s	sys 0.045s
I20260709 14:54:53.153792  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.293100  2751 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:53.155331  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:53.158152 10305 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.475s	user 0.004s	sys 0.000s
I20260709 14:54:53.106336 10210 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.302030  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:53.163825  9616 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.696s	user 0.017s	sys 0.003s
W20260709 14:54:53.165271  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:53.310266  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 337267us
W20260709 14:54:53.311301  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.311882  2650 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.180310 10199 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:53.086596 10271 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:53.077639 10260 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:53.096330 10257 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:53.332652  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.333770  2760 raft_consensus.cc:2376] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22. Re-sending same reply.
W20260709 14:54:53.213753  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.012s	sys 0.027s
I20260709 14:54:53.161615 10142 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:53.201468 10181 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.747s	user 0.015s	sys 0.000s
I20260709 14:54:53.221889  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:53.226043  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.352874  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.097069  9852 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.236049  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.200579 10083 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:53.257409  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.271917  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:53.277879 10145 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:53.363412  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 91649.2us
I20260709 14:54:53.147570  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:53.296849  2724 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.031s	sys 0.047s
I20260709 14:54:53.217367 10091 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:53.211771 10154 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:53.248283 10303 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
W20260709 14:54:53.290525  2576 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.017s	sys 0.045s
I20260709 14:54:53.269377 10185 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:54:53.313915  2650 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.274976 10152 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:53.325436  2607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.019s	sys 0.061s
I20260709 14:54:53.291232 10171 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:53.327033  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.385264  2751 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:53.335609  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.012s	sys 0.028s
I20260709 14:54:53.314592  9563 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:53.354771  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.345680 10257 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:53.364748  2724 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.031s	sys 0.047s
W20260709 14:54:53.397435  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 657857us
I20260709 14:54:53.397600  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.364790  2752 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 16 FOLLOWER]: Advancing to term 17
I20260709 14:54:53.317840 10052 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:53.320621  9616 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.359128 10181 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.418663 10181 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:53.421336  2610 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:53.370615  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.371143  9852 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:53.387992  3409 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.378453  2757 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
W20260709 14:54:53.383265  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 598508us
W20260709 14:54:53.426470  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 498639us
I20260709 14:54:53.384294  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1832) took 1100 ms. Trace:
W20260709 14:54:53.384714  2607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.019s	sys 0.061s
I20260709 14:54:53.432294  2607 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2095) took 1279 ms. Trace:
I20260709 14:54:53.373348 10260 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:53.373138 10147 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:53.361161  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.027s	sys 0.083s
I20260709 14:54:53.384409  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.398617  2760 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
W20260709 14:54:53.441884  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.028s	sys 0.083s
W20260709 14:54:53.402346  2724 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 104 similar messages]
W20260709 14:54:53.377373 10272 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.007s
I20260709 14:54:53.371340 10091 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:53.408034 10302 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.008s
I20260709 14:54:53.421002 10142 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:53.410982 10203 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:53.383107 10303 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:53.414507  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.373121 10276 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:53.416570  9616 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:53.423578  2684 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
W20260709 14:54:53.432907  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45024.9us
I20260709 14:54:53.394109  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.434402  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.408106 10210 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:53.395248 10152 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.462993  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.440111 10167 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:53.440225  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.430410  2683 rpcz_store.cc:276] 0709 14:54:52.283622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.377122 (+ 93500us) service_pool.cc:224] Handling call
0709 14:54:52.378017 (+   895us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:52.385757 (+  7740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:52.385806 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:52.394033 (+  8227us) log.cc:844] Serialized 52 byte log entry
0709 14:54:52.587399 (+193366us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:52.587433 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:52.587546 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:53.157184 (+569638us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:53.383882 (+226698us) raft_consensus.cc:1685] finished
0709 14:54:53.383912 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:53.384221 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8497,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":6331,"spinlock_wait_cycles":53672832,"thread_start_us":127343,"threads_started":1,"wal-append.queue_time_us":274577}
W20260709 14:54:53.406457 10188 consensus_meta.cc:287] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.814s	user 0.014s	sys 0.000s
W20260709 14:54:53.442507  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 128632us
W20260709 14:54:53.443272  2724 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.432786  2607 rpcz_store.cc:276] 0709 14:54:52.152735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.154933 (+  2198us) service_pool.cc:224] Handling call
0709 14:54:52.155268 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:52.155489 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:52.201913 (+ 46424us) log.cc:844] Serialized 14 byte log entry
0709 14:54:52.765652 (+563739us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b18004f2760. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:53.431576 (+665924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:53.431910 (+   334us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:53.431978 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:53.432076 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:53.432232 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2116,"prepare.queue_time_us":45057,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":12539648,"thread_start_us":665321,"threads_started":1}
I20260709 14:54:53.379359  9913 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:53.383735 10185 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:53.404558 10257 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:53.438716  3412 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:53.409744 10145 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:53.437894  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.463929  2612 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:53.469678  2611 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:54:53.471473  2681 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:53.478818  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.480269  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:53.479063  2650 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.481325  2751 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:53.481833  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:53.480391  2609 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:54:53.479821  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.471035  2606 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:53.464331 10214 consensus_meta.cc:287] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.800s	user 0.015s	sys 0.000s
I20260709 14:54:53.441061 10260 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:53.488621  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.512846  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.515043  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.486181 10154 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:53.446023  9852 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:53.446571 10307 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:53.492363  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.507432  2683 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 22
W20260709 14:54:53.508841  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 70284.4us
W20260709 14:54:53.511601  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 68382.4us
W20260709 14:54:53.541761  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.515717  2681 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:54:53.543813  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.544691  2681 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 14:54:53.518568  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:53.519016  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.545688  2751 raft_consensus.cc:2410] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 14:54:53.509584  2603 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:53.520501  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.546507  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:53.546967  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.517678 10260 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:53.524204  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.526448  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:53.517951 10210 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:53.518155 10199 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 14:54:53.528311  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.529407 10164 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:54:53.518009 10214 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.534742 10091 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:53.458107 10271 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:53.565367  9937 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:53.547638  2603 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:53.521754  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.546569  3409 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.459012 10302 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.549819  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.572899  2685 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:54:53.573868  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.458321  2426 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:53.555267  9913 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:53.549739  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.515105 10203 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:54:53.553572  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.027s	sys 0.040s
I20260709 14:54:53.557590  2607 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:53.602196  2608 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:53.558856  2755 raft_consensus.cc:2376] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22. Re-sending same reply.
W20260709 14:54:53.602618  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.027s	sys 0.040s
W20260709 14:54:53.563315  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 489872us
I20260709 14:54:53.604423  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.489473 10188 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.565277  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.562212  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.566210  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.553457 10145 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:54:53.571944  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.565347 10055 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:53.583963  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.585358  9937 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.586524 10214 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:53.586764 10302 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:53.636673  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:53.559916 10164 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:53.564930 10210 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:53.647778 10210 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:53.588992 10270 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:53.591573  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.544338  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:54:53.542815  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:53.600291  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91464us
W20260709 14:54:53.652984  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 110222us
I20260709 14:54:53.553510  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:53.563414  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:53.654435  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 101021us
I20260709 14:54:53.654906  2760 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 21: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:53.656512  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.607237  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.659313  2755 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:53.607336  9913 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:53.621656 10147 raft_consensus.cc:3060] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:53.608632  2426 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 20. New cstate: current_term: 20 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:53.612910  2681 raft_consensus.cc:1275] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:53.613481  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:53.616484  2683 raft_consensus.cc:2468] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:53.691020  2603 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:53.617223  2682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.013s	sys 0.025s
W20260709 14:54:53.692009  2682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.020s	sys 0.028s
W20260709 14:54:53.618913  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47111.9us
I20260709 14:54:53.621332 10058 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:53.620337  2686 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:53.617091  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.700389  2607 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:54:53.622474  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.572726 10272 raft_consensus.cc:3060] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:53.601925  9616 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:53.638780  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.705766  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:53.706112  2751 raft_consensus.cc:2393] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:53.560199 10307 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 14:54:53.661036  2605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.008s	sys 0.040s
I20260709 14:54:53.658450  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:53.706781  2605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.008s	sys 0.040s
W20260709 14:54:53.676920  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 132660us
I20260709 14:54:53.708351  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:53.709321  2607 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:53.560966 10274 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:53.682611  2724 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.624055 10145 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:53.578347  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:53.686278  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:53.704824 10256 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:53.706511  2612 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:54:53.709985  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:53.715346  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:53.716040  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.716401  2760 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:53.716776  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:53.704012  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81634us
I20260709 14:54:53.710470  2575 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:53.719218  2576 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.729905  2609 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:54:53.693207 10258 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:53.749593  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.699124 10055 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:53.748306  2609 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 14:54:53.750795  2751 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:53.747661  2603 raft_consensus.cc:2410] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 14:54:53.601424 10185 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:53.710582  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46878.3us
I20260709 14:54:53.777493  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.778204  2606 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 14:54:53.772037  2650 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:53.779402  2755 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
W20260709 14:54:53.780449  2650 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:53.781409  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55395.2us
I20260709 14:54:53.720804 10214 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:53.721531 10197 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:53.791285  2607 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:53.792793  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.794509  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.797128  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1906) took 2146 ms. Trace:
I20260709 14:54:53.721403 10156 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:53.815186  2756 rpcz_store.cc:276] 0709 14:54:51.650366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:51.868231 (+217865us) service_pool.cc:224] Handling call
0709 14:54:52.782237 (+914006us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b5800003998. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:52.782615 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:52.787864 (+  5249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:52.787917 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:52.788685 (+   768us) log.cc:844] Serialized 52 byte log entry
0709 14:54:53.442383 (+653698us) raft_consensus.cc:1645] Marking committed up to 8
0709 14:54:53.442430 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:53.442551 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:53.775169 (+332618us) raft_consensus.cc:1685] finished
0709 14:54:53.775205 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:53.797001 (+ 21796us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15799,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":6293,"spinlock_wait_cycles":61198592,"thread_start_us":595447,"threads_started":1,"wal-append.queue_time_us":825435}
I20260709 14:54:53.818790  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.820700  2756 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:53.722987 10145 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:54:53.727015 10157 raft_consensus.cc:2804] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:53.702792 10270 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 14:54:53.790819  2760 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:53.801965  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.833776  2686 raft_consensus.cc:2468] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:54:53.757890 10258 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:53.837061 10258 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:53.837953  9913 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:53.839314 10091 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:53.834226  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.841645  2760 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:53.825199 10126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.072s	sys 0.134s
I20260709 14:54:53.746342 10058 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 14:54:53.782876 10318 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:53.836112  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.851917  2686 raft_consensus.cc:2468] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:53.717902 10302 consensus_queue.cc:1048] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 14:54:53.796845 10276 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:53.728026 10203 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 14:54:53.856240  2760 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:53.798491 10055 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:53.729091 10256 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:53.847585 10126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.073s	sys 0.134s
I20260709 14:54:53.774308  9616 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
W20260709 14:54:53.880710  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.032s	sys 0.067s
W20260709 14:54:53.882104  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.032s	sys 0.067s
I20260709 14:54:53.883049  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:53.883728  2607 raft_consensus.cc:1275] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 14:54:53.805240 10199 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 14:54:53.886991  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.887919  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:53.870121 10310 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.346s	user 0.000s	sys 0.002s
W20260709 14:54:53.890815  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:53.892342  2760 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:53.851519 10157 raft_consensus.cc:697] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:53.834343  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:53.905164  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:53.823863  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:53.895074  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100708us
I20260709 14:54:53.922993  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:53.924113  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 108041us
I20260709 14:54:53.924204  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:53.925397  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:53.879693 10126 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:53.768777 10164 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:53.897764  2752 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 17.
I20260709 14:54:53.955053  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42032 (request call id 1434) took 1036 ms. Trace:
I20260709 14:54:53.899698  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:53.876893 10203 consensus_queue.cc:1243] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:53.921200  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38962 (request call id 1421) took 1102 ms. Trace:
I20260709 14:54:53.957964  2608 rpcz_store.cc:276] 0709 14:54:52.819596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.819848 (+   252us) service_pool.cc:224] Handling call
0709 14:54:53.920924 (+1101076us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b540011e754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:53.921120 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745963392}
I20260709 14:54:53.749341  2426 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:54:53.958813  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:53.895852  2751 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:54:53.934770  2760 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:54:53.935678  2679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.020s	sys 0.048s
W20260709 14:54:53.960553  2679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.020s	sys 0.049s
I20260709 14:54:53.943768  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:54:53.883471 10276 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:53.899312  9913 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:53.904579 10091 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:53.902587 10270 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:53.955578  2752 rpcz_store.cc:276] 0709 14:54:52.919056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.147157 (+228101us) service_pool.cc:224] Handling call
0709 14:54:53.954941 (+807784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:53.864688 10318 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:53.929819 10307 raft_consensus.cc:2804] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:53.941705 10259 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.004848 10188 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.000976  9913 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:54.039917  9913 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.041395  9913 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:54.005715 10091 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.042608  2426 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:54.045364 10091 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:54.006158 10175 consensus_meta.cc:287] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.268s	user 0.021s	sys 0.019s
W20260709 14:54:54.051465 10311 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 14:54:54.052956  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1917) took 1321 ms. Trace:
I20260709 14:54:54.053404  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1434) took 1897 ms. Trace:
I20260709 14:54:54.053412  2758 rpcz_store.cc:276] 0709 14:54:52.731440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.732258 (+   818us) service_pool.cc:224] Handling call
0709 14:54:52.732667 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:52.733047 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:52.733077 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:52.733687 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 14:54:53.194608 (+460921us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:53.194668 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:53.194750 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:54.052734 (+857984us) raft_consensus.cc:1685] finished
0709 14:54:54.052759 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.052892 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10148,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":449212,"threads_started":1,"wal-append.queue_time_us":994599}
I20260709 14:54:54.053884  2679 rpcz_store.cc:276] 0709 14:54:52.156218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.156357 (+   139us) service_pool.cc:224] Handling call
0709 14:54:52.179183 (+ 22826us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:52.212667 (+ 33484us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:52.225367 (+ 12700us) log.cc:844] Serialized 14 byte log entry
0709 14:54:52.525242 (+299875us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b1800398fe0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:54:52.703611 (+178369us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b1800398fe0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:54:54.001315 (+1297704us) log.cc:844] Serialized 14 byte log entry
0709 14:54:54.001614 (+   299us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:54.002318 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 14:54:54.002485 (+   167us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:54.002521 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.002638 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:54.053153 (+ 50515us) raft_consensus.cc:1685] finished
0709 14:54:54.053175 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.053341 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54915,"prepare.queue_time_us":8316,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":36164,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":6624,"spinlock_wait_cycles":16481024,"thread_start_us":1256899,"threads_started":2,"wal-append.queue_time_us":1221430}
I20260709 14:54:54.059038 10181 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.059687 10181 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:54.062343 10181 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:54.058717  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:54.044648  2426 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 19 to 20, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:53.941463 10319 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:54.100919 10319 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.101392 10319 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:54.100349  2426 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 14, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:54.107468  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2107) took 1356 ms. Trace:
I20260709 14:54:54.108040  2605 rpcz_store.cc:276] 0709 14:54:52.751350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.754211 (+  2861us) service_pool.cc:224] Handling call
0709 14:54:52.754546 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:52.787747 (+ 33201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:52.787803 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:52.788592 (+   789us) log.cc:844] Serialized 26 byte log entry
0709 14:54:53.707689 (+919097us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:53.707745 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:53.707839 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:54.107145 (+399306us) raft_consensus.cc:1685] finished
0709 14:54:54.107175 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.107395 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15372,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"thread_start_us":859219,"threads_started":1,"wal-append.queue_time_us":1094135}
I20260709 14:54:54.115425  2756 raft_consensus.cc:2468] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:54.130860  2759 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 14:54:54.132560  2759 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 14:54:54.135169 10259 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:54.135876 10259 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:54.138165 10259 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:54.139310 10157 consensus_queue.cc:237] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:54.146133  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 202453us
I20260709 14:54:54.146970  2752 raft_consensus.cc:1240] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:54.147133  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:54.147691  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1445) took 1167 ms. Trace:
I20260709 14:54:54.148015  2752 rpcz_store.cc:276] 0709 14:54:52.980473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.146392 (+1165919us) service_pool.cc:224] Handling call
0709 14:54:54.146765 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.147426 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.147631 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:54:54.071422  2678 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:54.005160 10009 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:53.944891 10316 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.073940  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1440) took 1093 ms. Trace:
I20260709 14:54:54.149870  2684 rpcz_store.cc:276] 0709 14:54:52.980169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.980335 (+   166us) service_pool.cc:224] Handling call
0709 14:54:53.133081 (+152746us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:53.133394 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:53.468000 (+334606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.073508 (+605508us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400269b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:54.073844 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710826496}
W20260709 14:54:54.074796 10323 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 14:54:54.077260  2751 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
W20260709 14:54:54.079826  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 139373us
I20260709 14:54:54.064544 10055 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.083427  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:54.159286  2686 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:54.084082 10167 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.088795 10152 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.091187 10317 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:54.176931 10317 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:54.178099 10317 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.093716  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.160640 10058 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 14:54:54.094863  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.179777 10058 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:54.181208 10058 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.148547  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:54.031647 10307 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.183706 10307 raft_consensus.cc:3060] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:54.184265 10147 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:54.138723 10259 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:54.185505 10259 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:54.186693 10083 consensus_meta.cc:287] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.783s	user 0.012s	sys 0.000s
I20260709 14:54:54.187247 10083 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:54.162933  2650 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.188374  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:54.167972  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 453902us
W20260709 14:54:54.190001  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.190443  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:54.142961 10154 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:54.072296 10164 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:54.169965  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1433) took 2017 ms. Trace:
I20260709 14:54:54.196856  2682 rpcz_store.cc:276] 0709 14:54:52.152824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.155783 (+  2959us) service_pool.cc:224] Handling call
0709 14:54:52.198744 (+ 42961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:52.218550 (+ 19806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:52.218589 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:52.225534 (+  6945us) log.cc:844] Serialized 26 byte log entry
0709 14:54:52.748034 (+522500us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18003ca6c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:54:53.692451 (+944417us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:54:53.692509 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:53.692596 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:54.169598 (+477002us) raft_consensus.cc:1685] finished
0709 14:54:54.169626 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.169893 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28278,"raft.queue_time_us":19864,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":8042,"spinlock_wait_cycles":4600576,"thread_start_us":944203,"threads_started":1,"wal-append.queue_time_us":1663092}
W20260709 14:54:54.198552  2757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2051) took 2085 ms (client timeout 2071 ms). Trace:
W20260709 14:54:54.171551 10142 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.000s
W20260709 14:54:54.157851 10257 consensus_meta.cc:287] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.004s
I20260709 14:54:54.211882  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.162448 10217 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.160867 10009 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:54.175923 10258 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:54.199255  2757 rpcz_store.cc:269] 0709 14:54:52.112750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.118187 (+  5437us) service_pool.cc:224] Handling call
0709 14:54:54.198301 (+2080114us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001ff9d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:54.198461 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2895593216}
W20260709 14:54:54.201483  2576 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.229652  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:54.215498  2680 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:54:54.165930 10167 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:54.235888 10175 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:54.199834 10259 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.224193 10257 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.237943 10319 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.233557 10142 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.279544 10142 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:54.280768  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:54.281172  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:54.281597  2751 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:54.281876  2752 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:54.281991 10142 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.283143  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:54.283828  2759 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:54:54.285943 10260 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.726s	user 0.001s	sys 0.012s
I20260709 14:54:54.224025 10320 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:54.240072 10257 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:54.241667 10318 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:54.245551 10171 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.850s	user 0.009s	sys 0.005s
I20260709 14:54:54.251554  2677 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 14:54:54.232329  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.192274 10256 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.207378 10232 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.210879 10293 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:54.228834 10126 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.104908  2426 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:54.264163  2651 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.265616  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:54.296221  2757 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:54.297484  2604 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 14:54:54.286967  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.298035  2604 pending_rounds.cc:85] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 9
I20260709 14:54:54.298369  2604 pending_rounds.cc:107] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 14:54:54.298511  2607 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:54.298676  2604 raft_consensus.cc:2889] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:54.272580  2723 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.299880 10150 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.300101 10203 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:54.296573  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.218298  2678 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:54.293227  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:54.291989  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.025s	sys 0.017s
W20260709 14:54:54.302973  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.025s	sys 0.017s
W20260709 14:54:54.288733  9937 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.030s
W20260709 14:54:54.294763  2685 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.016s	sys 0.027s
W20260709 14:54:54.309979  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77706.7us
W20260709 14:54:54.313325  2576 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.302304  2651 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:54.314538  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.286533 10260 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:54.315699  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.324267  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.326279  2607 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:54.291760 10210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.027s	sys 0.086s
W20260709 14:54:54.316005  2685 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.016s	sys 0.027s
I20260709 14:54:54.293933 10320 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:54.374464 10320 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:54.328403  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:54.379518  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51249us
I20260709 14:54:54.379812  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1827) took 2105 ms. Trace:
I20260709 14:54:54.380399  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.381556  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.383324  9563 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:54.265671 10145 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:54.306473 10203 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.320876 10181 raft_consensus.cc:2804] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:54.341006  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1418) took 1589 ms. Trace:
I20260709 14:54:54.300657 10150 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:54.312242 10180 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:54.389379  2604 rpcz_store.cc:276] 0709 14:54:52.752026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.753835 (+  1809us) service_pool.cc:224] Handling call
0709 14:54:52.799127 (+ 45292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.299418 (+1500291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.340596 (+ 41178us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400530054. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:54.340906 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3444615552}
I20260709 14:54:54.348723  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.348526  2426 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:54.290647 10167 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.316728 10257 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.400900  2426 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:54:54.332846 10210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.028s	sys 0.086s
I20260709 14:54:54.397538 10203 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:54.352631 10260 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:54.353638 10052 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:54:54.355964  2682 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:54:54.364114  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1442) took 1383 ms. Trace:
I20260709 14:54:54.426987  2680 rpcz_store.cc:276] 0709 14:54:52.980844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.981079 (+   235us) service_pool.cc:224] Handling call
0709 14:54:53.514351 (+533272us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:54:53.514613 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.230561 (+715948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.363685 (+133124us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b5400377654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:54.364015 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651906688}
I20260709 14:54:54.365360 10271 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:54.307222 10318 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:54.383891  9563 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:54.438946  9563 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:54.380263  2677 rpcz_store.cc:276] 0709 14:54:52.274751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.367673 (+ 92922us) service_pool.cc:224] Handling call
0709 14:54:54.053560 (+1685887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.292667 (+239107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.349390 (+ 56723us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:54.379543 (+ 30153us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400444650. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:54.379739 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4330655872}
I20260709 14:54:54.385986  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:54.441437  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 55547.3us
I20260709 14:54:54.442317  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:54.382525  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:54.387429  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:54.329417 10272 consensus_meta.cc:287] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.002s
I20260709 14:54:54.446193 10272 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:54.315375 10171 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.423496 10319 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:54.455052  2754 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.013s	sys 0.017s
W20260709 14:54:54.455670  2754 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.013s	sys 0.017s
W20260709 14:54:54.425552  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.395195 10259 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:54.396909 10142 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.395830 10232 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:54.471237  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:54.420137 10181 raft_consensus.cc:697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:54.472090  2724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.039s	sys 0.031s
W20260709 14:54:54.497728  2724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.039s	sys 0.031s
I20260709 14:54:54.487030  2684 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:54.494735 10258 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:54.547576  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 76388.4us
I20260709 14:54:54.549424  2577 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:54.426342 10052 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:54.430867 10083 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:54.482131 10232 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.468839 10259 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.421518 10257 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.482496  9913 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:54.555949 10302 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:54.497766 10175 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:54.537693 10293 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:54.620501  3410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.002s	sys 0.007s
W20260709 14:54:54.471756  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.009s	sys 0.022s
I20260709 14:54:54.608198 10232 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:54.690286  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.012s	sys 0.024s
W20260709 14:54:54.468868  2606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.020s
W20260709 14:54:54.691987  2606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.014s	sys 0.021s
W20260709 14:54:54.667654  2755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.005s	sys 0.007s
W20260709 14:54:54.693185  2755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.005s	sys 0.008s
W20260709 14:54:54.626183  3410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.002s	sys 0.007s
W20260709 14:54:54.627384  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.009s
W20260709 14:54:54.618927 10324 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 14:54:54.614452 10259 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:54.619194 10156 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.004s	sys 0.027s
I20260709 14:54:54.675484 10293 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.679719 10154 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:54.495867 10142 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.686252  9913 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:54.689128  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 306662us
W20260709 14:54:54.695392  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.001s	sys 0.011s
I20260709 14:54:54.623487 10330 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:54.691301  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 898612us
I20260709 14:54:54.623490 10175 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:54.712484  2681 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
W20260709 14:54:54.713886  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 326535us
W20260709 14:54:54.716171  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 331562us
W20260709 14:54:54.664011  2679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.009s	sys 0.024s
W20260709 14:54:54.787602  2679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.009s	sys 0.025s
W20260709 14:54:54.713369 10307 consensus_meta.cc:287] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
W20260709 14:54:54.714646 10156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.005s	sys 0.027s
I20260709 14:54:54.810976  2682 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:54.813045  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:54.728967  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1842) took 1298 ms. Trace:
I20260709 14:54:54.816636  2684 rpcz_store.cc:276] 0709 14:54:53.430238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.151269 (+721031us) service_pool.cc:224] Handling call
0709 14:54:54.486732 (+335463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.535634 (+ 48902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.723078 (+187444us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54000f0350. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:54:54.728858 (+  5780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770406272}
I20260709 14:54:54.731379  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:54.732086  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:54.818733  2607 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:54.733467  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:54.819658  2603 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:54.742250  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.825944  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:54.720431 10009 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:54.783584  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45953.9us
I20260709 14:54:54.778417  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:54.826867  2759 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:54:54.719079 10145 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:54.721517 10154 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.814203  2604 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:54.712491 10180 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:54.834767  2685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.013s	sys 0.018s
W20260709 14:54:54.833096  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.013s	sys 0.017s
W20260709 14:54:54.833017  2756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.013s	sys 0.017s
I20260709 14:54:54.866019  2607 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:54:54.869849  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1888) took 1699 ms. Trace:
W20260709 14:54:54.868378  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.014s	sys 0.017s
I20260709 14:54:54.790962 10175 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:54.870467  2685 rpcz_store.cc:276] 0709 14:54:53.170572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.573654 (+403082us) service_pool.cc:224] Handling call
0709 14:54:53.574006 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:53.574239 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:54.333742 (+759503us) log.cc:844] Serialized 14 byte log entry
0709 14:54:54.869377 (+535635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:54.869451 (+    74us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:54.869516 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.869610 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.869771 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":640979,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":1216076,"threads_started":2}
I20260709 14:54:54.533190  2426 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 16, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:54.770761 10258 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.848439  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1425) took 2019 ms. Trace:
W20260709 14:54:54.889528  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.844380 10331 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.704805 10181 consensus_queue.cc:237] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:54.868757  2756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.014s	sys 0.017s
I20260709 14:54:54.723656 10055 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 14:54:54.885398  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.791374 10203 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:54.856647 10157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.021s	sys 0.082s
I20260709 14:54:54.889271  2754 rpcz_store.cc:276] 0709 14:54:52.829123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:52.829256 (+   133us) service_pool.cc:224] Handling call
0709 14:54:52.829632 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:53.231740 (+402108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:53.231775 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:53.232331 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 14:54:53.896235 (+663904us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:54.456060 (+559825us) log.cc:844] Serialized 14 byte log entry
0709 14:54:54.456325 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.792960 (+336635us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:54.793043 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:54.793119 (+    76us) raft_consensus.cc:1685] finished
0709 14:54:54.848146 (+ 55027us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:54.848364 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":551427,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":153453,"raft.run_cpu_time_us":562,"raft.run_wall_time_us":64311,"spinlock_wait_cycles":923595136,"thread_start_us":1114624,"threads_started":2,"wal-append.queue_time_us":948969}
I20260709 14:54:54.893810  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:54.705760 10052 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:54.902123  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:54.904394  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.872222 10154 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:54:54.888638 10315 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 14:54:54.907161  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:54.790580 10271 consensus_queue.cc:1059] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 14:54:54.892248  9937 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.888407 10167 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:54.894819 10199 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.903321 10156 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:54.906188 10157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.082s
I20260709 14:54:54.912665 10055 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:54.957950  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 177265us
W20260709 14:54:54.905157  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:54.827482 10307 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:54.791234 10330 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:54.872969 10260 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 14:54:54.972278  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 422920us
I20260709 14:54:54.879174 10210 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 14:54:54.950923  9563 consensus_meta.cc:287] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.003s
I20260709 14:54:55.031040  9563 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:54.913029 10145 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:54.865036  2603 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:54.930480 10052 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:54.973613  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:54.978296  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:54.990248 10167 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:54.961009  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:55.033051  2680 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:55.036070  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1895) took 1649 ms. Trace:
I20260709 14:54:55.044777  2681 rpcz_store.cc:276] 0709 14:54:53.386294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.689925 (+303631us) service_pool.cc:224] Handling call
0709 14:54:53.690381 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.786646 (+1096265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.035621 (+248975us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540022cc54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:55.035967 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349926784}
W20260709 14:54:55.037285  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:54.940217 10058 raft_consensus.cc:2804] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:54:55.050459  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 145329us
W20260709 14:54:55.051769  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:55.054246  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80752.5us
I20260709 14:54:55.055462  2577 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:55.056886  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:54.906235 10258 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:55.083933  9563 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:55.059968 10167 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:54.904567 10331 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:55.094507  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2060) took 1914 ms. Trace:
I20260709 14:54:55.091940 10351 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:55.095082  2758 rpcz_store.cc:276] 0709 14:54:53.180703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.246910 (+1066207us) service_pool.cc:224] Handling call
0709 14:54:54.247412 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.273278 (+ 25866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.273327 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.292675 (+ 19348us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.618917 (+326242us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:54.670569 (+ 51652us) log.cc:844] Serialized 14 byte log entry
0709 14:54:54.930405 (+259836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.930523 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.094226 (+163703us) raft_consensus.cc:1685] finished
0709 14:54:55.094257 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.094428 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71733,"prepare.queue_time_us":48613,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":16738,"thread_start_us":550455,"threads_started":2,"wal-append.queue_time_us":628623}
I20260709 14:54:54.968375 10055 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:54:54.974565 10337 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:55.060483 10327 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.432s	user 0.002s	sys 0.000s
I20260709 14:54:55.101923  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:55.047696 10058 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:54.960129 10199 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:55.095019  9913 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:55.129842  2758 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 16. Current term is 17. Ops: []
I20260709 14:54:55.130524  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:55.143491  2754 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:55.084017 10185 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:55.137121 10210 raft_consensus.cc:3055] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:54:55.137378  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:54.963670  9937 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:55.156396  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:55.156204  2752 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:55.157513  2754 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:54:54.969900 10156 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:55.060940 10157 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:55.109122 10145 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:54:55.151270  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:55.175858  2759 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:55.132845 10333 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
I20260709 14:54:55.154673  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:55.179468  2751 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
I20260709 14:54:55.084396 10257 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:55.138183 10058 raft_consensus.cc:3060] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:54:55.166381 10352 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.445s	user 0.001s	sys 0.001s
I20260709 14:54:55.199108  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1838) took 1770 ms. Trace:
I20260709 14:54:55.246109  2683 rpcz_store.cc:276] 0709 14:54:53.428854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.691252 (+262398us) service_pool.cc:224] Handling call
0709 14:54:53.691574 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:53.695716 (+  4142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:53.695768 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:53.696475 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.303588 (+607113us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:54.303625 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.303685 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.198689 (+895004us) raft_consensus.cc:1685] finished
0709 14:54:55.198727 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.199006 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23720,"raft.queue_time_us":5115,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":16207,"thread_start_us":606663,"threads_started":1,"wal-append.queue_time_us":904862}
I20260709 14:54:55.117707 10351 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:55.047999 10180 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:55.155820 10293 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:55.240954 10260 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:54:55.253400  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 307114us
I20260709 14:54:55.155725 10210 raft_consensus.cc:740] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:55.241683 10337 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:55.289537 10258 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.274524 10167 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:55.337872  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 414802us
W20260709 14:54:55.337147  2686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.012s	sys 0.033s
I20260709 14:54:55.343218  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1434) took 1458 ms. Trace:
I20260709 14:54:55.292178 10293 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:55.345057  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.012s	sys 0.033s
I20260709 14:54:55.345772  2606 rpcz_store.cc:276] 0709 14:54:53.884697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.884899 (+   202us) service_pool.cc:224] Handling call
0709 14:54:53.902883 (+ 17984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:53.903341 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:53.903386 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:53.922233 (+ 18847us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.692652 (+770419us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:54.692726 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.692844 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.342863 (+650019us) raft_consensus.cc:1685] finished
0709 14:54:55.342889 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.343142 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":17449,"thread_start_us":769857,"threads_started":1,"wal-append.queue_time_us":705296}
I20260709 14:54:55.343966  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:55.329583 10164 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.109s	user 0.016s	sys 0.000s
W20260709 14:54:55.348116  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 397987us
I20260709 14:54:55.298316  2759 raft_consensus.cc:2468] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:55.356061  3409 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:55.300806  2650 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:55.368124  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 67418.8us
W20260709 14:54:55.369251  2650 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.369818  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:55.126178  2426 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 14 to 18, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:54:55.260234 10257 raft_consensus.cc:2764] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 14:54:55.361694 10340 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.432s	user 0.001s	sys 0.001s
I20260709 14:54:55.392236 10368 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:55.286111 10260 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:55.387363  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:55.368557 10349 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.421s	user 0.001s	sys 0.001s
I20260709 14:54:55.362947 10307 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.359768 10330 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:55.401160  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 348617us
I20260709 14:54:55.403381  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1841) took 1973 ms. Trace:
I20260709 14:54:55.341732 10203 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.407434  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:55.391562  2650 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.410884  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:55.325074 10337 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:55.442107  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:55.443311  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 51796.3us
I20260709 14:54:55.263976 10362 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.177s
I20260709 14:54:55.436308  2679 rpcz_store.cc:276] 0709 14:54:53.429933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.066287 (+636354us) service_pool.cc:224] Handling call
0709 14:54:54.066653 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.067113 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.067147 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.144501 (+ 77354us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.788128 (+643627us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:54.788177 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.788253 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.402958 (+614705us) raft_consensus.cc:1685] finished
0709 14:54:55.402997 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.403287 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1625,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":20943,"thread_start_us":633684,"threads_started":1,"wal-append.queue_time_us":733600}
I20260709 14:54:55.394986 10367 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:55.380402 10164 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:55.455368  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 77951.3us
I20260709 14:54:55.444121  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:55.444164  2650 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:55.399495 10318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.000s	sys 0.012s
W20260709 14:54:55.448653 10335 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
W20260709 14:54:55.441510  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.422536 10260 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:55.361506 10293 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:55.447315  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 469117us
I20260709 14:54:55.450882  2752 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
W20260709 14:54:55.462867  2724 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.511909  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:55.463485  2650 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.473978  2577 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:55.514067  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50617.5us
W20260709 14:54:55.515267  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:55.515318  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:55.515558  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:55.515935  2650 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.447655 10307 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:54:55.506340  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:55.468547 10276 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:55.476682  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 120765us
W20260709 14:54:55.468797  2726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.015s	sys 0.043s
W20260709 14:54:55.608611  2726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.016s	sys 0.043s
W20260709 14:54:55.609511  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 831178us
W20260709 14:54:55.399952  2678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.002s
W20260709 14:54:55.610389  2678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.007s	sys 0.002s
W20260709 14:54:55.437829 10320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.008s	sys 0.023s
I20260709 14:54:55.616326  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:55.502372 10167 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:55.380977 10210 consensus_queue.cc:260] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:55.504336 10345 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
W20260709 14:54:55.474093 10318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.000s	sys 0.015s
I20260709 14:54:55.471611 10145 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:55.501799 10342 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.346s	user 0.000s	sys 0.002s
I20260709 14:54:55.580255  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1456) took 1882 ms. Trace:
I20260709 14:54:55.602571  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:55.654105  2604 raft_consensus.cc:2393] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 27: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:55.616739 10320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.008s	sys 0.024s
I20260709 14:54:55.619521  2607 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:55.641044  2760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.012s	sys 0.035s
W20260709 14:54:55.655076  2760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.012s	sys 0.036s
I20260709 14:54:55.656051  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1930) took 1394 ms. Trace:
I20260709 14:54:55.433934 10257 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.169s
I20260709 14:54:55.573509  9913 raft_consensus.cc:697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:55.573225  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56208.4us
W20260709 14:54:55.667793  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:55.669440  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:55.670293  2650 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:55.473368 10164 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:55.454511 10369 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:54:55.653497  2680 rpcz_store.cc:276] 0709 14:54:53.697792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.428270 (+730478us) service_pool.cc:224] Handling call
0709 14:54:54.428742 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:55.047628 (+618886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.579873 (+532245us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400376754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:55.580164 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389191040}
I20260709 14:54:55.673341 10164 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.674182 10164 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 14:54:55.643285 10375 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.263s	user 0.000s	sys 0.002s
I20260709 14:54:55.623417 10167 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:55.505707 10126 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.146s
I20260709 14:54:55.618044 10307 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:55.609299  9937 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.618818 10199 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:55.674618 10164 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:55.656435  2760 rpcz_store.cc:276] 0709 14:54:54.262075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.266977 (+  4902us) service_pool.cc:224] Handling call
0709 14:54:54.267426 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.267912 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:54.739458 (+471546us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.110392 (+370934us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.655507 (+545115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:55.655593 (+    86us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:55.655673 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.655764 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.655961 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180594,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":5306,"raft.queue_time_us":912185,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":17683,"thread_start_us":1362732,"threads_started":3}
I20260709 14:54:55.705981 10164 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
W20260709 14:54:55.651620 10338 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 14:54:55.618522 10391 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:55.578374 10374 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.159s
I20260709 14:54:55.710412 10391 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:55.683852 10167 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:55.700536 10293 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.687193 10307 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:55.700271  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1427) took 2869 ms. Trace:
I20260709 14:54:55.704370 10351 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:55.643796 10318 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:55.732533  2755 rpcz_store.cc:276] 0709 14:54:52.830649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.995550 (+1164901us) service_pool.cc:224] Handling call
0709 14:54:54.034177 (+ 38627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.034662 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.034697 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.065975 (+ 31278us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.715448 (+649473us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:54.790943 (+ 75495us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.255882 (+464939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.255989 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.699904 (+443915us) raft_consensus.cc:1685] finished
0709 14:54:55.699946 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.700167 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93752,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":571557,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":18397,"thread_start_us":1046804,"threads_started":2,"wal-append.queue_time_us":807878}
I20260709 14:54:55.680263 10388 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:55.711246  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1924) took 2134 ms. Trace:
I20260709 14:54:55.683540 10210 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:55.744500  2753 rpcz_store.cc:276] 0709 14:54:53.577049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.123382 (+546333us) service_pool.cc:224] Handling call
0709 14:54:54.123879 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.124392 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.124433 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.125105 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 14:54:54.616392 (+491287us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:54.616452 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:54.616546 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.710928 (+1094382us) raft_consensus.cc:1685] finished
0709 14:54:55.710951 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.711141 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14836,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":490797,"threads_started":1,"wal-append.queue_time_us":1201463}
I20260709 14:54:55.751313  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:55.773255 10150 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:55.761502 10375 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 14:54:55.753727 10293 raft_consensus.cc:2804] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:55.749635 10388 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:55.758165 10156 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.789196  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:55.785712 10399 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:55.827404 10379 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
W20260709 14:54:55.828276 10185 consensus_meta.cc:287] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.673s	user 0.017s	sys 0.007s
W20260709 14:54:55.829805 10380 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 14:54:55.840804 10232 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.128s	user 0.017s	sys 0.000s
I20260709 14:54:55.986279 10232 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:55.902983  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 223994us
I20260709 14:54:55.905444  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1898) took 1775 ms. Trace:
I20260709 14:54:55.991086  2677 rpcz_store.cc:276] 0709 14:54:54.129845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.440969 (+311124us) service_pool.cc:224] Handling call
0709 14:54:54.534388 (+ 93419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:54.938244 (+403856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.938288 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:55.030255 (+ 91967us) log.cc:844] Serialized 52 byte log entry
0709 14:54:55.400616 (+370361us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:55.400688 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.400806 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.904626 (+503820us) raft_consensus.cc:1685] finished
0709 14:54:55.904689 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.905061 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":351721,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":140,"thread_start_us":736163,"threads_started":2,"wal-append.queue_time_us":700920}
I20260709 14:54:55.908557  2678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1845) took 1137 ms. Trace:
I20260709 14:54:55.910265  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1455) took 2212 ms. Trace:
I20260709 14:54:55.915416  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1438) took 1186 ms. Trace:
I20260709 14:54:55.807999 10293 raft_consensus.cc:697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:54:55.847349 10350 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.580s	user 0.004s	sys 0.000s
I20260709 14:54:55.808666 10408 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:54:55.968714 10152 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:55.850142 10388 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:55.779637 10145 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:55.977285 10257 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:55.978056 10406 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:55.832016 10391 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:55.978258 10302 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:55.795120  9913 consensus_queue.cc:237] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:55.840236 10399 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.021523 10385 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:56.021960 10399 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:55.841313 10185 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:56.022890  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.023069 10385 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.024258 10152 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:55.984474 10341 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 14:54:55.815399 10318 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:55.892522  2426 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:56.021027 10293 consensus_queue.cc:237] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:55.979115  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 189782us
I20260709 14:54:55.845947 10337 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:56.035606 10410 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 14:54:55.978080 10171 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:56.029021 10257 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.025454  2686 rpcz_store.cc:276] 0709 14:54:53.697452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.368475 (+671023us) service_pool.cc:224] Handling call
0709 14:54:54.369114 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.369710 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.369754 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.370584 (+   830us) log.cc:844] Serialized 26 byte log entry
0709 14:54:55.346891 (+976307us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:55.346941 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.347047 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.909520 (+562473us) raft_consensus.cc:1685] finished
0709 14:54:55.909578 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.909968 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13352,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":961897,"threads_started":1,"wal-append.queue_time_us":1069733}
I20260709 14:54:56.047266 10302 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.033439 10406 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.055431 10091 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:56.007493 10232 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.059520 10408 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:56.059931 10091 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:56.061323 10363 log.cc:927] Time spent T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.798s	user 0.001s	sys 0.002s
I20260709 14:54:56.037087  2426 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 18, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:56.071625 10406 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:56.077591 10302 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:55.995085  2678 rpcz_store.cc:276] 0709 14:54:54.771543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.812647 (+ 41104us) service_pool.cc:224] Handling call
0709 14:54:54.813112 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.813490 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:54.858007 (+ 44517us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.906784 (+1048777us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.907497 (+   713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:55.907622 (+   125us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:55.907818 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.908019 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.908320 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":260964,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":10378,"raft.queue_time_us":63861,"raft.run_cpu_time_us":640,"raft.run_wall_time_us":20602,"spinlock_wait_cycles":46011392,"thread_start_us":1047499,"threads_started":2,"wal-append.queue_time_us":603927}
W20260709 14:54:56.080633  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 617668us
I20260709 14:54:56.081197 10369 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:56.081821  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 464167us
I20260709 14:54:56.082157 10351 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.087560 10203 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:56.088948 10276 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:54:56.089823 10052 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.042s	user 0.018s	sys 0.005s
I20260709 14:54:56.027113  2611 rpcz_store.cc:276] 0709 14:54:54.728499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.728684 (+   185us) service_pool.cc:224] Handling call
0709 14:54:54.729057 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:54.729479 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:54.729549 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:54.770422 (+ 40873us) log.cc:844] Serialized 26 byte log entry
0709 14:54:55.277168 (+506746us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:55.277231 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:55.277331 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:55.904344 (+627013us) raft_consensus.cc:1685] finished
0709 14:54:55.904411 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:55.915134 (+ 10723us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41252,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":27634,"thread_start_us":431711,"threads_started":1,"wal-append.queue_time_us":1050943}
I20260709 14:54:56.091615 10330 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.092901 10257 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.093820 10091 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.106370  2608 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:54:56.061864 10408 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.107586  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1923) took 2502 ms. Trace:
W20260709 14:54:56.100333 10055 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.947s	user 0.017s	sys 0.019s
I20260709 14:54:56.108759  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:56.107684 10185 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 14:54:56.105785 10383 consensus_queue.cc:1059] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 14:54:56.199833  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:56.172354 10091 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:56.204319  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:56.204771 10091 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.206079  2605 raft_consensus.cc:2410] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 14:54:56.207366  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.208092 10369 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.210743  2610 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:54:56.211086 10272 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.172271 10055 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.211884 10272 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.213233  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.213757 10396 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:56.214246 10055 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:56.215077  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:56.215297  2677 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.202184  2607 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:54:56.191926 10232 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:56.149135  2756 rpcz_store.cc:276] 0709 14:54:53.576810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:54.118805 (+541995us) service_pool.cc:224] Handling call
0709 14:54:54.190167 (+ 71362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:54.190432 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:54:54.900808 (+710376us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.500938 (+600130us) log.cc:844] Serialized 14 byte log entry
0709 14:54:55.830599 (+329661us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b18003adf80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5272a6 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f
0709 14:54:55.832111 (+  1512us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4800150258. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa17356faa1 00007fa173566b37 00007fa1735669ca 00007fa17365e26f 00007fa17365df37 00007fa17367255c 00007fa173672508 00007fa173672488 00007fa173672443 00007fa1736716ec 00007fa17259ba96
0709 14:54:56.077609 (+245498us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b18003adf80. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:54:56.078048 (+   439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:56.078208 (+   160us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:54:56.078353 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:56.078520 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:56.078790 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600869,"prepare.queue_time_us":257794,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":87,"raft.queue_time_us":841607,"raft.run_cpu_time_us":4639,"raft.run_wall_time_us":258595,"spinlock_wait_cycles":88821504,"thread_start_us":1050649,"threads_started":2,"wal-append.queue_time_us":897665}
I20260709 14:54:56.216933 10167 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.232906 10091 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.234023 10331 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:56.207463  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:56.235754  2686 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:54:56.215845  2426 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 15, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:54:56.263530  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 822016us
I20260709 14:54:56.216586 10150 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 14:54:56.236580  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.215961 10395 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.243732  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.244184  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.245024 10371 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.197693  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:56.196717 10330 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.245908  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.246733  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.247035 10370 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:56.247233  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:56.247553  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.199545 10142 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.264262  2426 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 15, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:56.259514  2682 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:56.288959 10167 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:56.289548 10397 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.482s	user 0.004s	sys 0.000s
I20260709 14:54:56.214820 10396 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:56.291056  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 276348us
I20260709 14:54:56.292761 10396 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.258934 10232 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.260097 10276 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:56.260767 10336 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:54:56.261693  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54457.4us
W20260709 14:54:56.264956  2576 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.248572 10385 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:56.278371  2681 raft_consensus.cc:2410] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:54:56.279822  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:56.285742 10175 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.409s	user 0.011s	sys 0.016s
I20260709 14:54:56.286993  2680 raft_consensus.cc:2410] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 14:54:56.229606 10203 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:56.289240  2426 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 18, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:56.230499 10185 consensus_queue.cc:1243] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 14:54:56.296744  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:56.315070  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58338.8us
I20260709 14:54:56.195170 10351 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.301499  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.317255  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:56.318204  2759 raft_consensus.cc:3060] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 24
I20260709 14:54:56.318732 10232 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.319197 10336 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:56.307248  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:56.321273  2681 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.320768 10336 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.321532  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:56.322616  2754 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:56.363898  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:56.363734  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 98840.9us
W20260709 14:54:56.364900  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.308223  2752 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 14:54:56.307294 10055 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.308620  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.367427 10055 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.287009 10197 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:56.311226 10330 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:56.295418 10371 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.312069  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:56.267505 10150 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:56.313144  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:56.313514  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:56.324928 10276 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:56.307672  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.326843 10391 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:56.327723  9937 raft_consensus.cc:3055] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:56.327432 10203 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.329697 10272 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.308290 10142 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:56.333384  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:56.334316 10259 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.614s	user 0.010s	sys 0.015s
I20260709 14:54:56.336148  2426 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:56.341400  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:54:56.344511  2755 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:56.345139 10258 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 14:54:56.349906  2681 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:56.350991 10369 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:56.353808 10317 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:56.358260 10175 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.307411 10395 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:56.362527 10154 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.431s	user 0.006s	sys 0.020s
W20260709 14:54:56.377244  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.375922  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:56.305557  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:56.429163  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.436901  2684 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:56.392330  2677 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:56.394893  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:56.370864  2650 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.377347 10091 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:56.408970  2757 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:56.408907  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.413054 10055 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.388363 10276 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:56.415881 10152 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.417061  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.416461 10197 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:56.418697 10384 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.811s	user 0.007s	sys 0.000s
I20260709 14:54:56.420712 10371 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.422883 10388 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:56.466130 10415 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:54:56.467481  2756 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:56.468919  2603 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:56.394150 10391 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:56.471442 10391 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.393769  9937 raft_consensus.cc:740] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:56.475277 10188 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:54:56.475792 10276 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.478193 10276 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.480571 10091 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.440982 10154 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:56.484606 10260 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.910s	user 0.005s	sys 0.021s
I20260709 14:54:56.485251 10260 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.410110 10259 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:56.438663  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 105434us
W20260709 14:54:56.451097  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.441190  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:56.449175  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 140363us
I20260709 14:54:56.441030  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:56.449616  2758 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:56.476004  9937 consensus_queue.cc:260] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.461303  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:56.495323 10351 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:56.462456 10369 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:54:56.523033  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.523846  2686 raft_consensus.cc:2468] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 27.
I20260709 14:54:56.527981 10150 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.529026 10260 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:56.530479  2677 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:56.543987  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.466305 10171 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:54:56.504299 10188 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:56.550056  2679 raft_consensus.cc:1240] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:56.549867 10188 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:54:56.553311  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 212137us
W20260709 14:54:56.555994  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.540115 10369 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:56.547020 10415 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:56.577752  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.580355  2685 raft_consensus.cc:1240] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 24. Current term is 25. Ops: []
W20260709 14:54:56.584933  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 394102us
I20260709 14:54:56.591408  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:56.596611  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 219478us
I20260709 14:54:56.577906 10369 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:56.611858  2759 raft_consensus.cc:2468] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
W20260709 14:54:56.616992  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 319299us
I20260709 14:54:56.617478  2426 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:54:56.618991  2650 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.619549  2426 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 15, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:54:56.623461  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:56.624295  2609 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.627658  2685 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: [17.10-22.12]
I20260709 14:54:56.629010  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:56.632021 10356 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:56.641613 10126 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:56.642238  2683 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 24. Current term is 25. Ops: []
I20260709 14:54:56.642594 10126 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:56.641912  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:56.645884 10307 consensus_meta.cc:287] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.886s	user 0.014s	sys 0.007s
I20260709 14:54:56.646515 10307 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:56.648800  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 423564us
W20260709 14:54:56.658591  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 293785us
I20260709 14:54:56.681267  2755 raft_consensus.cc:2468] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
W20260709 14:54:56.685429  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65085.6us
W20260709 14:54:56.679668  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52090.3us
I20260709 14:54:56.647009  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:56.686619  2650 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.621724  2426 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 12 to 18, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 18 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:54:56.649340 10171 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:56.655786 10150 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 16 FOLLOWER]: Advancing to term 17
W20260709 14:54:56.720192 10164 consensus_meta.cc:287] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.888s	user 0.005s	sys 0.022s
I20260709 14:54:56.720969 10154 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.722476 10197 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:56.723696  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 181128us
I20260709 14:54:56.638540 10157 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:54:56.726958 10058 consensus_meta.cc:287] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.333s	user 0.013s	sys 0.026s
W20260709 14:54:56.727660  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44492.2us
W20260709 14:54:56.763670  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.728271  2575 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:56.656962 10203 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:56.729327 10210 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.959s	user 0.008s	sys 0.012s
I20260709 14:54:56.658799  9937 raft_consensus.cc:3060] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:56.658656 10091 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.676206 10175 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.739568  2609 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:56.729842 10351 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:54:56.767318 10171 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:56.791095 10351 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:56.791647 10351 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:56.792191 10171 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.748019  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:56.749557  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:56.743870  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 105508us
I20260709 14:54:56.795755 10259 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.796653  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:56.752200 10362 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:56.797752 10362 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:56.753671  2724 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.799163  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:56.755398 10164 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:56.761349  2650 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:56.803447 10197 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:56.803813  2604 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:56.804783 10197 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.779407 10058 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.805622 10197 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.807288 10408 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.808233 10408 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:56.655066 10330 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 14:54:56.783273 10157 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:56.784471 10272 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:56.666627 10180 consensus_queue.cc:1048] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 14:54:56.723724  2426 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 12 to 17, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:54:56.831760  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94815.3us
W20260709 14:54:56.831735  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 103638us
I20260709 14:54:56.835376  2677 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
W20260709 14:54:56.839148  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105214us
I20260709 14:54:56.859297 10362 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:56.859787  2426 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 8 to 13, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:54:56.877215  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:56.891224  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47253.8us
W20260709 14:54:56.893656  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105094us
I20260709 14:54:56.834252 10164 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:56.906629  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:56.940344  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.837620 10272 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:56.904028 10185 raft_consensus.cc:2804] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:54:56.834007 10391 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.928049 10152 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:56.955250  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:56.950183  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 205811us
I20260709 14:54:56.958810 10185 raft_consensus.cc:697] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:56.956260 10152 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:56.960443 10152 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:56.960203 10185 consensus_queue.cc:237] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:56.944947 10362 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:54:56.975382  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 236931us
I20260709 14:54:56.958865  2426 catalog_manager.cc:5697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:54:56.980187  2426 catalog_manager.cc:5697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:54:56.992673  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:56.985808 10317 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.004s
I20260709 14:54:57.001470 10319 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.002118 10319 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.006255 10272 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:57.012270 10395 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.012859  2605 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 14:54:57.039287  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:57.014777 10428 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.383s	user 0.000s	sys 0.004s
I20260709 14:54:57.016876 10203 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:56.967316 10058 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.021520 10391 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:57.029507 10356 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:54:57.030081 10126 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.066112  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:57.061753  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 168568us
I20260709 14:54:57.032763 10307 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:56.989015  2651 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:57.038174  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:56.928663 10180 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.058s
W20260709 14:54:57.068634 10411 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 14:54:57.048117  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 251630us
W20260709 14:54:57.054152  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80310.6us
I20260709 14:54:57.059186  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:57.080490  2681 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:56.991286 10408 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.051090  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:57.060348 10083 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:57.051071 10203 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:57.078989  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.094398  2678 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 14:54:57.102900  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:57.110410  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 121516us
I20260709 14:54:57.108134  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:57.122431  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:57.134011  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:57.125926  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:57.093474 10415 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:57.061164 10351 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:57.171589  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49321.1us
I20260709 14:54:57.108665 10126 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:57.173625  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 46230.8us
I20260709 14:54:57.174432  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:57.157524  2651 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:57.070454 10356 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:54:57.158355  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 90637.7us
I20260709 14:54:57.117772 10055 raft_consensus.cc:2804] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:54:57.189851  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:57.201092  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:57.232808  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:57.233461 10392 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 14:54:57.175021 10197 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:57.285689  2683 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:57.287390  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:57.288955  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 161647us
W20260709 14:54:57.315706 10432 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 14:54:57.197249 10356 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:54:57.374797  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 144632us
W20260709 14:54:57.386173  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 213020us
I20260709 14:54:57.125476 10083 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:57.488350 10083 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:57.392735  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.489707  2758 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:54:57.408068  2724 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.068s	sys 0.167s
W20260709 14:54:57.491050  2724 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.069s	sys 0.167s
I20260709 14:54:57.229661 10415 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:57.491436 10437 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.195s	user 0.001s	sys 0.003s
I20260709 14:54:57.234675 10126 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:57.491459  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 622763us
W20260709 14:54:57.480873  2684 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.014s	sys 0.024s
I20260709 14:54:57.189076 10058 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:54:57.480603  2680 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.017s	sys 0.025s
I20260709 14:54:57.232913  2608 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:57.481031 10142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.035s
I20260709 14:54:57.314177 10367 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:57.480203  2677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.015s	sys 0.023s
W20260709 14:54:57.501189  2680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.017s	sys 0.025s
W20260709 14:54:57.502774  2677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.015s	sys 0.024s
W20260709 14:54:57.496646  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 261257us
W20260709 14:54:57.485137  2650 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:57.499696 10188 consensus_meta.cc:287] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.946s	user 0.021s	sys 0.006s
I20260709 14:54:57.505717  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.506817  2759 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:57.496485 10369 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.006s
I20260709 14:54:57.481658 10150 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:57.507647 10142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.035s
I20260709 14:54:57.484429 10055 raft_consensus.cc:697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:57.528445  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.532606  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.498718 10164 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:54:57.530597  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45679.8us
I20260709 14:54:57.462801 10203 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.509976 10157 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.547269 10157 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.548003 10157 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:57.549444  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 401479us
I20260709 14:54:57.522008 10058 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:57.544934  2650 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.551841  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:57.521667 10367 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:57.557016 10433 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.423s	user 0.010s	sys 0.000s
W20260709 14:54:57.559590 10439 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 14:54:57.562054 10126 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:57.562288 10431 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.537s	user 0.003s	sys 0.001s
I20260709 14:54:57.512239 10385 raft_consensus.cc:2804] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:54:57.564666  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52074.2us
I20260709 14:54:57.550954 10408 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 14:54:57.567281 10408 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:54:57.567704 10408 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:57.569123 10152 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.005s
I20260709 14:54:57.569718 10152 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:57.552091 10058 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.574775  2755 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:57.565865  3409 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:57.577688 10164 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.579383 10391 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:57.580173  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:57.580608 10164 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:57.583940 10180 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:57.584652 10180 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:57.585150 10180 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:54:57.586369 10440 log.cc:927] Time spent T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260709 14:54:57.562985 10258 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.517918 10188 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:57.592083  2751 raft_consensus.cc:1275] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:54:57.568674 10408 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.590413  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.590945  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.558490 10367 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:57.618106  2650 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.624457  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:57.590873  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.524693 10369 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:57.628623  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.550742 10157 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:57.634752  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54670.6us
I20260709 14:54:57.631942  2751 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:57.582237 10083 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 14:54:57.632434  2752 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:57.587554 10415 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:57.563050 10351 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:57.586983 10385 raft_consensus.cc:697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:57.636240  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:57.659039  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146718us
I20260709 14:54:57.638675  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.640892  2754 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:57.647648  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:57.649034 10369 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.649679  2686 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26. Ops: []
I20260709 14:54:57.654910  2760 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 14:54:57.632148 10319 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:57.613658 10150 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:57.665448 10276 consensus_meta.cc:287] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.142s	user 0.010s	sys 0.010s
I20260709 14:54:57.666217 10150 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.667199 10055 consensus_queue.cc:237] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:57.668222 10157 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:57.668900 10157 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.631675 10272 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:54:57.674633 10385 consensus_queue.cc:237] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:57.677539  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:57.679251  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.680069  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:57.679380 10391 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:54:57.667820  2753 raft_consensus.cc:1240] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:54:57.681034  2682 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1470) took 1254 ms. Trace:
I20260709 14:54:57.688508  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.689077  2604 pending_rounds.cc:85] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 10
I20260709 14:54:57.689563  2604 pending_rounds.cc:107] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 14:54:57.689891  2604 raft_consensus.cc:2889] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:54:57.690228  2605 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:57.691233 10362 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.691785  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:57.692313  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:57.692883  2610 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:57.691114  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:57.693998  2612 raft_consensus.cc:2410] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 14:54:57.692974  2611 raft_consensus.cc:3060] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:57.689570  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:57.695317  2751 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:54:57.617147 10258 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:57.709707  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:57.711026  2608 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:57.684216  2682 rpcz_store.cc:276] 0709 14:54:56.426334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.675978 (+249644us) service_pool.cc:224] Handling call
0709 14:54:56.704264 (+ 28286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:56.746162 (+ 41898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:56.746206 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:57.025227 (+279021us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b2c00053ed8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:54:57.025883 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 14:54:57.500272 (+474389us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:57.500318 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:57.500394 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:57.658942 (+158548us) raft_consensus.cc:1685] finished
0709 14:54:57.658975 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:57.680913 (+ 21938us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23910,"raft.queue_time_us":23916,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":297,"spinlock_wait_cycles":29902592,"thread_start_us":473831,"threads_started":1,"wal-append.queue_time_us":567941}
I20260709 14:54:57.680243 10188 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:57.715842  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.717854  2758 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 18. Current term is 19. Ops: []
I20260709 14:54:57.721119  2753 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:57.694943 10408 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:57.687244 10150 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.684031 10391 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:57.729940  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 82408.2us
I20260709 14:54:57.702702 10319 raft_consensus.cc:3060] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:54:57.748914 10259 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:57.760735  2753 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:54:57.757364  2757 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
W20260709 14:54:57.757038  2650 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.772748  2754 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:54:57.720198 10197 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 14:54:57.774032  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:57.781028  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:57.782589  2650 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:57.783471  2650 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:57.785276  2650 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.691812 10362 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:57.791941  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.792778  2760 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:57.794047 10362 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.788658  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.796922 10362 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:57.797086  3409 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:57.797887 10362 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:57.799999  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.800321  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.800885  2758 raft_consensus.cc:2368] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18.
W20260709 14:54:57.801247  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 122063us
W20260709 14:54:57.802224  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.802852  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.803109 10188 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:57.803241  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.803828 10188 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:57.805055 10362 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.805261  2755 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:57.804930 10188 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:57.803834  2753 raft_consensus.cc:2393] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:57.802142  2754 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:54:57.799806 10391 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:57.817727  2755 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 14:54:57.809499  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.811231  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.812151  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.816444 10258 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:57.819057  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:57.819609  3409 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:57.818550  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.820184  2754 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:57.821307  2753 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:54:57.824585  2752 raft_consensus.cc:2468] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:57.710374 10336 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 14:54:57.828423  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.829339  2757 raft_consensus.cc:2393] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 19: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:57.829463 10157 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:57.830330  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.852015  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.852653  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:57.834666  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:57.861045  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.758801 10150 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:57.868495  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:57.870256  2755 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:54:57.830861 10272 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:57.874977  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72820.9us
I20260709 14:54:57.779042 10259 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:57.874997 10152 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:57.868500 10197 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:57.855492 10356 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:57.893381  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68862.6us
I20260709 14:54:57.884392 10157 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:54:57.914533  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.916914  2755 raft_consensus.cc:2376] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23. Re-sending same reply.
W20260709 14:54:57.922006  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103489us
I20260709 14:54:57.924986 10356 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:54:57.925501  2576 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:57.933127  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45644 (request call id 1854) took 1531 ms. Trace:
I20260709 14:54:57.933743  2686 rpcz_store.cc:276] 0709 14:54:56.401578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.606745 (+205167us) service_pool.cc:224] Handling call
0709 14:54:56.662159 (+ 55414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:57.661284 (+999125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:57.932767 (+271483us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400444b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:54:57.933047 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270558464}
I20260709 14:54:57.938591 10391 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:57.964303  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:57.993106  2611 raft_consensus.cc:2468] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
W20260709 14:54:58.001966  2684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.000s	sys 0.035s
W20260709 14:54:57.999683  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 643212us
I20260709 14:54:58.010088  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1469) took 1616 ms. Trace:
I20260709 14:54:58.019253  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.020378  2758 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:54:58.000159 10171 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.000s	sys 0.029s
I20260709 14:54:58.028817  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.053563  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.055464  2755 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:58.033821  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.057951  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.061450  2752 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:58.015231 10232 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:54:58.066453  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.069010  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.018324  2684 rpcz_store.cc:276] 0709 14:54:56.394106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.603982 (+209876us) service_pool.cc:224] Handling call
0709 14:54:56.639479 (+ 35497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:56.698536 (+ 59057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:56.698591 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:56.772385 (+ 73794us) log.cc:844] Serialized 52 byte log entry
0709 14:54:56.880270 (+107885us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:54:57.499335 (+619065us) log.cc:844] Serialized 14 byte log entry
0709 14:54:58.005884 (+506549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.009523 (+  3639us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:58.009647 (+   124us) raft_consensus.cc:1685] finished
0709 14:54:58.009668 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.009976 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84368,"prepare.queue_time_us":105047,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":60005,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":32873,"thread_start_us":1130841,"threads_started":3,"wal-append.queue_time_us":346728}
W20260709 14:54:58.050501 10185 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.022s
W20260709 14:54:58.075363  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 149881us
W20260709 14:54:58.078665  2576 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:58.085088  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.086525  2681 raft_consensus.cc:2393] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 27: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:58.087692  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.074595  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:58.089936  2686 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:58.092746  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:54:58.074240 10317 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.097982 10317 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:58.100000  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.101696 10317 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:58.102806 10197 consensus_queue.cc:1059] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:58.104040 10197 raft_consensus.cc:3055] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:58.104511 10197 raft_consensus.cc:740] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:58.105530 10197 consensus_queue.cc:260] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:58.107645 10259 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:58.107610 10197 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:58.110127 10203 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:58.015496 10258 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 14:54:58.112545  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.114091  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.116504  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.116336 10276 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:54:58.117276  2757 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:58.117405  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.117591 10276 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:58.118106  2754 raft_consensus.cc:2376] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23. Re-sending same reply.
I20260709 14:54:58.118063 10276 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:58.119693 10276 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:58.121212 10276 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:58.125551  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.032270 10272 consensus_queue.cc:1059] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 14:54:58.127727 10272 consensus_queue.cc:1243] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 21.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:54:58.128378 10272 raft_consensus.cc:3055] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:58.121342  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.128742 10272 raft_consensus.cc:740] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:58.133167 10180 consensus_meta.cc:287] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.004s
W20260709 14:54:58.121309  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:58.077843 10185 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.008s	sys 0.022s
I20260709 14:54:58.129943 10272 consensus_queue.cc:260] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:58.151014  2755 raft_consensus.cc:1275] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 14:54:58.152014  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73408.3us
I20260709 14:54:58.153923  2760 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:58.158303  2677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.025s
W20260709 14:54:58.158875  2677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.017s	sys 0.025s
I20260709 14:54:58.107616 10150 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:58.187760  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.193086  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.193099  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.198679  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.183759  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.243928  2755 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:58.183759  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:54:58.184230  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.207526 10272 raft_consensus.cc:3060] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:58.200371  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.227483  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.247501  2678 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:58.188761  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1475) took 1601 ms. Trace:
I20260709 14:54:58.170761  2757 raft_consensus.cc:2468] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:54:58.248236  2677 rpcz_store.cc:276] 0709 14:54:56.595987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.845578 (+249591us) service_pool.cc:224] Handling call
0709 14:54:56.846002 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:56.846340 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:54:57.508086 (+661746us) log.cc:844] Serialized 14 byte log entry
0709 14:54:58.159524 (+651438us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:58.159593 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:58.188286 (+ 28693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.188379 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.188681 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67312,"prepare.queue_time_us":138852,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":113,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":9547,"thread_start_us":1168794,"threads_started":2}
I20260709 14:54:58.192242  2752 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
W20260709 14:54:58.253840  2756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.029s	sys 0.019s
W20260709 14:54:58.254554  2756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.029s	sys 0.019s
I20260709 14:54:58.260486  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:58.258652  2577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.012s	sys 0.036s
I20260709 14:54:58.284299  2611 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:54:58.290257  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.285167  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.300014  2752 raft_consensus.cc:2468] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:54:58.292033  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.010s	sys 0.020s
W20260709 14:54:58.193342 10058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.034s	sys 0.055s
I20260709 14:54:58.293047  2678 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:54:58.295765  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.307763  2608 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 21: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:54:58.311298  2686 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
W20260709 14:54:58.292405  2759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.003s	sys 0.027s
W20260709 14:54:58.313650  2759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.003s	sys 0.027s
I20260709 14:54:58.313520  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.318797  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.320107  2757 raft_consensus.cc:2393] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:54:58.284324  2577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.015s	sys 0.036s
I20260709 14:54:58.193616 10154 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:54:58.146344 10180 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:58.314774  2603 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
W20260709 14:54:58.306910 10058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.035s	sys 0.055s
W20260709 14:54:58.327579  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.328949  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:58.305214  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.011s	sys 0.021s
I20260709 14:54:58.335898  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:54:58.342500  2760 raft_consensus.cc:2376] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23. Re-sending same reply.
I20260709 14:54:58.339334  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:54:58.347213  3411 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.010s	sys 0.031s
W20260709 14:54:58.348652  3411 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.010s	sys 0.031s
I20260709 14:54:58.341167  2753 raft_consensus.cc:2376] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23. Re-sending same reply.
W20260709 14:54:58.350653  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 814628us
I20260709 14:54:58.192777 10367 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 14:54:58.352418  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.344578  2678 raft_consensus.cc:2376] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25. Re-sending same reply.
I20260709 14:54:58.307454 10150 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:58.357498  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:58.359064  2753 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 14:54:58.350204  2685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.017s	sys 0.024s
W20260709 14:54:58.359949  2685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.017s	sys 0.024s
W20260709 14:54:58.360108  2575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.019s	sys 0.021s
W20260709 14:54:58.361183  2575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.020s	sys 0.022s
W20260709 14:54:58.361747  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 857510us
I20260709 14:54:58.374181  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:58.347644 10157 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.381970  2753 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:54:58.383842  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:58.387367  2650 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:58.390926  2755 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:58.398079  2760 raft_consensus.cc:2410] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:54:58.399204  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:58.399118  2651 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:58.369257 10142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.029s	sys 0.018s
I20260709 14:54:58.409571  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:58.405575 10142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.029s	sys 0.018s
I20260709 14:54:58.424520  2611 raft_consensus.cc:1217] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 21.12->[22.13-22.13]   Dedup: 22.13->[]
I20260709 14:54:58.425668  2611 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 14:54:58.422513  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.372548 10197 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 14:54:58.420233  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 44553.2us
W20260709 14:54:58.416124  2609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.031s	sys 0.020s
W20260709 14:54:58.432994  2609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.031s	sys 0.020s
I20260709 14:54:58.447402  2751 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:54:58.434979  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:58.448333  2612 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:54:58.433960  2680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.035s	sys 0.025s
I20260709 14:54:58.453322  2755 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 14:54:58.453591  2680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.036s	sys 0.025s
I20260709 14:54:58.450851  2726 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:58.459463  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:58.373194 10150 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:58.413123 10451 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:58.430801 10259 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:58.423388 10185 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:58.463431  3411 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:46277 timed out after 3.194s (SENT) [suppressed 119 similar messages]
I20260709 14:54:58.432754 10197 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:54:58.450476 10126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.026s	sys 0.030s
I20260709 14:54:58.470230  2680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1474) took 1902 ms. Trace:
I20260709 14:54:58.414018 10157 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.515228  2680 rpcz_store.cc:276] 0709 14:54:56.568161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.735566 (+167405us) service_pool.cc:224] Handling call
0709 14:54:56.760894 (+ 25328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:56.767980 (+  7086us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:57.502995 (+735015us) log.cc:844] Serialized 14 byte log entry
0709 14:54:58.454587 (+951592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:58.454654 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:58.465974 (+ 11320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.466091 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.470152 (+  4061us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47103,"prepare.queue_time_us":216564,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":310,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":6538,"thread_start_us":1608162,"threads_started":2}
I20260709 14:54:58.517336 10157 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:58.454445 10362 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:58.518196 10362 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:58.494141 10203 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:54:58.464143 10385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.021s	sys 0.049s
W20260709 14:54:58.494624  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43906.3us
I20260709 14:54:58.494275 10142 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:54:58.498332 10276 raft_consensus.cc:2804] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 14:54:58.503129   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 3581)
W20260709 14:54:58.503701  2578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.023s	sys 0.020s
W20260709 14:54:58.559787  2578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.024s	sys 0.020s
I20260709 14:54:58.422204 10367 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 14:54:58.506319  2651 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:58.466269 10351 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:54:58.526248  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.446940 10415 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:58.550694  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:58.534258 10126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.026s	sys 0.030s
W20260709 14:54:58.551770 10385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.021s	sys 0.049s
W20260709 14:54:58.550119 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.012s	sys 0.040s
W20260709 14:54:58.556401  2426 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.022s	sys 0.017s
W20260709 14:54:58.603508  2426 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.023s	sys 0.017s
W20260709 14:54:58.556754 10188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.004s	sys 0.023s
W20260709 14:54:58.616273  2652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.011s	sys 0.030s
W20260709 14:54:58.616951  2652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.012s	sys 0.031s
W20260709 14:54:58.587677  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81451.8us
I20260709 14:54:58.497653 10259 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:58.562029 10157 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.495050 10185 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.600283 10451 raft_consensus.cc:2804] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:58.594045 10276 raft_consensus.cc:697] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:58.612401  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.620002  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.608188  2612 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 14:54:58.623770  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:58.589634 10203 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:58.634653  2608 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
W20260709 14:54:58.637589  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:58.619756 10452 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 14:54:58.616299 10188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.005s	sys 0.024s
I20260709 14:54:58.646653  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.647589  2610 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:54:58.649920  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1453) took 1126 ms. Trace:
I20260709 14:54:58.650524  2759 rpcz_store.cc:276] 0709 14:54:57.523769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:57.524162 (+   393us) service_pool.cc:224] Handling call
0709 14:54:57.539971 (+ 15809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:57.629127 (+ 89156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:57.629184 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:57.644364 (+ 15180us) log.cc:844] Serialized 26 byte log entry
0709 14:54:58.314670 (+670306us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:58.314741 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.314882 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:58.649664 (+334782us) raft_consensus.cc:1685] finished
0709 14:54:58.649690 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.649848 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3917,"raft.queue_time_us":14116,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":25697,"thread_start_us":620793,"threads_started":1,"wal-append.queue_time_us":884679}
I20260709 14:54:58.660079  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:58.626788 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.012s	sys 0.041s
W20260709 14:54:58.661609  2683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.005s	sys 0.015s
W20260709 14:54:58.662029  2683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.005s	sys 0.015s
W20260709 14:54:58.664825  3409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.000s	sys 0.019s
W20260709 14:54:58.665345  3409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.000s	sys 0.019s
I20260709 14:54:58.624938 10171 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:58.650101  3411 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:58.667906  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:58.681270  2651 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:54:58.701772  3412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.012s	sys 0.022s
W20260709 14:54:58.703490  3412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.013s	sys 0.023s
W20260709 14:54:58.694231  2754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.006s	sys 0.029s
W20260709 14:54:58.706394  2754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.006s	sys 0.030s
I20260709 14:54:58.619496 10453 consensus_queue.cc:1048] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.166s
I20260709 14:54:58.715540  2751 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 14:54:58.633034 10157 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:58.730189  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.680015 10188 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:58.736613  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:58.681622 10451 raft_consensus.cc:697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:58.666646 10465 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.624418 10197 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.686954 10142 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:54:58.650380 10203 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.633428 10450 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.106s
I20260709 14:54:58.736675  2610 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
W20260709 14:54:58.744532  2758 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.003s	sys 0.063s
I20260709 14:54:58.707470 10385 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:54:58.699657 10152 raft_consensus.cc:2804] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:54:58.707301 10462 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:58.749256  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 386472us
I20260709 14:54:58.747455  2685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1910) took 2022 ms. Trace:
I20260709 14:54:58.774132  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.715875 10468 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.717245 10058 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:54:58.828732 10463 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 14:54:58.830070 10058 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.830574  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1906) took 1142 ms. Trace:
I20260709 14:54:58.830900 10058 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:58.831086  2604 rpcz_store.cc:276] 0709 14:54:57.688183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:57.688342 (+   159us) service_pool.cc:224] Handling call
0709 14:54:57.688820 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:57.691324 (+  2504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:57.691357 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:57.692020 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 14:54:58.354646 (+662626us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:54:58.354717 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.354821 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:58.830310 (+475489us) raft_consensus.cc:1685] finished
0709 14:54:58.830342 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.830500 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50,"raft.queue_time_us":114542,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":651648,"threads_started":1,"wal-append.queue_time_us":863482}
W20260709 14:54:58.742964 10319 consensus_meta.cc:287] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.934s	user 0.017s	sys 0.000s
I20260709 14:54:58.833251 10319 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 14:54:58.792515  2758 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.003s	sys 0.063s
I20260709 14:54:58.774448  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.707000 10369 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 14:54:58.707516 10171 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:58.801843 10455 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.305s	user 0.008s	sys 0.000s
I20260709 14:54:58.840772  2611 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 14:54:58.834441 10462 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:58.772701 10150 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:58.871368  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 320742us
I20260709 14:54:58.821041  2685 rpcz_store.cc:276] 0709 14:54:56.724745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:56.994492 (+269747us) service_pool.cc:224] Handling call
0709 14:54:56.995005 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:56.995416 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:56.995919 (+   503us) log.cc:844] Serialized 14 byte log entry
0709 14:54:58.740173 (+1744254us) log.cc:844] Serialized 14 byte log entry
0709 14:54:58.742376 (+  2203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:58.742442 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:58.747144 (+  4702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.747225 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.747384 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35170,"prepare.queue_time_us":8897,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":34184,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":4173,"thread_start_us":1717404,"threads_started":3,"wal-append.queue_time_us":876811}
W20260709 14:54:58.856554 10447 log.cc:927] Time spent T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 14:54:58.887766  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:58.892035  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:58.895561  2604 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:58.830804 10451 consensus_queue.cc:237] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:58.903595 10460 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 14:54:58.845717 10152 raft_consensus.cc:697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:54:58.907971 10152 consensus_queue.cc:237] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:58.909539  2649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.016s
W20260709 14:54:58.909997  2649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.008s	sys 0.016s
I20260709 14:54:58.911689 10185 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:58.912472 10185 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:58.912828 10185 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:58.913705 10185 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:58.919107 10458 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
W20260709 14:54:58.923223 10469 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 14:54:58.924552 10058 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:58.925787 10180 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:58.928830 10391 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.989s	user 0.012s	sys 0.002s
I20260709 14:54:58.930954 10203 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:58.931842 10259 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:58.932607  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 306799us
I20260709 14:54:58.933699  3411 leader_election.cc:400] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 17 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18.
I20260709 14:54:58.935910 10450 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:58.936609 10450 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:58.939637 10454 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.431s	user 0.005s	sys 0.004s
I20260709 14:54:58.848877 10468 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:58.943606 10468 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:58.843768 10197 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:54:58.748787 10188 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:58.962502  2603 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 27
I20260709 14:54:58.914525  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:54:58.977653  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1454) took 1436 ms. Trace:
W20260709 14:54:58.983124  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68677.3us
I20260709 14:54:58.984865  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:58.981263  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:58.790918 10465 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.978433  2756 rpcz_store.cc:276] 0709 14:54:57.541196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:57.541405 (+   209us) service_pool.cc:224] Handling call
0709 14:54:57.583767 (+ 42362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:57.584528 (+   761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:57.584577 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:57.585970 (+  1393us) log.cc:844] Serialized 52 byte log entry
0709 14:54:58.299102 (+713132us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:58.299341 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:58.299467 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:58.918141 (+618674us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:54:58.953904 (+ 35763us) raft_consensus.cc:1685] finished
0709 14:54:58.953928 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:58.977539 (+ 23611us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10139,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":16716,"spinlock_wait_cycles":41787648,"thread_start_us":688941,"threads_started":1,"wal-append.queue_time_us":1160649}
I20260709 14:54:58.804857 10276 consensus_queue.cc:237] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:58.942641 10450 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:58.955510 10362 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:58.989281 10185 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:59.011694  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.011994  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:58.983729 10203 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:54:59.021438  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.032155  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.035215  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.036532  2680 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:59.038604  2682 raft_consensus.cc:1275] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 14:54:59.040150  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.041117  2681 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:54:59.042536  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.043437  2677 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:54:58.999742 10319 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:54:59.047187  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112462us
I20260709 14:54:59.019769 10482 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.004673 10164 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:54:59.070326  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.070410  2686 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:54:59.072239  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.070529  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.023842 10203 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:59.076387  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.046454 10180 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.085594  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.086968  2685 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
I20260709 14:54:59.082541  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.115604  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.120704  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1956) took 1010 ms. Trace:
I20260709 14:54:59.121846  2754 rpcz_store.cc:276] 0709 14:54:58.109889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:58.119027 (+  9138us) service_pool.cc:224] Handling call
0709 14:54:58.119435 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:58.119969 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:58.120018 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:58.120760 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 14:54:58.707091 (+586331us) raft_consensus.cc:1645] Marking committed up to 16
0709 14:54:59.106100 (+399009us) log.cc:844] Serialized 14 byte log entry
0709 14:54:59.120180 (+ 14080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:59.120318 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:59.120392 (+    74us) raft_consensus.cc:1685] finished
0709 14:54:59.120405 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:59.120607 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13554,"raft.queue_time_us":149,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":928287,"threads_started":2,"wal-append.queue_time_us":814942}
I20260709 14:54:59.084580 10203 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:54:59.138178  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.081815 10319 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:54:59.150009  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:59.153201  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 76929us
I20260709 14:54:59.153275  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:59.156878  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:59.158411  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:54:59.162961  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.119176 10126 raft_consensus.cc:3060] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:54:59.072525 10164 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:59.167567  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.172765  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:59.175530  2759 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:54:59.177088  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.068950 10482 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:59.171890 10486 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:54:59.152825 10462 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.105870 10180 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:59.198184  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1954) took 1090 ms. Trace:
I20260709 14:54:59.196808  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:59.172752 10457 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.664s	user 0.001s	sys 0.003s
I20260709 14:54:59.200349  2759 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:59.201716  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.203135  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.192821  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:59.216043  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.211386  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.211592  2685 raft_consensus.cc:1240] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 22. Current term is 23. Ops: []
I20260709 14:54:59.223729  2575 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:54:59.191665 10157 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.162565 10474 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 14:54:59.192509 10486 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:54:59.201753  2758 rpcz_store.cc:276] 0709 14:54:58.108218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:58.114959 (+  6741us) service_pool.cc:224] Handling call
0709 14:54:58.115318 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:54:58.115663 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:54:58.835069 (+719406us) log.cc:844] Serialized 14 byte log entry
0709 14:54:59.197628 (+362559us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:54:59.197718 (+    90us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:59.197804 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:59.197908 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:59.198104 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":939018,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4941,"thread_start_us":1069041,"threads_started":2}
I20260709 14:54:59.270044 10385 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:54:59.294212  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 131274us
I20260709 14:54:59.295033  2754 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:54:59.295431 10385 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:59.246580  2650 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:59.269037  2759 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:54:59.260862  2678 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 14:54:59.270864  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.271500  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:54:59.307787  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 92659.3us
I20260709 14:54:59.274319  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:54:59.308382  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.277573  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:54:59.307941  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 415960us
I20260709 14:54:59.260356  2683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1486) took 1207 ms. Trace:
I20260709 14:54:59.273144  2608 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 14:54:59.275322  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.312933  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:54:59.244392 10150 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:54:59.314857  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:59.315338 10150 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 14:54:59.315744 10180 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.315901 10150 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:59.316030  2577 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:54:59.182168 10391 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 14:54:59.316952  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.218056 10473 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:54:59.215636 10415 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:59.259199 10468 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:54:59.259779 10492 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:54:59.239094 10362 consensus_queue.cc:1048] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 14:54:59.302503 10486 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:54:59.327862 10474 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:54:59.309754  2683 rpcz_store.cc:276] 0709 14:54:58.053268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:58.063564 (+ 10296us) service_pool.cc:224] Handling call
0709 14:54:58.083764 (+ 20200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:58.084159 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:58.084198 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:58.131321 (+ 47123us) log.cc:844] Serialized 26 byte log entry
0709 14:54:58.662691 (+531370us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:54:59.241677 (+578986us) log.cc:844] Serialized 14 byte log entry
0709 14:54:59.259940 (+ 18263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:59.260051 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:59.260099 (+    48us) raft_consensus.cc:1685] finished
0709 14:54:59.260116 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:59.260287 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17600,"prepare.queue_time_us":241960,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":17751,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":24284,"thread_start_us":958484,"threads_started":2,"wal-append.queue_time_us":789140}
I20260709 14:54:59.371968  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.327358 10415 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:54:59.339747 10468 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:59.346309 10157 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:54:59.399699  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 81225.8us
I20260709 14:54:59.357579 10188 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:59.394532  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:54:59.409685  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:54:59.371441 10473 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:54:59.375744 10490 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:59.410706  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:54:59.415437  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.380152 10474 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:59.385620 10489 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:59.370872 10481 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:54:59.403187 10468 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:59.357877 10391 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 14:54:59.408042 10367 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.430583  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.437820  2607 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:54:59.408854 10450 raft_consensus.cc:2804] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:54:59.420918  2608 raft_consensus.cc:3060] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:59.340369 10319 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.372479 10492 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:54:59.424744  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 129402us
I20260709 14:54:59.458392  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.375916 10150 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:59.476527  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 194244us
I20260709 14:54:59.477874 10501 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:54:59.478482 10501 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:54:59.478917 10501 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:54:59.484731  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.493278  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.504383  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.423642 10465 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:59.422798 10157 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:59.468421 10164 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:59.469331 10491 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.435845 10481 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:59.534379  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:59.470186 10171 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:59.413851 10473 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:54:59.423288 10369 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:59.464516  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:54:59.518615  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 202661us
I20260709 14:54:59.523310  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.519248  2678 raft_consensus.cc:2368] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
I20260709 14:54:59.531587  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.537472  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.546844  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:54:59.540386  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 279975us
I20260709 14:54:59.539124  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.518754 10450 raft_consensus.cc:697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:59.553259  2683 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:54:59.518157 10188 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:54:59.559553   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 3581)
I20260709 14:54:59.554273  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.425940 10499 consensus_queue.cc:1048] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 14:54:59.457036 10451 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 14:54:59.456719 10367 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:59.435570 10490 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:54:59.570073  2724 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:59.570747  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:59.567297  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.546657 10482 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:59.573985  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.538112 10481 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:54:59.456236 10489 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:59.519203 10484 consensus_queue.cc:1059] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 14:54:59.585391  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.584362  2608 raft_consensus.cc:2468] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:54:59.577607  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.556936 10171 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:54:59.583729  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.591873  2611 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:54:59.433964 10474 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.599591 10474 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:59.599802 10514 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.600546 10514 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:59.603097 10516 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:54:59.603667 10484 raft_consensus.cc:3055] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:54:59.604141 10484 raft_consensus.cc:740] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:54:59.541520 10491 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:59.606408 10490 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.598340  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.615496  2606 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:54:59.615621  2608 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 14:54:59.600102 10485 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 14:54:59.598210 10494 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:54:59.608429  2760 raft_consensus.cc:2410] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 14:54:59.527964 10415 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:54:59.597457  2603 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.612s	user 0.016s	sys 0.004s
W20260709 14:54:59.596237 10272 consensus_meta.cc:287] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.330s	user 0.013s	sys 0.010s
I20260709 14:54:59.635314  2603 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 27.
I20260709 14:54:59.558631 10369 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:59.613981  2611 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:54:59.635589  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.564388 10188 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:59.586511 10408 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 14:54:59.607053 10491 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.673935 10492 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:59.653544 10272 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.669694 10415 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:54:59.595333 10157 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:54:59.684247  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1911) took 1994 ms. Trace:
W20260709 14:54:59.687794 10512 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.252s	user 0.002s	sys 0.001s
I20260709 14:54:59.624244 10391 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:54:59.699144  2609 rpcz_store.cc:276] 0709 14:54:57.690107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:57.694057 (+  3950us) service_pool.cc:224] Handling call
0709 14:54:57.724183 (+ 30126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:58.449915 (+725732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:58.449972 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:58.493849 (+ 43877us) log.cc:844] Serialized 26 byte log entry
0709 14:54:59.004396 (+510547us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:54:59.004477 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:54:59.004922 (+   445us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:54:59.683998 (+679076us) raft_consensus.cc:1685] finished
0709 14:54:59.684024 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:54:59.684168 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42629,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":79,"thread_start_us":1053768,"threads_started":2,"wal-append.queue_time_us":715567}
I20260709 14:54:59.646955 10473 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.689913 10171 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:54:59.697728  2650 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:59.717505  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 154119us
I20260709 14:54:59.734318  2575 leader_election.cc:400] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
I20260709 14:54:59.735162  2575 leader_election.cc:403] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:54:59.733088  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:54:59.741717  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.688822 10514 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:54:59.748811  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.749708  2679 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 14:54:59.743798  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:59.753858  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:59.751228  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 225058us
W20260709 14:54:59.730686 10197 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.009s
I20260709 14:54:59.708843 10517 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:54:59.763754  2752 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 14:54:59.768676  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.770592 10188 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:54:59.709059 10272 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.773363 10272 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:54:59.778241 10126 consensus_meta.cc:287] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.001s
I20260709 14:54:59.760286 10171 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
W20260709 14:54:59.780495 10523 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 14:54:59.782637 10508 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 14:54:59.782966 10517 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.655697 10450 consensus_queue.cc:237] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:59.785868 10517 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:59.791348 10482 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.792116 10482 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:54:59.749352 10473 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:54:59.796379 10171 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:54:59.798656  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:54:59.799744  2755 raft_consensus.cc:2393] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:54:59.800109 10367 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:59.802414  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.802973 10489 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:54:59.803517  2686 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 14:54:59.806033 10126 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:59.811802  2752 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
W20260709 14:54:59.824831  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59429.6us
I20260709 14:54:59.828145  2606 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:54:59.829274  2607 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:54:59.831212  2726 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:54:59.813727  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:54:59.835971  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:54:59.835846  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:54:59.836239  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.836874  2681 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:54:59.838356  2606 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
W20260709 14:54:59.840185  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 96437.4us
W20260709 14:54:59.841256  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:54:59.841897  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:59.844971 10468 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:54:59.846962 10203 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.002s
I20260709 14:54:59.847524 10203 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:54:59.848268 10259 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 14:54:59.848922  2756 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:59.841092  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:59.855021 10482 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:54:59.840775 10185 consensus_meta.cc:287] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.000s
I20260709 14:54:59.855929 10453 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:54:59.856766 10453 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 14:54:59.860729  2604 raft_consensus.cc:2410] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 14:54:59.860949 10391 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:54:59.862147  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.845031 10500 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:54:59.863224  2612 raft_consensus.cc:2410] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 14:54:59.864473  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.865793  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.865931  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:54:59.832280 10164 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.867403 10164 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:54:59.875905 10142 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 14:54:59.878114  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:54:59.847867 10468 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:54:59.899596  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.903451  2612 raft_consensus.cc:1275] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 14:54:59.944720  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 367168us
I20260709 14:54:59.945170  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.946062  2606 raft_consensus.cc:2376] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22. Re-sending same reply.
I20260709 14:54:59.948729  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.958743  2606 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
W20260709 14:54:59.882941  3412 connection.cc:256] RPC call timeout handler was delayed by -1.0368332950000649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 14:54:59.965850  2755 raft_consensus.cc:1275] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 14:54:59.967540  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:54:59.884483 10203 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:54:59.883740 10453 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:54:59.987906  2651 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:54:59.988468  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 147246us
I20260709 14:54:59.951524 10482 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.981881  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:54:59.972641  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 146578us
I20260709 14:54:59.980749  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.994890  2612 raft_consensus.cc:2368] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21.
I20260709 14:54:59.981499  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:54:59.995877  2606 raft_consensus.cc:2393] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:54:59.986753 10272 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:54:59.970638  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.993973  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:54:59.997908  2757 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:59.965991 10489 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:54:59.999801 10521 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 14:55:00.002916 10486 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.611s	user 0.013s	sys 0.000s
I20260709 14:55:00.003114 10521 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:00.003322  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:00.003587 10521 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:00.003489 10486 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:00.005013  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:00.007308  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:00.009231  2612 raft_consensus.cc:1275] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:54:59.999716 10482 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:00.007017 10272 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:00.004133 10180 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:00.040278  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45784.5us
I20260709 14:55:00.015095  2759 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 17. Current term is 18. Ops: []
I20260709 14:55:00.032630  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.027160  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:00.024058  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:00.040297  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:00.043700  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54017.8us
I20260709 14:55:00.010788  2612 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 18. Current term is 19. Ops: []
I20260709 14:55:00.049641  3412 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.051934  3412 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.054862  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:00.054940  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:00.022760 10505 consensus_queue.cc:1048] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 14:54:59.971060 10462 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 14:55:00.068186  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:00.040845 10058 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:00.022209 10482 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:00.042891 10486 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:00.029918 10272 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:00.085346  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2084) took 1023 ms. Trace:
I20260709 14:55:00.062449 10126 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:55:00.090211  2756 rpcz_store.cc:276] 0709 14:54:59.069651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.069939 (+   288us) service_pool.cc:224] Handling call
0709 14:55:00.084996 (+1015057us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54000e6854. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:00.085247 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787810560}
I20260709 14:55:00.086423  2754 raft_consensus.cc:1240] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:55:00.057108 10473 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:00.092058  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:00.093098  2681 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:00.096803  3411 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:00.063277 10516 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:00.070340 10510 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:00.098737 10489 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:00.061200 10188 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 14:55:00.082304 10415 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:00.111312  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.111485 10367 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:00.029202 10481 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:00.143356  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.146721  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:00.126688 10501 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.647s	user 0.015s	sys 0.001s
I20260709 14:55:00.161825  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:00.129781 10500 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:00.153618 10126 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:00.161070 10522 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:00.164037  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.142721 10486 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:00.171917  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.177732  2609 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 26. Current term is 27. Ops: []
I20260709 14:55:00.149163 10518 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:55:00.184303  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 73056.3us
W20260709 14:55:00.185580  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.187927  2724 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.193928 10502 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:00.196877  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:00.198201 10491 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:00.198618 10491 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:00.199010 10491 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:00.205327 10502 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:00.215451 10319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.015s
W20260709 14:55:00.218572 10319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.015s
I20260709 14:55:00.221045 10491 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:00.268440  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 303290us
I20260709 14:55:00.273394  2577 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:00.278520 10517 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:00.279841 10517 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:00.280133 10276 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:00.280680 10517 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:00.280731 10276 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:00.281318  2603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 2132) took 1620 ms (client timeout 1636 ms). Trace:
W20260709 14:55:00.281957  2603 rpcz_store.cc:269] 0709 14:54:58.660526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:58.667472 (+  6946us) service_pool.cc:224] Handling call
0709 14:55:00.281009 (+1613537us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400128254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:00.281186 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636318464}
W20260709 14:55:00.276520 10533 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 14:55:00.289680  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:00.291731  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:00.293913 10462 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:00.294618 10462 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:00.295115 10462 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:00.310616 10185 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:00.311489 10185 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:00.339387  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 66099.4us
W20260709 14:55:00.339721  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.345415  2724 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.143749 10516 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:00.351528  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 262286us
I20260709 14:55:00.352730  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.354542 10276 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:00.355301 10126 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:00.154253 10415 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:00.158888 10367 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:00.168457 10259 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:00.372778  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55870 (request call id 2139) took 1317 ms. Trace:
W20260709 14:55:00.372609  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:00.373862  2604 rpcz_store.cc:276] 0709 14:54:59.055056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.055631 (+   575us) service_pool.cc:224] Handling call
0709 14:55:00.368392 (+1312761us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54002fe4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:00.372662 (+  4270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775799040}
I20260709 14:55:00.180615 10482 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:00.150691 10489 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:00.378410  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 206584us
I20260709 14:55:00.397600  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.376267  2610 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 14:55:00.398802  2610 pending_rounds.cc:85] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 16
I20260709 14:55:00.399223  2610 pending_rounds.cc:107] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 14:55:00.399590  2610 raft_consensus.cc:2889] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:00.384658  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.387925  3411 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:00.392235  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 53144us
I20260709 14:55:00.189282 10197 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:00.190325 10522 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:00.166954 10501 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:00.419313  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:00.426455  2724 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.393668 10232 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:00.372015 10259 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:00.536417  2759 raft_consensus.cc:1275] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:55:00.477361 10391 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:00.374886 10527 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:00.519160  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.183226 10486 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:00.395561 10518 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:00.545480  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 253002us
I20260709 14:55:00.545804  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:00.516438  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:00.560186   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:00.414388 10157 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:00.477329 10276 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:00.530786  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 468471us
W20260709 14:55:00.529925 10498 log.cc:927] Time spent T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 14:55:00.570389  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 143994us
W20260709 14:55:00.625059  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.625898  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.395131 10150 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
W20260709 14:55:00.580019  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 182503us
I20260709 14:55:00.627851  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:00.541597 10259 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:55:00.580018  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.532639 10164 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:00.638736  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 17.
W20260709 14:55:00.629984  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.632876  2652 leader_election.cc:400] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21.
I20260709 14:55:00.640663  2652 leader_election.cc:403] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:55:00.638264  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50090.5us
I20260709 14:55:00.640723  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.548309 10491 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:00.412142 10521 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:00.652338  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:00.652678  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72727.7us
I20260709 14:55:00.560470 10185 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:00.663414  2576 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.566584 10527 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:00.659186  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 424930us
I20260709 14:55:00.571123 10415 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:00.522079 10197 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:00.661335  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:00.585285 10188 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 14:55:00.668129  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 280333us
I20260709 14:55:00.675505  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:00.586874 10482 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:00.632019 10203 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:00.681877  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:00.525611 10532 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.132s
I20260709 14:55:00.647244 10232 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:00.685637  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:00.689173  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:00.702656  2754 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:00.704088  2759 raft_consensus.cc:2393] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 22: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:00.680672  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:00.692744  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:00.690806  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:00.419010 10484 consensus_queue.cc:260] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:00.705303  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:00.707885  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.682403 10276 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:00.717199  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 329258us
I20260709 14:55:00.673506 10055 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 21
I20260709 14:55:00.687446 10473 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:00.719568  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56202.5us
W20260709 14:55:00.720331  2724 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.672359 10527 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:00.685220 10529 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:00.724078  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:00.733047  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.737016  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.743871  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.714533 10478 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:00.751756  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:00.724727 10497 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:00.755514  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:00.764194  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:00.764683  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.764974  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.742023 10180 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:00.754619  2754 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:00.757587 10164 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:00.736629 10481 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:00.765226  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.747246 10188 raft_consensus.cc:2804] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:00.751114 10484 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:00.735895 10232 consensus_queue.cc:1048] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 14:55:00.776651  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:00.789147  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.799052  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:00.805902  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.807680 10188 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:00.810216 10180 consensus_queue.cc:1059] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:00.810907 10526 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:00.811678 10526 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:00.812263 10526 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:00.757560 10554 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:00.810217  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 53812.4us
I20260709 14:55:00.817540  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.742240 10492 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 14:55:00.819353  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 138713us
I20260709 14:55:00.806327  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:00.705654 10522 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:00.736349 10563 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
W20260709 14:55:00.890625  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92727us
W20260709 14:55:00.977074  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 155669us
I20260709 14:55:00.978049  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:00.796263 10468 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:00.982170  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 205590us
W20260709 14:55:00.984329  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.984962  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:00.960474  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 171444us
I20260709 14:55:00.785116 10473 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:55:00.997148  2576 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:00.800361 10559 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:00.822793 10185 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:00.823366 10188 raft_consensus.cc:3060] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:00.803223 10276 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:00.805497 10164 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.010362  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:00.999707 10367 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:01.012189 10512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.240s	user 0.049s	sys 0.036s
I20260709 14:55:01.013751 10391 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.014657  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.017674  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1980) took 1061 ms. Trace:
I20260709 14:55:00.720975 10518 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.126672  2751 rpcz_store.cc:276] 0709 14:54:59.956626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.956798 (+   172us) service_pool.cc:224] Handling call
0709 14:54:59.979781 (+ 22983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:59.980098 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:59.980125 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:59.997406 (+ 17281us) log.cc:844] Serialized 26 byte log entry
0709 14:55:00.539399 (+541993us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:00.539439 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.539537 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.017354 (+477817us) raft_consensus.cc:1685] finished
0709 14:55:01.017386 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.017590 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":499103,"threads_started":1,"wal-append.queue_time_us":971527}
I20260709 14:55:01.019460 10466 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 14:55:00.982137 10482 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:01.020766 10473 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:01.027153 10154 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 14:55:01.028069 10557 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 14:55:01.029227 10567 consensus_queue.cc:1048] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:01.050509  2754 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:01.063890  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:01.069531  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 72430.3us
W20260709 14:55:01.053366  2612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.014s	sys 0.016s
I20260709 14:55:01.008354 10481 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:01.011686 10545 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
W20260709 14:55:01.079418  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:01.035964 10524 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:00.991425  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.036710 10501 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.115568 10565 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:01.120880  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 180085us
W20260709 14:55:01.074985 10517 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.794s	user 0.001s	sys 0.013s
W20260709 14:55:01.123749  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 183376us
W20260709 14:55:01.109639 10512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.050s	sys 0.036s
W20260709 14:55:01.132187  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 117632us
I20260709 14:55:01.108636 10486 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:01.123478 10542 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
W20260709 14:55:01.135731  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126472us
W20260709 14:55:01.173622  2612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.015s	sys 0.021s
W20260709 14:55:01.210222  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 218850us
W20260709 14:55:01.173327  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.238981  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 65736.6us
W20260709 14:55:01.239828  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.240763  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.241132  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.180160  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.241994  2751 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:01.124164 10197 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.175773  2605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1474) took 1657 ms. Trace:
I20260709 14:55:01.229682  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.246817  2754 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:01.134287 10369 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.083120 10164 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.153064 10521 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:01.238713  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.251955  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.256597  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.256742  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.134651 10391 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.147035 10154 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:01.243127  2605 rpcz_store.cc:276] 0709 14:54:59.518366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.729443 (+211077us) service_pool.cc:224] Handling call
0709 14:54:59.759878 (+ 30435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:59.766376 (+  6498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:59.766432 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:59.790309 (+ 23877us) log.cc:844] Serialized 26 byte log entry
0709 14:55:00.030942 (+240633us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b18000b6260. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:55:00.214424 (+183482us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:55:00.214463 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.214536 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.148365 (+933829us) raft_consensus.cc:1685] finished
0709 14:55:01.148405 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.175664 (+ 27259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22179,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":36596992,"thread_start_us":183245,"threads_started":1,"wal-append.queue_time_us":963551}
I20260709 14:55:01.292232  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.294857  2751 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
W20260709 14:55:01.264130  2724 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.255610 10185 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:01.290170 10259 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.002s
I20260709 14:55:01.306613  2751 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 14:55:01.307848 10570 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.237071 10517 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.297420 10203 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:01.297027 10481 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:01.327373  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1987) took 1128 ms. Trace:
I20260709 14:55:01.230652 10524 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.332397  2752 rpcz_store.cc:276] 0709 14:55:00.200104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.212658 (+ 12554us) service_pool.cc:224] Handling call
0709 14:55:00.213140 (+   482us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:55:00.213768 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:00.213801 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:55:00.214547 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 14:55:00.480535 (+265988us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:55:00.480597 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.480704 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.326954 (+846250us) raft_consensus.cc:1685] finished
0709 14:55:01.326992 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.327269 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6610,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":265502,"threads_started":1,"wal-append.queue_time_us":467819}
I20260709 14:55:01.305493 10575 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:55:01.310943 10539 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.474s	user 0.001s	sys 0.001s
W20260709 14:55:01.313097 10552 log.cc:927] Time spent T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.420s	user 0.000s	sys 0.002s
I20260709 14:55:01.338793  2611 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 22. Current term is 23. Ops: []
W20260709 14:55:01.322630 10543 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 14:55:01.323792 10510 consensus_meta.cc:287] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.204s	user 0.014s	sys 0.012s
I20260709 14:55:01.341566  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.378870  2603 raft_consensus.cc:2410] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
W20260709 14:55:01.325745 10538 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.544s	user 0.001s	sys 0.001s
I20260709 14:55:01.348218 10577 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.320083 10259 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.372834 10517 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.376736  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1982) took 1412 ms. Trace:
I20260709 14:55:01.377897  2686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1932) took 1543 ms. Trace:
I20260709 14:55:01.400568  2686 rpcz_store.cc:276] 0709 14:54:59.834510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.834748 (+   238us) service_pool.cc:224] Handling call
0709 14:54:59.839760 (+  5012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:54:59.844861 (+  5101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:59.844909 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:54:59.845569 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 14:55:00.147322 (+301753us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:00.147397 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.147527 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.377559 (+1230032us) raft_consensus.cc:1685] finished
0709 14:55:01.377595 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.377808 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9750,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":279790,"threads_started":1,"wal-append.queue_time_us":549378}
I20260709 14:55:01.387454  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.336014 10562 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:01.404337 10571 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 14:55:01.398799  2758 rpcz_store.cc:276] 0709 14:54:59.964373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.964550 (+   177us) service_pool.cc:224] Handling call
0709 14:54:59.965369 (+   819us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:59.966675 (+  1306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:54:59.966709 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:54:59.984098 (+ 17389us) log.cc:844] Serialized 52 byte log entry
0709 14:55:00.216138 (+232040us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:00.530962 (+314824us) log.cc:844] Serialized 14 byte log entry
0709 14:55:00.531457 (+   495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.531570 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.376426 (+844856us) raft_consensus.cc:1685] finished
0709 14:55:01.376455 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.376647 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":794,"prepare.queue_time_us":453587,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":5228,"raft.queue_time_us":2004,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":36020,"spinlock_wait_cycles":67200,"thread_start_us":506572,"threads_started":2,"wal-append.queue_time_us":799034}
I20260709 14:55:01.382624 10510 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:01.422435  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:01.411016 10462 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 1.108s	user 0.011s	sys 0.008s
I20260709 14:55:01.322873 10570 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:01.436411  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.436661  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.437768  2611 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:01.384758 10126 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 14:55:01.377694 10575 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:01.462242  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 226383us
I20260709 14:55:01.463775  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:01.463912  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 227412us
I20260709 14:55:01.398200 10577 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.477182  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.394033 10154 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.478552  2605 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
W20260709 14:55:01.481388  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 160090us
I20260709 14:55:01.436074 10462 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.503448 10486 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.504390 10486 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.496050 10197 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.506759  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1985) took 1308 ms. Trace:
I20260709 14:55:01.523213  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1930) took 1755 ms. Trace:
W20260709 14:55:01.509872 10541 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.925s	user 0.000s	sys 0.002s
I20260709 14:55:01.511080  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1986) took 1312 ms. Trace:
I20260709 14:55:01.524936  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.525365  2753 rpcz_store.cc:276] 0709 14:55:00.198630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.203854 (+  5224us) service_pool.cc:224] Handling call
0709 14:55:00.204255 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:00.360611 (+156356us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:55:00.360648 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:00.361265 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 14:55:00.361455 (+   190us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:00.361493 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.510420 (+1148927us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540050e1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:01.510529 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.510596 (+    67us) raft_consensus.cc:1685] finished
0709 14:55:01.510614 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.510990 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358746496}
I20260709 14:55:01.512130 10486 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.512251 10577 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.511739  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2172) took 1496 ms. Trace:
W20260709 14:55:01.521968 10579 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 14:55:01.516857 10576 log.cc:927] Time spent T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 14:55:01.523765  2679 rpcz_store.cc:276] 0709 14:54:59.768353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.768552 (+   199us) service_pool.cc:224] Handling call
0709 14:54:59.768907 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:54:59.807333 (+ 38426us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 14:54:59.807693 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 14:55:00.084379 (+276686us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:55:00.085599 (+  1220us) log.cc:844] Serialized 52 byte log entry
0709 14:55:00.085837 (+   238us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:00.085891 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:00.086040 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.522921 (+1436881us) raft_consensus.cc:1685] finished
0709 14:55:01.522947 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.523125 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":1498,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":247,"thread_start_us":275720,"threads_started":1,"wal-append.queue_time_us":510954}
I20260709 14:55:01.528872  2610 rpcz_store.cc:276] 0709 14:55:00.015681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.118229 (+102548us) service_pool.cc:224] Handling call
0709 14:55:00.368603 (+250374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:00.422171 (+ 53568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.511460 (+1089289us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54002fe4d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:01.511661 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574784768}
W20260709 14:55:01.529213  2724 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:01.523475  2756 rpcz_store.cc:276] 0709 14:55:00.198216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.198945 (+   729us) service_pool.cc:224] Handling call
0709 14:55:00.199425 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:00.518768 (+319343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:00.518848 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:00.518921 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:00.518973 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.506417 (+987444us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000695d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:01.506495 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.506676 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733664128}
I20260709 14:55:01.535849 10259 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.537842 10524 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.540050 10561 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 14:55:01.540654 10561 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:01.515702 10197 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.543900 10516 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.545635 10197 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.521477 10489 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.546325 10154 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:01.547175  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2098) took 1063 ms. Trace:
I20260709 14:55:01.547866  2760 rpcz_store.cc:276] 0709 14:55:00.483933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.484294 (+   361us) service_pool.cc:224] Handling call
0709 14:55:01.546797 (+1062503us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:01.547026 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135003392}
W20260709 14:55:01.552559 10526 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.740s	user 0.016s	sys 0.000s
I20260709 14:55:01.553202 10526 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.554780  2678 raft_consensus.cc:1240] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
I20260709 14:55:01.556260 10575 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.558774 10570 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.538041 10462 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.559072  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.563436 10482 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.564647 10152 raft_consensus.cc:3055] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:55:01.564854  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.565165 10152 raft_consensus.cc:740] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:01.565374 10197 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.565708 10516 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:01.566176 10197 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.567451 10516 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.569957 10197 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.570170 10516 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.567118 10489 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:01.568157 10164 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.576290  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:01.578032 10055 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.831s	user 0.015s	sys 0.000s
I20260709 14:55:01.564257 10482 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:01.566459 10152 consensus_queue.cc:260] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.499908  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:01.580390   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:01.541810 10561 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.578653  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:01.580372  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:01.592746  2726 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.617916  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.581843  2609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1473) took 2078 ms. Trace:
I20260709 14:55:01.619284  2609 rpcz_store.cc:276] 0709 14:54:59.476546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:59.724545 (+247999us) service_pool.cc:224] Handling call
0709 14:54:59.748360 (+ 23815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:00.178336 (+429976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.554044 (+1375708us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002feed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:01.554504 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986804480}
I20260709 14:55:01.624742  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:01.601572 10527 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.850s	user 0.009s	sys 0.005s
I20260709 14:55:01.641278  2760 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 14:55:01.607822 10578 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.144s	user 0.002s	sys 0.000s
I20260709 14:55:01.632849  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:01.591578 10157 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.969s	user 0.015s	sys 0.000s
W20260709 14:55:01.580812 10484 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.770s	user 0.007s	sys 0.008s
I20260709 14:55:01.649956 10152 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:01.581847 10582 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 14:55:01.578624 10055 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21.
W20260709 14:55:01.605659 10555 log.cc:927] Time spent T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.890s	user 0.005s	sys 0.016s
W20260709 14:55:01.537879 10564 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.403s	user 0.004s	sys 0.000s
I20260709 14:55:01.635124  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.602173 10391 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.578962 10526 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.652791 10565 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:01.650089 10157 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:01.627422 10197 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:01.646271 10489 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:01.672933  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.674389  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.679334  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:01.690735  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:01.697393 10203 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:01.678088 10489 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:01.653753 10527 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.702541  2603 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:01.693450 10367 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:55:01.675962 10516 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:01.693223 10577 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.711032 10197 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.699936 10203 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:01.692572 10565 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 14:55:01.717665  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:01.724180  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.725904  2753 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 11.
I20260709 14:55:01.706144 10489 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:01.724903  2760 raft_consensus.cc:2376] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21. Re-sending same reply.
I20260709 14:55:01.746243  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.722169 10203 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:01.731143 10482 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.739374 10385 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.752251 10385 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.754374  2685 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:55:01.755692  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.757351  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.739617  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.760720  2755 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:01.743497  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.711072 10573 raft_consensus.cc:3060] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:01.751324  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.768436  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.773208  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.776036  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.774961  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.777211  2760 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
I20260709 14:55:01.775108  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.774441 10484 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.780298  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.781340  2608 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:01.749873 10462 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.789779  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:01.792495  2751 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:01.792990  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:01.794708  2758 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:55:01.794869  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:01.795461  2760 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:01.795980  2752 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:01.722412 10577 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:01.801611  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.729539 10197 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:01.827528  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:01.827868  2724 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.845140  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:01.844914  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.779898 10484 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.802973 10574 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:01.859004  2724 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.809510 10518 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:01.839027 10126 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:01.867002  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.799055 10462 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.847759 10367 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:01.896111  2751 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:01.891089  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.856892 10502 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:01.909543  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 102069us
I20260709 14:55:01.910895  2760 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:55:01.870378 10486 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.913195 10486 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:01.867161 10391 consensus_queue.cc:1059] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 14:55:01.915038 10391 raft_consensus.cc:3055] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:55:01.915621 10391 raft_consensus.cc:740] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:01.916564 10486 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.917385 10484 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.916838 10391 consensus_queue.cc:260] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.918624 10197 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.870496 10574 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.867540 10516 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 14:55:01.907953  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:01.946238  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 52073.8us
I20260709 14:55:01.947254  2683 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:01.925396  2605 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:01.949005  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1504) took 1253 ms. Trace:
W20260709 14:55:01.945307  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.949640  2677 rpcz_store.cc:276] 0709 14:55:00.695351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.695541 (+   190us) service_pool.cc:224] Handling call
0709 14:55:00.716593 (+ 21052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:00.765164 (+ 48571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:00.765223 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:00.765292 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:00.765360 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.948697 (+1183337us) spinlock_profiling.cc:243] Waited 48.2 ms on lock 0x7b54002b3754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:01.948755 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:01.948944 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110886656}
I20260709 14:55:01.950736  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.953613  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.954698  2680 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:01.910562 10276 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:01.925156 10462 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.955799 10559 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:01.959589  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.959765 10559 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.960851 10577 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:01.961220  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:01.962803  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:01.964439  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:01.965250  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.967082 10559 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.931699 10527 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.955798 10502 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:01.909202 10126 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:01.972795 10502 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:01.925315 10574 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:01.968763 10500 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:01.943614 10385 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:01.982528  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:01.957741  2724 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.987707  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:01.989336  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78843us
I20260709 14:55:01.988364  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.992944 10152 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:01.993692 10152 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:01.994035 10502 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:01.990628  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:01.996480  2724 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:01.999583 10527 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:02.001363 10581 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:02.001076 10527 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:02.004940  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:02.006054 10527 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:02.005973  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:01.996671 10152 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:02.008464 10573 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:02.009927  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.010066 10581 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:55:02.010684 10581 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:02.010545 10462 consensus_queue.cc:1048] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:55:02.011237 10581 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:55:02.012404  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:02.014148 10351 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 14:55:02.018959  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.023588 10502 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:55:02.025799  2724 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:02.027372  2724 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.031013  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.032480 10501 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:02.033267 10561 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:55:02.034405  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:01.901892 10367 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:02.038436  2612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2167) took 2150 ms. Trace:
I20260709 14:55:02.047502  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:02.060839  2756 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 14:55:02.069029  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 122012us
I20260709 14:55:02.071130  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:02.095651  2760 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:55:02.094466  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 84639.1us
W20260709 14:55:02.073459  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:01.918794 10391 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:02.085964  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:02.074137  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.098183  2577 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.084282  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:02.120653  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.122200  2751 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:02.039347  2612 rpcz_store.cc:276] 0709 14:54:59.888540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.055616 (+167076us) service_pool.cc:224] Handling call
0709 14:55:00.082337 (+ 26721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:00.340317 (+257980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:00.340355 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:00.403776 (+ 63421us) log.cc:844] Serialized 26 byte log entry
0709 14:55:01.951015 (+1547239us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18004f1860. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1
0709 14:55:01.951278 (+   263us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:01.951340 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.951455 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:01.951530 (+    75us) raft_consensus.cc:1685] finished
0709 14:55:01.951551 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:02.038218 (+ 86667us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179157,"raft.queue_time_us":198357,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":117,"spinlock_wait_cycles":5399424,"thread_start_us":699645,"threads_started":1,"wal-append.queue_time_us":747260}
I20260709 14:55:02.092638  2575 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:02.138785  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 54655us
I20260709 14:55:02.140573  3411 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.133227  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.145360  2759 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:02.158761  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.056368 10367 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:02.160145  2677 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
W20260709 14:55:02.163110  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:02.166467  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.177887  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.179260  2751 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:55:02.124222 10188 consensus_meta.cc:287] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 1.002s	user 0.012s	sys 0.004s
I20260709 14:55:02.184799  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.190480  2606 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:02.093699 10486 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:02.079187 10573 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:01.973155 10126 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:02.198136  2610 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:02.209964  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.211959  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:02.212260  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:02.213502  2751 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:55:02.213892  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.203763  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:02.215806  2603 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 23: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:02.230336  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:02.230409  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.231491  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:02.145816 10462 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:02.232272  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:02.160300 10563 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:55:02.248752  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.236289  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.250664  2603 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:02.253644  2723 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:02.255452  2603 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:55:02.233282  2681 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:02.195915 10518 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:55:02.217530 10185 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:02.256973  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.257608  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.275678  2609 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:55:02.264945 10203 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.000s
I20260709 14:55:02.218907 10188 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:02.258464  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:02.257862  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:02.261425  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.296358  2677 raft_consensus.cc:2393] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:02.120558 10319 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 14:55:02.205732 10517 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:02.219822 10486 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:02.266342  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.217916 10573 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:02.236474  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.173730 10559 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 14:55:02.273957  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:02.274441 10574 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:02.307296  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.275398 10462 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:02.247805 10539 log.cc:927] Time spent T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 14:55:02.310621  2751 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:55:02.311343  2605 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 27.
W20260709 14:55:02.312824  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.312980  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.316075  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.316393  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.325338  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:02.286706 10484 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:02.330015  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:02.333842  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76226.1us
I20260709 14:55:02.303790 10319 consensus_queue.cc:1243] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 4f1347e243fd46a5bb7cee5b2e23b665 log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:55:02.303470 10516 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 14:55:02.313287 10486 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:02.282032 10185 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:02.315766 10573 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:02.292023 10203 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:02.358361  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46581.7us
I20260709 14:55:02.375845  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:02.361192 10581 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:02.355458  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:02.337218 10489 consensus_meta.cc:287] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.002s
I20260709 14:55:02.331130 10484 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:02.327147 10500 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:02.376389  2679 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:55:02.335884 10197 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:02.411136  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:02.313643 10517 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:02.419294  2681 raft_consensus.cc:2468] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:02.425571  2679 raft_consensus.cc:1275] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:02.365420 10462 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:02.322252 10574 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:02.419682 10578 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 14:55:02.484340  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:02.353835 10529 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 14:55:02.492434  2608 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 14:55:02.417497 10484 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:02.517268 10126 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:02.518690 10126 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 14:55:02.519917 10126 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:02.520267 10126 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:02.520625 10126 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:02.523612 10596 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 14:55:02.407262 10489 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:02.562655  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 332256us
I20260709 14:55:02.420631 10197 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:02.568044  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:02.563572  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 465610us
I20260709 14:55:02.581769  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:02.583142  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:02.598747  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 243498us
I20260709 14:55:02.602901  2760 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
W20260709 14:55:02.610008  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.610915  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:02.611624   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 3581)
W20260709 14:55:02.579795  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 326243us
I20260709 14:55:02.609017  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.615813  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:02.617182  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:02.619503  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:02.621027  2723 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:02.624624  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.585338 10563 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:02.627813  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:02.629216  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 536683us
I20260709 14:55:02.595655 10462 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:02.626189  2577 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:02.603793 10185 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:02.629786  2576 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:02.626686  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 429909us
I20260709 14:55:02.629889  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.634688 10462 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:02.631569  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.639375 10462 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:02.639914  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:02.641542  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.619577 10164 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:02.621182 10150 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 14:55:02.643465  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.650179  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:02.650977  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.652848  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:02.637223 10188 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:02.643704 10157 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:02.649435 10559 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 14:55:02.649715 10502 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:02.675179 10581 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:02.686779 10517 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:02.677495 10559 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:02.683439 10164 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:02.682209 10188 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:02.710868 10574 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:02.712042 10581 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:02.735596 10581 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:02.736136 10581 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:02.737689 10600 log.cc:927] Time spent T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.221s	user 0.002s	sys 0.001s
I20260709 14:55:02.714594 10164 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:02.739311 10164 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:02.739991 10593 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
I20260709 14:55:02.678479 10612 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:02.743206 10622 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:02.744442 10622 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:02.745245 10622 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:02.734654 10595 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 14:55:02.743378 10612 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:02.703179 10157 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 14:55:02.704180 10203 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:02.685055 10502 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:02.739352 10598 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 14:55:02.759433 10602 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 14:55:02.761452  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.726562 10609 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:02.725158 10563 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:02.748862 10601 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
W20260709 14:55:02.757201 10597 log.cc:927] Time spent T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 14:55:02.747179 10614 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:55:02.727298 10188 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:02.792366  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.749477 10574 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 14:55:02.797292  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 147276us
W20260709 14:55:02.826977  2576 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:02.828549  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:02.764974 10559 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:02.777045  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 151094us
I20260709 14:55:02.831755  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.752646 10612 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:02.830596 10559 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:02.843360  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:02.845257  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:02.848219  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:02.771054 10197 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:02.771054 10489 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:02.867898  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 226454us
I20260709 14:55:02.868291  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:02.751389 10573 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:02.799466 10185 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:02.875116  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:02.804077 10203 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:55:02.888296  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 247579us
W20260709 14:55:02.893085  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 240297us
I20260709 14:55:02.821671 10614 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:02.897282  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 257522us
I20260709 14:55:02.898314  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 1941) took 1106 ms. Trace:
I20260709 14:55:02.901577  2604 rpcz_store.cc:276] 0709 14:55:01.792224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.792609 (+   385us) service_pool.cc:224] Handling call
0709 14:55:01.793958 (+  1349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.794215 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.794243 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.810154 (+ 15911us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.250289 (+440135us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:02.250340 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.250418 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:02.897993 (+647575us) raft_consensus.cc:1685] finished
0709 14:55:02.898031 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:02.898242 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":360126,"threads_started":1,"wal-append.queue_time_us":616316}
I20260709 14:55:02.899009  2577 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:02.885622 10602 log.cc:927] Time spent T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 14:55:02.806052 10126 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:02.893532 10574 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:02.906157 10150 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:02.964643  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:02.954097  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 184250us
I20260709 14:55:02.905514 10614 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:55:02.956920  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1506) took 1166 ms. Trace:
I20260709 14:55:02.917358 10489 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:02.894199 10462 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:02.897138 10203 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:02.996775  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 128621us
I20260709 14:55:02.991587  2757 rpcz_store.cc:276] 0709 14:55:01.791019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.802500 (+ 11481us) service_pool.cc:224] Handling call
0709 14:55:01.823132 (+ 20632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.823775 (+   643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.823829 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.851799 (+ 27970us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.309729 (+457930us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:02.309782 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.309868 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:02.745162 (+435294us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:02.942180 (+197018us) raft_consensus.cc:1685] finished
0709 14:55:02.942216 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:02.956799 (+ 14583us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5806,"raft.queue_time_us":165,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12631,"spinlock_wait_cycles":6588672,"thread_start_us":399682,"threads_started":1,"wal-append.queue_time_us":643801}
W20260709 14:55:02.990675 10561 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.953s	user 0.020s	sys 0.000s
W20260709 14:55:03.015841 10391 consensus_meta.cc:287] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.886s	user 0.013s	sys 0.005s
I20260709 14:55:02.993479 10604 consensus_queue.cc:1048] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 14:55:03.027666  2612 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:02.995661 10609 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.041292  2757 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:03.042881  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:02.965113 10517 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 14:55:02.966197 10574 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:02.970458 10607 consensus_queue.cc:1059] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 14:55:03.031874 10489 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.007892 10630 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:03.040638 10618 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.005141 10484 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.058050 10391 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:03.036401 10197 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:02.918869 10188 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.041445 10613 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.142794  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:03.065318  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 166407us
W20260709 14:55:03.095978  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:03.135032  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
W20260709 14:55:03.231163  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:03.232733  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 136867us
I20260709 14:55:03.103492  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.140000  2609 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 14:55:03.088512 10486 consensus_meta.cc:287] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.003s
I20260709 14:55:03.038151 10626 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:03.100688 10632 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 14:55:03.145874 10559 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:03.127759 10590 log.cc:927] Time spent T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.865s	user 0.004s	sys 0.001s
I20260709 14:55:03.089202  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:03.243180  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:03.078178 10502 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.066162 10628 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:03.209904  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.235742  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:03.100916 10484 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:03.235642  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.258908  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.145390  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:03.099943 10620 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 14:55:03.050091 10634 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:03.258285  2577 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:03.143177 10462 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.093436 10150 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:03.256067  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 109715us
I20260709 14:55:03.131476 10640 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.143850 10573 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.180371  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:03.261490  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:03.285403  2757 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
W20260709 14:55:03.285625  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:03.259455  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 49730.8us
I20260709 14:55:03.131232 10636 raft_consensus.cc:3055] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:03.140480 10574 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.141829 10203 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.089788 10625 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.092679 10630 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:03.267220 10626 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:03.263718 10502 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:03.259835 10559 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:03.268986  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 179875us
I20260709 14:55:03.271101 10643 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:03.093719 10618 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.246474 10613 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.275453 10627 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:55:03.275960 10581 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.539s	user 0.014s	sys 0.000s
I20260709 14:55:03.262678 10517 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:03.289698 10126 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.122792 10185 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:03.262792 10367 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 1.055s	user 0.017s	sys 0.002s
I20260709 14:55:03.276729 10608 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:03.277082  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:03.324570  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:03.249089 10612 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.090315 10489 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.271117  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1512) took 1099 ms. Trace:
I20260709 14:55:03.298306 10203 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:03.232151 10391 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.286186  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.255537 10628 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:03.052871 10609 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:03.342892  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57308.6us
I20260709 14:55:03.232455 10197 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.269104  2610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1492) took 1084 ms. Trace:
I20260709 14:55:03.324294 10185 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:55:03.273314 10591 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.969s	user 0.004s	sys 0.000s
I20260709 14:55:03.297081 10626 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:03.301410  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.07964e+06us
I20260709 14:55:03.288661 10636 raft_consensus.cc:740] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:03.311532  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2102) took 2728 ms. Trace:
I20260709 14:55:03.325202 10638 consensus_queue.cc:1059] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:03.326668 10618 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.297870 10630 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:03.343580 10164 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.002s
I20260709 14:55:03.276757  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.329023  2752 rpcz_store.cc:276] 0709 14:55:02.171929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:02.596256 (+424327us) service_pool.cc:224] Handling call
0709 14:55:02.596801 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:02.597252 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:02.597287 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:02.626471 (+ 29184us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.889192 (+262721us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:02.889286 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.889509 (+   223us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.270784 (+381275us) raft_consensus.cc:1685] finished
0709 14:55:03.270811 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.271034 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":22033,"thread_start_us":253315,"threads_started":1,"wal-append.queue_time_us":616912}
I20260709 14:55:03.328397  2611 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:55:03.302649 10645 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.280663 10634 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:03.344485  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.251304 10486 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.346743  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.309999 10517 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:03.346269  2610 rpcz_store.cc:276] 0709 14:55:02.185060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:02.237789 (+ 52729us) service_pool.cc:224] Handling call
0709 14:55:02.238339 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:02.627337 (+388998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:02.627390 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:02.632458 (+  5068us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.775933 (+143475us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:02.775990 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.776111 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.268774 (+492663us) raft_consensus.cc:1685] finished
0709 14:55:03.268804 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.269024 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112197,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5295,"thread_start_us":499794,"threads_started":2,"wal-append.queue_time_us":617711}
I20260709 14:55:03.288475 10640 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.360239  2754 rpcz_store.cc:276] 0709 14:55:00.583001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.259083 (+676082us) service_pool.cc:224] Handling call
0709 14:55:01.260289 (+  1206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.312785 (+ 52496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.312841 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.323791 (+ 10950us) log.cc:844] Serialized 26 byte log entry
0709 14:55:01.561871 (+238080us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b18001ad700. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e60d1 00007fa16e601a0f
0709 14:55:01.792404 (+230533us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:01.792469 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:01.792587 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.125876 (+1333289us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:03.311172 (+185296us) raft_consensus.cc:1685] finished
0709 14:55:03.311205 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.311413 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51777,"raft.queue_time_us":58702,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":9102,"spinlock_wait_cycles":20157952,"thread_start_us":230278,"threads_started":1,"wal-append.queue_time_us":809999}
I20260709 14:55:03.344589 10641 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 14:55:03.334230 10642 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:03.366402  2606 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:03.288111 10646 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 14:55:03.334649 10489 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.298103 10625 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.312460 10581 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.380630  2756 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:03.379804  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.341542 10608 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:03.344882 10628 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:03.362113  2757 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:55:03.385715  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:03.359503 10516 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.413216  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.389671  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:03.398465  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 54026.6us
I20260709 14:55:03.414795  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.309458 10643 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:03.424057 10599 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.905s	user 0.000s	sys 0.006s
I20260709 14:55:03.400851  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:03.424933  2611 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:03.425614  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1510) took 1626 ms. Trace:
I20260709 14:55:03.426442 10559 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.427855 10634 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:03.402325  2683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 1933) took 2596 ms (client timeout 1973 ms). Trace:
W20260709 14:55:03.428551  2683 rpcz_store.cc:269] 0709 14:55:00.798220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:00.798604 (+   384us) service_pool.cc:224] Handling call
0709 14:55:03.394466 (+2595862us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:03.394622 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944238464}
I20260709 14:55:03.342815 10203 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.406953 10612 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.407611 10608 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.406965 10581 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.408999 10627 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:03.412963 10516 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.331501 10367 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.384064 10517 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:03.418903  2754 raft_consensus.cc:2368] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
W20260709 14:55:03.419049  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.380887 10164 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.459399 10529 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:03.460067 10164 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.423251  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.461639  2605 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 21. Current term is 22. Ops: []
I20260709 14:55:03.462123  2757 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:03.462389  2604 raft_consensus.cc:1240] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:03.426072  2758 rpcz_store.cc:276] 0709 14:55:01.799204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.799371 (+   167us) service_pool.cc:224] Handling call
0709 14:55:01.799814 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.808935 (+  9121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.808962 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.850095 (+ 41133us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.221179 (+371084us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:02.587112 (+365933us) log.cc:844] Serialized 14 byte log entry
0709 14:55:02.597993 (+ 10881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.598108 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.414111 (+816003us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:03.425295 (+ 11184us) raft_consensus.cc:1685] finished
0709 14:55:03.425405 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.425550 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2804,"prepare.queue_time_us":49093,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":62,"raft.queue_time_us":41560,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":39276,"spinlock_wait_cycles":91673472,"thread_start_us":641293,"threads_started":2,"wal-append.queue_time_us":633181}
I20260709 14:55:03.382730 10645 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.367266 10626 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:55:03.429598  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 43980.9us
I20260709 14:55:03.429876 10605 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:03.432469 10636 consensus_queue.cc:260] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.397461 10618 raft_consensus.cc:3055] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:03.403347  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.440531  2611 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:03.490971  2759 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 14:55:03.491300  2760 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:03.398682  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.445889  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.446607  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:03.452770  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 1.0769e+06us
W20260709 14:55:03.493845  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74834.8us
I20260709 14:55:03.453692 10472 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:03.454643 10391 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:03.494647  2576 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 119 similar messages]
I20260709 14:55:03.456246  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1486) took 1286 ms. Trace:
I20260709 14:55:03.456550 10625 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.402730 10489 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:03.463616  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.549126  2758 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:03.474740  2577 leader_election.cc:400] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
I20260709 14:55:03.550379  2577 leader_election.cc:403] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 14:55:03.431721 10640 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.453292 10486 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.496824  2756 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 18: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:03.496629  2608 rpcz_store.cc:276] 0709 14:55:02.170086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:02.203198 (+ 33112us) service_pool.cc:224] Handling call
0709 14:55:02.240864 (+ 37666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:02.501241 (+260377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.455974 (+954733us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540011ec54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:03.456183 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577319296}
I20260709 14:55:03.526085  2757 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:03.543948  3411 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:03.554045  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 79580.4us
I20260709 14:55:03.555119  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.560482  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.560645  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.560954  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:03.561856  2758 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:03.418217 10630 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.466404 10276 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.566251  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:03.487767 10605 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:03.564168  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69994.9us
I20260709 14:55:03.592463  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:55:03.567808  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 105168us
I20260709 14:55:03.601738  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:03.602276  2760 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:03.490985 10618 raft_consensus.cc:740] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:03.548383 10612 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:03.550009 10472 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:03.595144 10646 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:03.608326  2577 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:03.610721  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:03.438016 10614 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.611593  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:03.612488   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:03.564445 10462 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27.
I20260709 14:55:03.563862 10522 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:03.623574  2650 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:03.547758 10638 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.557840 10489 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.601219 10639 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:55:03.595831 10486 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:03.609426 10276 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.460065 10529 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.604774 10559 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.615625 10574 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:55:03.564823 10391 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:03.639685  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:03.640990  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.666191  2760 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:55:03.675506  2757 raft_consensus.cc:1240] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 19. Current term is 20. Ops: []
W20260709 14:55:03.669052  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 65981.3us
I20260709 14:55:03.677451  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.601929 10605 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:03.648140  2606 raft_consensus.cc:1240] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 17. Current term is 18. Ops: []
I20260709 14:55:03.627202 10472 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:03.634123 10646 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:03.642794 10638 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.642607 10642 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.639531 10522 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.651562 10658 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.654528 10152 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:55:03.655055 10575 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:03.632603 10055 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.711158 10575 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.713815 10575 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:03.703222 10647 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 14:55:03.717981 10638 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.719417 10645 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.704742 10658 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.721393 10630 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.678578  2760 raft_consensus.cc:2368] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23.
I20260709 14:55:03.680567 10516 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.681958 10574 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:03.659109  2650 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:03.731364  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 82222.8us
I20260709 14:55:03.731801 10574 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:03.733122  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.733844  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.709250 10529 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.709642 10522 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.711370 10055 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.688337 10646 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:03.707885 10526 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 14:55:03.737046 10055 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.737885 10055 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:03.657940  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1944) took 1706 ms. Trace:
W20260709 14:55:03.688159 10650 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 14:55:03.699996 10642 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.738790  2677 rpcz_store.cc:276] 0709 14:55:01.951115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:02.342861 (+391746us) service_pool.cc:224] Handling call
0709 14:55:02.343379 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:02.343719 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:55:02.344284 (+   565us) log.cc:844] Serialized 14 byte log entry
0709 14:55:03.451113 (+1106829us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b18004cb460. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:55:03.554795 (+103682us) log.cc:844] Serialized 14 byte log entry
0709 14:55:03.657357 (+102562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:03.657450 (+    93us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:03.657535 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.657649 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.657851 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169742,"prepare.queue_time_us":4984,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":144,"raft.queue_time_us":1132704,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":5195,"spinlock_wait_cycles":11784064,"thread_start_us":201784,"threads_started":2,"wal-append.queue_time_us":116834}
I20260709 14:55:03.740607  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:03.737089  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.682550 10618 consensus_queue.cc:260] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:03.742626  2650 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.743119 10618 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:03.745219 10660 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:03.746060 10660 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:03.747756 10203 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.665506 10559 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.727840 10614 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.729382 10367 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.634999 10150 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:55:03.687836 10662 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:03.738142 10526 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:55:03.727388 10609 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.752833  3411 leader_election.cc:400] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23.
I20260709 14:55:03.764273  3411 leader_election.cc:403] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:55:03.743175  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.758186  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1994) took 1493 ms. Trace:
I20260709 14:55:03.767624  2751 rpcz_store.cc:276] 0709 14:55:02.264573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.757163 (+1492590us) service_pool.cc:224] Handling call
0709 14:55:03.757580 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.757815 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:03.757838 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:03.757889 (+    51us) raft_consensus.cc:1645] Marking committed up to 17
0709 14:55:03.757920 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.757984 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.758117 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:03.781239  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.782871  2751 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
W20260709 14:55:03.778538  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:03.688671 10665 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:03.755218 10648 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.351s	user 0.003s	sys 0.001s
W20260709 14:55:03.789798  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:03.757160 10150 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:03.758153 10526 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:03.741307 10642 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.797343  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:03.756207 10628 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:03.786744  2650 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.826987  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1993) took 1571 ms. Trace:
I20260709 14:55:03.780226 10055 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:03.746728 10660 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:03.827720  2760 rpcz_store.cc:276] 0709 14:55:02.255463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.730486 (+1475023us) service_pool.cc:224] Handling call
0709 14:55:03.731492 (+  1006us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:55:03.732316 (+   824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:03.732353 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:55:03.733196 (+   843us) log.cc:844] Serialized 52 byte log entry
0709 14:55:03.786455 (+ 53259us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:03.825791 (+ 39336us) log.cc:844] Serialized 14 byte log entry
0709 14:55:03.826255 (+   464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.826361 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.826418 (+    57us) raft_consensus.cc:1685] finished
0709 14:55:03.826436 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.826900 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":263,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":120,"thread_start_us":91298,"threads_started":2,"wal-append.queue_time_us":68126}
I20260709 14:55:03.750710 10559 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.751291 10658 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.774701 10609 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.842154 10609 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.790961  2758 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:03.757553 10662 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:03.845772  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1995) took 1570 ms. Trace:
I20260709 14:55:03.846123 10197 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.819012 10526 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.797914 10665 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.820667 10575 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.821317  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1500) took 2058 ms. Trace:
I20260709 14:55:03.824173  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.750722 10643 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.826398  2650 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:03.811673 10630 raft_consensus.cc:3060] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 14:55:03.853854  2755 rpcz_store.cc:276] 0709 14:55:01.762909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.764921 (+  2012us) service_pool.cc:224] Handling call
0709 14:55:01.767921 (+  3000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.768878 (+   957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.768933 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.791685 (+ 22752us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.302801 (+511116us) raft_consensus.cc:1645] Marking committed up to 9
0709 14:55:02.302871 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.303050 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:03.746782 (+1443732us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b24001ccf88. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b597fb0 00007fa16e20e488 00007fa16e20ebe8 00007fa16e1e6398 00007fa16e1e176a 00007fa16e1e0bb6
0709 14:55:03.811934 (+ 65152us) raft_consensus.cc:1685] finished
0709 14:55:03.811962 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.821221 (+  9259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":342,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":4989,"spinlock_wait_cycles":5621632,"thread_start_us":449740,"threads_started":1,"wal-append.queue_time_us":812888}
I20260709 14:55:03.787556 10673 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:03.835232 10150 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.883193 10673 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.884186 10646 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.836516 10672 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:03.837682 10628 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.847579  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:03.886549 10672 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:03.887200 10185 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.004s
I20260709 14:55:03.887418 10673 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.887801 10185 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.846328  2758 rpcz_store.cc:276] 0709 14:55:02.275333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.844667 (+1569334us) service_pool.cc:224] Handling call
0709 14:55:03.845131 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.845376 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:03.845406 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:03.845460 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:03.845514 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.845578 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:03.845696 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:03.891378  2758 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:55:03.892408  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1998) took 1600 ms. Trace:
I20260709 14:55:03.892882  2758 rpcz_store.cc:276] 0709 14:55:02.292572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.890473 (+1597901us) service_pool.cc:224] Handling call
0709 14:55:03.891135 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.891845 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.892332 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:03.850407  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.894753  2683 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 18.
I20260709 14:55:03.834421  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.854645  2751 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:55:03.896418  2760 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 11.
I20260709 14:55:03.896860  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.856158  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:03.897586  2760 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:03.852769 10658 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:03.904263  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 117710us
W20260709 14:55:03.904544  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 57089.6us
I20260709 14:55:03.859823  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:03.866897 10575 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.868781  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:03.912534  2650 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:03.872773 10665 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.874483 10662 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.880525 10609 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.859244 10526 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:03.898367  2677 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:03.898380  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1999) took 1588 ms. Trace:
W20260709 14:55:03.898780  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 297111us
I20260709 14:55:03.859110  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.907431  2682 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:03.869585 10517 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.917218  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.927780  2760 rpcz_store.cc:276] 0709 14:55:02.309803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.897125 (+1587322us) service_pool.cc:224] Handling call
0709 14:55:03.897370 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.898127 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.898333 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:03.936892  2685 raft_consensus.cc:2393] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:03.927155  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:03.903270 10672 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:03.903487 10645 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:03.941210  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:03.919757 10643 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:03.913846 10385 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:55:03.946542 10472 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:03.947991  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:03.949105 10646 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:03.914491 10658 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.951265 10575 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.959777  2677 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:03.997757  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.998181  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:03.944288 10517 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:03.939417  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:04.001468  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 62699.4us
I20260709 14:55:04.002610  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.003343  2686 raft_consensus.cc:2376] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26. Re-sending same reply.
I20260709 14:55:03.962980 10670 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.008683 10613 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.008801 10670 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:03.963512 10672 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:04.011441 10672 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.012094 10658 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:03.963935 10626 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:03.965828 10185 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:03.966081 10630 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23.
W20260709 14:55:03.966372 10674 log.cc:927] Time spent T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 14:55:03.967295  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 1992) took 1712 ms. Trace:
I20260709 14:55:04.015079  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:03.918987 10671 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:04.015869  2684 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:04.016323 10671 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:04.017438 10671 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.001195 10660 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.977888 10643 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.020702 10626 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.020927 10660 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.021472 10643 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.012126 10672 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:04.015514  2757 rpcz_store.cc:276] 0709 14:55:02.255168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.565117 (+1309949us) service_pool.cc:224] Handling call
0709 14:55:03.565540 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.676160 (+110620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:03.966960 (+290800us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000e5e54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:03.967214 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252315008}
I20260709 14:55:04.023546  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:03.976049 10645 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:03.975399  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.026504  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:04.029397  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.990795  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.030503  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:04.031507  2757 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:04.034035 10489 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.035077  2683 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:04.041961  2757 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:55:04.042428 10276 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.004338  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.004947  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:04.063555  2650 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.043135 10670 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.043996  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:04.050197  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:04.064942  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:03.932119 10639 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:04.007913 10652 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.009s
W20260709 14:55:04.065641  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.065953  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.054874  2726 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.070979  2757 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:04.062356 10677 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:04.085574 10645 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.085132 10677 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:04.087010 10652 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.001s	sys 0.010s
I20260709 14:55:04.088593  2686 raft_consensus.cc:2410] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 14:55:04.089730 10643 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:04.067736 10605 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:04.080852  2650 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.099916  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:04.101385  2650 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.104127  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.104307  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.108139  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.109807  2678 raft_consensus.cc:2410] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 14:55:04.062641 10517 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.114073  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.114960  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.115942  2608 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.105465  2604 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 22.
I20260709 14:55:04.111955  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.119608  2686 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 14:55:04.112907  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 2002) took 1033 ms. Trace:
I20260709 14:55:04.113835  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:04.120203  2650 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.120828  2612 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
W20260709 14:55:04.121457  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.121533 10185 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.122478 10185 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.121898  2760 rpcz_store.cc:276] 0709 14:55:03.079689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.956827 (+877138us) service_pool.cc:224] Handling call
0709 14:55:03.957275 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.957521 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:55:03.968654 (+ 11133us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.112319 (+143665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.112458 (+   139us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:04.112533 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.112643 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.112822 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21797,"prepare.queue_time_us":10238,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"thread_start_us":125581,"threads_started":1}
I20260709 14:55:04.124809  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:04.125732 10185 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.129505  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.129834  2686 raft_consensus.cc:1240] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:55:04.132941  2760 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:04.136402  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.137331  2609 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 17.
I20260709 14:55:04.139498  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:04.140218  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.140311  2605 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:04.140750  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.138810  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.097730 10643 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:04.151551  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.154079  2757 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
W20260709 14:55:04.156356  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.157652  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.156322  2577 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.160355  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:04.161103  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:04.161418  2678 raft_consensus.cc:2368] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 23. Current term is 24.
I20260709 14:55:04.163751  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.164601  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:04.163395  2576 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.169389  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:04.172501  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:04.173245  2760 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:55:04.143362 10185 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:04.175107  3411 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:04.175086 10639 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.176582  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.187994  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.182307 10605 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.128664 10517 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.188663  2760 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:04.191459  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:04.185051  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.197389  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.199033  2608 raft_consensus.cc:2410] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:55:04.162727 10670 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:04.200188  2684 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
I20260709 14:55:04.164781 10150 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:04.198606  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.208053 10677 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.209357 10605 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:55:04.210119 10605 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:04.211243 10605 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.212757  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.214277  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.214892 10606 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:04.215724  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.217373  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.217695 10517 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.218256  2685 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 18.
I20260709 14:55:04.221709  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.222373  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.222631  2684 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
W20260709 14:55:04.205137  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.223372  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.224992  2723 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.225436  2680 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:04.225172  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:04.204171  2650 leader_election.cc:336] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.228910  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.229903  2752 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:04.231529  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.232193  2752 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:04.233072  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.194614 10619 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:04.242164  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.242960  2752 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:04.202558 10645 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.244938  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.241744  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.246690  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.248041 10626 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:04.248637 10526 consensus_queue.cc:1059] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:55:04.256785 10526 raft_consensus.cc:3055] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:04.257215 10526 raft_consensus.cc:740] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:04.257373  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.222272 10628 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:04.258250 10526 consensus_queue.cc:260] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.259057 10644 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:04.222674 10670 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.259668 10644 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.242380 10643 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.261582 10276 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:04.252297 10677 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:04.255570 10645 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.227288 10673 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.268352  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.249485 10671 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 14:55:04.251686 10619 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:04.250483  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 2001) took 1171 ms. Trace:
I20260709 14:55:04.263878  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.258550  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.240768 10185 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:55:04.260018 10526 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:04.281059  2758 rpcz_store.cc:276] 0709 14:55:03.079323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.924048 (+844725us) service_pool.cc:224] Handling call
0709 14:55:03.943284 (+ 19236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.943592 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:55:03.943936 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.108303 (+164367us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.249736 (+141433us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b1800423f40. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b4d3c0a 00007fa174c13219 00007fa16b55ba40 00007fa16b5564e4 00007fa16b556455 00007fa16e63e060 00007fa16e650716 00007fa16e661287 00007fa16e661208
0709 14:55:04.249969 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.250041 (+    72us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:04.250114 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.250204 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.250405 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145262,"prepare.queue_time_us":18152,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":172708,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":17061,"spinlock_wait_cycles":80363648,"thread_start_us":137774,"threads_started":1,"wal-append.queue_time_us":229053}
I20260709 14:55:04.230376 10150 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:04.262143 10276 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.273100  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:04.270407 10500 raft_consensus.cc:2804] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:04.290527  2723 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.273344  2752 raft_consensus.cc:1275] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 14:55:04.261914 10670 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.285378  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.272861  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.290275  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.269897 10673 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.296239  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.275105 10636 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:04.297731  2758 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:55:04.305624  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:04.292438  2576 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.279965 10652 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:04.280217 10626 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:04.293298 10367 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:04.297885  2723 leader_election.cc:400] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 23. Current term is 24.
W20260709 14:55:04.291910  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.292224  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2117) took 1007 ms. Trace:
I20260709 14:55:04.306797  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.297672  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:04.307899  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.306457  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:04.291265 10276 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:04.290390 10604 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.310711  2723 leader_election.cc:403] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 14:55:04.311041  2752 rpcz_store.cc:276] 0709 14:55:03.284871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.272507 (+987636us) service_pool.cc:224] Handling call
0709 14:55:04.272919 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.291911 (+ 18992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.292163 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:04.291050 10622 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:04.297952 10683 log.cc:927] Time spent T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 14:55:04.294523 10232 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.308965 10489 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:55:04.309161 10559 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:04.344388  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:04.345445  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 52326.6us
I20260709 14:55:04.346259 10671 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:04.307389 10639 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:04.347113 10150 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.310074 10670 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.352881  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.304783 10673 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.361136 10677 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.385476  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
W20260709 14:55:04.353295  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60886.6us
I20260709 14:55:04.344676 10604 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:04.391283  2576 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.311555 10644 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.392300  2576 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.395032  2752 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 14:55:04.398880  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2120) took 1102 ms. Trace:
I20260709 14:55:04.408773  2758 raft_consensus.cc:1240] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 14:55:04.408985  2752 rpcz_store.cc:276] 0709 14:55:03.296624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.392138 (+1095514us) service_pool.cc:224] Handling call
0709 14:55:04.394650 (+  2512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:04.398371 (+  3721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.398760 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:04.410569  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2118) took 1121 ms. Trace:
I20260709 14:55:04.411635  2758 rpcz_store.cc:276] 0709 14:55:03.289588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.390070 (+1100482us) service_pool.cc:224] Handling call
0709 14:55:04.408353 (+ 18283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:04.410003 (+  1650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.410474 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 14:55:04.370283 10232 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.401036  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 2003) took 1313 ms. Trace:
W20260709 14:55:04.398533  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:04.369668 10185 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 14:55:04.392634 10677 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:04.383272 10622 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.383670 10613 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.379674 10489 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.369019 10203 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 23. Current term is 24.
I20260709 14:55:04.399766 10628 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:04.402868 10658 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.448150 10658 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:04.449805 10666 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.672s	user 0.002s	sys 0.000s
I20260709 14:55:04.414608  2757 rpcz_store.cc:276] 0709 14:55:03.087636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.205199 (+1117563us) service_pool.cc:224] Handling call
0709 14:55:04.242385 (+ 37186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.242734 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 14:55:04.268991 (+ 26257us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.400354 (+131363us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.400566 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.400619 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:04.400686 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.400768 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.400941 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1992,"raft.queue_time_us":9289,"spinlock_wait_cycles":6400,"thread_start_us":124621,"threads_started":1}
I20260709 14:55:04.420811 10622 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.455912 10622 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.383904 10619 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.460083  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1511) took 2294 ms. Trace:
I20260709 14:55:04.460625  2759 rpcz_store.cc:276] 0709 14:55:02.165717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:02.516181 (+350464us) service_pool.cc:224] Handling call
0709 14:55:02.516702 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:03.491540 (+974838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.459737 (+968197us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:04.460009 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239636224}
I20260709 14:55:04.403194 10652 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:04.418329 10677 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.388799 10670 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:04.392860 10604 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.414867  2576 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.402676 10367 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:04.426298 10613 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.423046 10489 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:04.434439  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.435175  2606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38962 (request call id 1506) took 1148 ms. Trace:
I20260709 14:55:04.513926  2606 rpcz_store.cc:276] 0709 14:55:03.286413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.458140 (+171727us) service_pool.cc:224] Handling call
0709 14:55:03.458545 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:03.686085 (+227540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.434813 (+748728us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540016dad4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:04.435108 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432164096}
I20260709 14:55:04.435340  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.436100  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.436699 10185 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.516361 10645 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.444032 10607 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.517544 10607 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.520387 10607 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.447022  2751 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 14:55:04.522248  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.523087  2751 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:04.489671 10516 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.528273 10516 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:04.447403 10595 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 1.628s	user 0.009s	sys 0.009s
I20260709 14:55:04.468883  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2119) took 1173 ms. Trace:
I20260709 14:55:04.476987  3412 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.478542  2758 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 14:55:04.312520 10672 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.482695  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.489404 10658 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.489585 10517 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.417577 10232 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.496963 10673 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.545487  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.504786  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.546283  2677 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:55:04.509028  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 110547us
I20260709 14:55:04.463347 10652 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.516029 10185 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.440737  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.531405  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.532239  2756 rpcz_store.cc:276] 0709 14:55:03.295255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.398288 (+103033us) service_pool.cc:224] Handling call
0709 14:55:04.468529 (+1070241us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b540040e7d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:04.468775 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7378688}
I20260709 14:55:04.534379  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2124) took 1227 ms. Trace:
I20260709 14:55:04.576264  2758 rpcz_store.cc:276] 0709 14:55:03.306801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.413035 (+1106234us) service_pool.cc:224] Handling call
0709 14:55:04.478157 (+ 65122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:04.533338 (+ 55181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.534289 (+   951us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 14:55:04.535346  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 52764.5us
W20260709 14:55:04.535315  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 58478.9us
I20260709 14:55:04.539450  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.546470  2606 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:55:04.582386  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.582741  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.512637 10606 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:04.505904 10367 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:04.587438 10367 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.587815 10626 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:04.589125 10626 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:55:04.589613 10626 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:04.572644  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.591462  2607 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 17.
I20260709 14:55:04.592720  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.592959  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.586377 10518 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 14:55:04.604346 10188 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.605511 10482 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:04.606456 10482 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.610474 10482 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.600884  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.583192  2606 raft_consensus.cc:3060] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:04.503712 10670 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.586591  2609 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: []
W20260709 14:55:04.534034 10502 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.109s	user 0.028s	sys 0.027s
I20260709 14:55:04.612663  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.617673  2605 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:04.539611 10645 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:04.613523  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.623260  2611 raft_consensus.cc:2393] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 27: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:04.618191  2608 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:04.594547  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.540485 10667 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:04.546252 10607 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.618674  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:04.633759   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:04.636020  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1503) took 2840 ms. Trace:
I20260709 14:55:04.636879  2753 rpcz_store.cc:276] 0709 14:55:01.781604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:01.783317 (+  1713us) service_pool.cc:224] Handling call
0709 14:55:01.784522 (+  1205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:01.785602 (+  1080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:01.785657 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:01.792076 (+  6419us) log.cc:844] Serialized 26 byte log entry
0709 14:55:02.145978 (+353902us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:02.211317 (+ 65339us) log.cc:844] Serialized 14 byte log entry
0709 14:55:02.586691 (+375374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:02.586808 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:04.447025 (+1860217us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b24001ccf88. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b3df9c4 00007fa16b3ddacb 00007fa16b598dc9 00007fa16b598a55 00007fa16b59809d 00007fa16e1ca10e 00007fa16e1e188c 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221
0709 14:55:04.614737 (+167712us) raft_consensus.cc:1685] finished
0709 14:55:04.621180 (+  6443us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.621345 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5451,"prepare.queue_time_us":71542,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":92957,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":9228,"spinlock_wait_cycles":16391936,"thread_start_us":677924,"threads_started":2,"wal-append.queue_time_us":790460}
I20260709 14:55:04.642170  2753 pending_rounds.cc:85] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 11
I20260709 14:55:04.642550  2753 pending_rounds.cc:107] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 14:55:04.642975  2753 raft_consensus.cc:2889] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:04.626775 10667 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:04.621573 10502 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.028s	sys 0.028s
I20260709 14:55:04.653244  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.545409 10517 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.649251  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.667847  2608 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:04.590575 10626 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.650146 10667 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:04.628196 10662 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 14:55:04.649045 10645 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:04.679169  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 381603us
I20260709 14:55:04.680051  2607 raft_consensus.cc:1240] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:04.681131  2605 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:04.627023 10607 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.660696 10658 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:04.669646 10516 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.696269  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.692970  2606 raft_consensus.cc:2468] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 28.
I20260709 14:55:04.706243  2608 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:55:04.730726  2754 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:04.737485  2760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2116) took 1456 ms. Trace:
I20260709 14:55:04.744578 10472 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:04.700307 10658 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.744815 10673 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.731283  2609 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:04.750000  2609 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:04.729470  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.703091 10626 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:04.688786 10276 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.756008  2605 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:04.715804  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:04.751185  2609 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.760854  2609 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:04.708704 10500 raft_consensus.cc:697] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:04.749473  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.742517  2760 rpcz_store.cc:276] 0709 14:55:03.281667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.198194 (+916527us) service_pool.cc:224] Handling call
0709 14:55:04.199955 (+  1761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:04.285782 (+ 85827us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:55:04.285810 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:04.303397 (+ 17587us) log.cc:844] Serialized 26 byte log entry
0709 14:55:04.531279 (+227882us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:55:04.531340 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.531444 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:04.737192 (+205748us) raft_consensus.cc:1685] finished
0709 14:55:04.737223 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.737406 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73169,"raft.queue_time_us":86399,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":4467,"thread_start_us":179882,"threads_started":1,"wal-append.queue_time_us":325041}
I20260709 14:55:04.775527  2612 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26. Ops: []
I20260709 14:55:04.774428  2685 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:04.776607  2608 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:55:04.766350  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:04.712769 10575 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:55:04.758409 10658 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:04.792538  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.790971  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.799448  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:04.800475  2608 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:04.777767 10613 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.781167 10575 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:04.794430  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.798416 10150 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 14:55:04.817379  2606 raft_consensus.cc:3055] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:04.842151  2606 raft_consensus.cc:740] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:04.836098  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1529) took 1146 ms. Trace:
I20260709 14:55:04.786029 10672 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:04.796804 10319 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:04.841033  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 46713.6us
I20260709 14:55:04.851074  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.801656 10385 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:04.825402 10652 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.845814  2757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1524) took 1289 ms. Trace:
I20260709 14:55:04.842092 10150 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:04.830935 10185 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.876454 10658 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.881765 10518 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.882607 10518 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.885155 10518 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:04.887365 10689 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 14:55:04.889061 10691 log.cc:927] Time spent T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 14:55:04.772997 10645 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.857214 10667 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.852551  2609 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 14:55:04.847124  2606 consensus_queue.cc:260] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:04.898300  2609 pending_rounds.cc:85] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 12
I20260709 14:55:04.898732  2609 pending_rounds.cc:107] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 14:55:04.899322  2609 raft_consensus.cc:2889] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:04.825026 10607 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.868460 10150 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:55:04.876454 10232 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.803712 10605 consensus_queue.cc:1059] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 14:55:04.842702  2756 rpcz_store.cc:276] 0709 14:55:03.690071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.575208 (+885137us) service_pool.cc:224] Handling call
0709 14:55:04.575593 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.575879 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 14:55:04.828150 (+252271us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.835355 (+  7205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.835424 (+    69us) raft_consensus.cc:1645] Marking committed up to 14
0709 14:55:04.835492 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.835566 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.836014 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3633,"prepare.queue_time_us":183140,"thread_start_us":185023,"threads_started":2}
I20260709 14:55:04.891242  2652 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:04.845681 10492 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:04.899011  2606 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:04.810951  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:04.900519  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2128) took 1292 ms. Trace:
W20260709 14:55:04.901206  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 110371us
I20260709 14:55:04.825616 10677 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.907964  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.927515  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:04.911733  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 100999us
I20260709 14:55:04.898926 10672 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:04.909901 10492 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:04.914048 10385 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:55:04.902304 10575 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:04.915764 10232 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:04.920462  2682 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:04.943069  2756 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:55:04.911240  2755 rpcz_store.cc:276] 0709 14:55:03.608564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.464964 (+856400us) service_pool.cc:224] Handling call
0709 14:55:04.465351 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.465678 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:55:04.759520 (+293842us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.900178 (+140658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.900237 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:04.900292 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.900355 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.900454 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":320587,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":1927,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6529,"thread_start_us":425541,"threads_started":2}
I20260709 14:55:04.930366  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:04.935053  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.905879 10517 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:04.911471 10565 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:04.941177  2680 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:55:04.941794  2681 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:04.863835 10319 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:04.947036  2758 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:55:04.937935  2606 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 14:55:04.850091 10500 consensus_queue.cc:237] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:04.949597  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:04.954993  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.957324  2606 pending_rounds.cc:85] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 11
I20260709 14:55:04.963225  2606 pending_rounds.cc:107] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 14:55:04.963605  2606 raft_consensus.cc:2889] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:04.939730 10492 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:04.937541 10673 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:04.966574  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.937857 10385 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:04.968079  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:04.973712  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:04.927439 10652 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 14:55:04.976060  2652 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:04.968329 10697 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 14:55:04.964818 10704 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.968600 10575 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:04.982765  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 134342us
I20260709 14:55:04.966385 10565 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.969878 10232 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:04.975569 10472 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:04.975777 10672 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:04.961848 10505 consensus_queue.cc:1059] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 14:55:04.976562 10385 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:04.978205 10630 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:04.975241 10606 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 14:55:04.996981  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.004391  2679 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 14:55:05.128536  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.129765  2678 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:55:05.153452  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.070173  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.154994  2604 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:05.131930  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.070768  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.022478  2677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1531) took 1029 ms. Trace:
I20260709 14:55:05.159296  2677 rpcz_store.cc:276] 0709 14:55:03.993004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.997335 (+  4331us) service_pool.cc:224] Handling call
0709 14:55:05.022159 (+1024824us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400444654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:05.022350 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258631424}
I20260709 14:55:05.069375  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.161761  2677 raft_consensus.cc:1240] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
W20260709 14:55:05.042639  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.163916  2679 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
W20260709 14:55:04.991851  2724 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:05.161687  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 235992us
I20260709 14:55:05.019989 10702 raft_consensus.cc:2804] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 14:55:05.061856 10319 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.059867 10667 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:05.054894 10604 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.002s
I20260709 14:55:04.980547 10701 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:05.169400 10606 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:55:05.170853  2686 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:05.175467  2686 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:05.170591 10604 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:05.001794 10705 log.cc:927] Time spent T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 14:55:05.177392  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 180497us
I20260709 14:55:05.165977  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:04.993342 10652 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:05.186151  2578 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.070188 10629 raft_consensus.cc:3055] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 14:55:05.187947  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 196156us
I20260709 14:55:05.191985  2755 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:55:05.194483  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:05.186450  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.180765  2686 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:05.174512 10462 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 14:55:05.174888 10606 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:05.203833  2751 raft_consensus.cc:1240] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
W20260709 14:55:05.183133 10489 consensus_meta.cc:287] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.005s
W20260709 14:55:05.197347  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.019011 10518 consensus_queue.cc:1059] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.168s
I20260709 14:55:05.182730 10577 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:04.992139 10704 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:05.193429 10702 raft_consensus.cc:697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:05.001353 10188 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.190s
I20260709 14:55:05.207747  2686 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:05.051170 10367 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.038029 10670 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:05.001061 10565 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:05.194588  2724 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:05.213853  2650 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:05.281186  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:05.214743  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.194123 10482 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:05.172029 10630 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.247023  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.247956  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.281035  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:05.286234  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:05.210137 10622 consensus_meta.cc:287] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.754s	user 0.016s	sys 0.000s
I20260709 14:55:05.193663 10629 raft_consensus.cc:740] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:05.203838 10276 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.273777  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.246722  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:05.250505  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.198695 10652 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.212172 10677 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:05.255967  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.295583  2752 raft_consensus.cc:1240] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:05.248709 10561 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:05.294642  2755 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 26: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:55:05.248127 10351 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:05.298264  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103712us
I20260709 14:55:05.296307  2679 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 14:55:05.296912  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 92091us
I20260709 14:55:05.294338  2677 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 21.
I20260709 14:55:05.301533  2683 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
W20260709 14:55:05.294483  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47930.7us
I20260709 14:55:05.308704  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:05.243724 10645 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 14:55:05.219050 10489 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.302716 10612 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:05.351771 10612 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.353737 10502 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:05.324515 10704 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:05.354513 10502 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:05.355515 10704 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.327143 10670 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:05.356215 10565 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.329692  2751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1528) took 1661 ms. Trace:
I20260709 14:55:05.303535 10561 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.357118 10502 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:05.273568 10563 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 14:55:05.315732 10517 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:05.359133 10517 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.359951 10604 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.292172 10622 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.363871 10367 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.310011 10692 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.310017 10351 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.385960 10622 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.387179  2756 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:05.369345 10472 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:05.356978  2751 rpcz_store.cc:276] 0709 14:55:03.668683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.523954 (+855271us) service_pool.cc:224] Handling call
0709 14:55:04.524305 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:05.209337 (+685032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.329362 (+120025us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400335ed4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:05.329608 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562247424}
I20260709 14:55:05.391680  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.393911  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.341466 10658 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:05.328238  2759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42502 (request call id 2005) took 1758 ms. Trace:
I20260709 14:55:05.358536 10612 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.399477 10630 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.380595 10561 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.408941  2759 rpcz_store.cc:276] 0709 14:55:03.570081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.461840 (+891759us) service_pool.cc:224] Handling call
0709 14:55:04.462357 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.525500 (+ 63143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:04.525547 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.525605 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:55:04.525650 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.327853 (+802203us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x7b540050dcd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:05.327935 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:05.328145 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144479488}
I20260709 14:55:05.410856  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.411613  2759 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:05.343528 10606 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:05.416637 10710 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.467s	user 0.002s	sys 0.000s
W20260709 14:55:05.420616  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.400967 10517 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:05.387187 10629 consensus_queue.cc:260] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.367511 10150 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.300565 10518 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:55:05.445077  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.446738  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.449996  2751 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:05.450979  2603 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 14:55:05.337024 10492 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.407572 10652 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:05.433357  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 153114us
W20260709 14:55:05.440033  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 131417us
I20260709 14:55:05.483120  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:05.442327  2686 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 14:55:05.386569 10351 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.475098  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.476789  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.494860  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:05.496307  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 203217us
I20260709 14:55:05.496675  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.422312 10385 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:05.500001  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:05.506021  2680 raft_consensus.cc:1240] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
I20260709 14:55:05.500635  2759 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:55:05.501688  3412 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.500542  2610 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 22: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:05.507529  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.508288  2751 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:55:05.508836  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:05.456175 10630 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:05.447872 10695 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:05.512919  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.516391  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:05.513459  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.503513  2760 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 14:55:05.526399  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.498906 10692 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 14:55:05.541337  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.488958 10232 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.510665  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.552757  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:05.554241  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 43694.1us
I20260709 14:55:05.555711  2607 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:05.552794 10482 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:05.504843 10501 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:05.558598  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:05.514740 10630 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:05.496307 10667 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 14:55:05.543879  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.489885 10517 consensus_queue.cc:1048] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 14:55:05.504904 10367 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:05.538300 10726 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 14:55:05.570604  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 356786us
I20260709 14:55:05.561213  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.507299 10518 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 14:55:05.517103 10652 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 14:55:05.573808  2610 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:05.563331  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.505089 10608 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 14:55:05.563235 10563 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:05.579783  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:05.580080 10613 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:05.572016 10367 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:05.584252  2682 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:05.562462 10501 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:05.585948  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:05.589295  2754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2126) took 2136 ms (client timeout 1827 ms). Trace:
I20260709 14:55:05.589730  2612 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:05.574067 10630 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:05.589949  2754 rpcz_store.cc:269] 0709 14:55:03.445037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:03.446229 (+  1192us) service_pool.cc:224] Handling call
0709 14:55:05.580817 (+2134588us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400423bd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:05.580960 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2844259712}
I20260709 14:55:05.562178 10482 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.543795 10702 consensus_queue.cc:237] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.544652 10695 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.580606 10726 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:05.605695  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1536) took 1578 ms. Trace:
I20260709 14:55:05.611765  2753 rpcz_store.cc:276] 0709 14:55:04.027800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.641437 (+613637us) service_pool.cc:224] Handling call
0709 14:55:04.641916 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:55:04.645776 (+  3860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:04.645823 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:55:05.029451 (+383628us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b2c000cb098. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16cf3fa11 00007fa16cf4f372 00007fa16e681a01 00007fa16e64643a 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2
0709 14:55:05.030495 (+  1044us) log.cc:844] Serialized 52 byte log entry
0709 14:55:05.276275 (+245780us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:05.288563 (+ 12288us) log.cc:844] Serialized 14 byte log entry
0709 14:55:05.289488 (+   925us) log.cc:844] Serialized 14 byte log entry
0709 14:55:05.291800 (+  2312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.291884 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:05.605314 (+313430us) raft_consensus.cc:1685] finished
0709 14:55:05.605374 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:05.605606 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1989,"prepare.queue_time_us":45401,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":3347,"raft.queue_time_us":11199,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":15314,"spinlock_wait_cycles":61056640,"thread_start_us":197071,"threads_started":1,"wal-append.queue_time_us":478830}
I20260709 14:55:05.604089  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:05.616659  2686 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 19: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:05.588521 10367 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:05.592770 10501 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:05.626327  2606 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:55:05.628192  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:05.628365  2649 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.601938 10489 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:05.634552   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:05.636148  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.635241  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.637964  2683 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:05.597373 10609 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:05.647557  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:05.660965  2682 raft_consensus.cc:2368] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:55:05.662364  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.663311 10150 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:05.665560  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.600766 10718 consensus_queue.cc:1059] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 14:55:05.668255  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:05.662482 10482 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.623750 10639 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 14:55:05.676128 10673 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:05.682742 10673 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:05.683229 10673 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:05.683383 10319 raft_consensus.cc:3055] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:05.683910 10319 raft_consensus.cc:740] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:05.685002 10319 consensus_queue.cc:260] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.686403 10319 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:05.676499 10492 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:05.688925 10716 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.438s	user 0.000s	sys 0.002s
I20260709 14:55:05.667714 10489 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:05.641760 10613 consensus_queue.cc:1059] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 14:55:05.679004  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104284us
I20260709 14:55:05.681265  2575 leader_election.cc:400] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:55:05.704686  2575 leader_election.cc:403] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 14:55:05.679412  2682 raft_consensus.cc:2393] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:55:05.694725  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 127613us
W20260709 14:55:05.673974  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 219701us
W20260709 14:55:05.716629  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 44882.1us
W20260709 14:55:05.717434  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:05.718019  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:05.669689 10609 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:05.749343  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 121372us
I20260709 14:55:05.706957 10489 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.736862 10612 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:05.759758 10489 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:05.759963 10695 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.716717 10726 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.771690 10489 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:55:05.760686 10660 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
W20260709 14:55:05.777804 10727 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.194s	user 0.002s	sys 0.000s
W20260709 14:55:05.780973 10708 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.831s	user 0.003s	sys 0.000s
W20260709 14:55:05.782217 10731 log.cc:927] Time spent T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 14:55:05.781970  2726 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.754209 10736 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.784224 10489 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:05.785007  2603 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 14:55:05.785621  2603 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:05.792719  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.786615  2603 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:05.797925  2603 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:05.799211  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.807590  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:05.807788  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.809401  2678 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:05.762882 10639 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 14:55:05.800382 10482 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:05.814581 10500 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.810529 10692 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.822254  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1954) took 1390 ms. Trace:
I20260709 14:55:05.823310 10609 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:05.825695  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1953) took 1394 ms. Trace:
I20260709 14:55:05.826246  2681 rpcz_store.cc:276] 0709 14:55:04.431782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.433614 (+  1832us) service_pool.cc:224] Handling call
0709 14:55:04.433935 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.953610 (+519675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.825326 (+871716us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b540012cd54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:05.825617 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167378304}
I20260709 14:55:05.831466  2684 rpcz_store.cc:276] 0709 14:55:04.432183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.435762 (+  3579us) service_pool.cc:224] Handling call
0709 14:55:05.821322 (+1385560us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b580025fb98. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b521b71 00007fa16b521a22 00007fa17443ac09 00007fa173594ee0 00007fa17365e0ab 00007fa17365e131 00007fa17361ffc0 00007fa17360a929 00007fa17360a3d2 00007fa16cb81bd4
0709 14:55:05.821610 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:05.821861 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:05.821885 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:05.821948 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:05.821988 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.822050 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:05.822174 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21240576}
I20260709 14:55:05.832456 10630 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.833333 10612 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
W20260709 14:55:05.837303  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.866863 10630 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:05.867355  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:05.869715  2612 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:05.876303  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:05.866508  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:05.899267  2576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.896250 10609 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:05.905373  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.904964 10609 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:05.911803  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 610326us
W20260709 14:55:05.912745  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:05.913378  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:05.911715  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.884756 10673 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:05.919118  2608 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 19.
W20260709 14:55:05.917882  2650 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:05.920058  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:05.925091  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:05.925529  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:05.926782  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:05.930545 10714 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.981s	user 0.004s	sys 0.000s
W20260709 14:55:05.930769  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:05.931430  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:05.935065  2603 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:55:05.911029 10609 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:05.936761  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 192823us
I20260709 14:55:05.924167 10489 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:05.937184 10501 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:05.949573 10517 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 14:55:05.998660  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:05.970238  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:05.992002  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2121) took 2693 ms. Trace:
I20260709 14:55:06.003890  2752 rpcz_store.cc:276] 0709 14:55:03.298960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.410538 (+1111578us) service_pool.cc:224] Handling call
0709 14:55:04.411033 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:05.300825 (+889792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:05.991584 (+690759us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:05.991911 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025378560}
I20260709 14:55:06.004060  2677 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:05.988065 10723 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:55:05.976575 10489 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:06.015332  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 97371.5us
I20260709 14:55:05.990299 10608 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:06.012661 10745 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:05.979290  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:06.037367  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58183.9us
I20260709 14:55:05.954095 10609 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:06.038504  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.019518 10723 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:55:06.040042  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 109325us
I20260709 14:55:06.028018 10489 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.180289  2612 raft_consensus.cc:1240] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:04.892335  2757 rpcz_store.cc:276] 0709 14:55:03.556283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.451284 (+895001us) service_pool.cc:224] Handling call
0709 14:55:04.451717 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:04.451965 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:55:04.763628 (+311663us) log.cc:844] Serialized 14 byte log entry
0709 14:55:04.845435 (+ 81807us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:04.845495 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:04.845558 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:04.845631 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:04.845751 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1200,"prepare.queue_time_us":298094,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":2510,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":41817,"thread_start_us":389506,"threads_started":2}
I20260709 14:55:06.179883  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.149536  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:06.151613  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:06.105360  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:06.188815  2605 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:06.130609  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:06.047904  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 108596us
W20260709 14:55:06.041612  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:06.122895  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:06.042049  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.191232  2608 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 22
I20260709 14:55:06.192828  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.036870 10726 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.189662  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.232964  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.194952  2612 raft_consensus.cc:1240] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:06.232995  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:06.235548  2752 raft_consensus.cc:2368] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
W20260709 14:55:06.223217  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 117964us
W20260709 14:55:06.231706  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 190114us
I20260709 14:55:06.047631 10646 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:06.062387 10742 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 14:55:06.112141 10629 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 14:55:06.128104 10718 raft_consensus.cc:3055] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 14:55:06.242669  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:06.083525 10639 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.123471 10753 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.060276 10673 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.209040 10728 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:06.051957 10501 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.123713 10713 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:06.244630  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:06.072492 10622 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.064551 10692 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.142113 10126 raft_consensus.cc:3055] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:06.230562 10630 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:06.091449 10671 raft_consensus.cc:3055] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:06.041782 10609 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.106544 10319 consensus_queue.cc:1059] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.146s
I20260709 14:55:06.253953 10718 raft_consensus.cc:740] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.171103 10605 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.269033 10717 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:06.055805 10367 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.129577 10695 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:06.250409 10646 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.275939 10622 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.280110 10501 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 14:55:06.278035 10692 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.250092 10726 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.265622 10639 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.265937 10753 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.286219 10606 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:06.291743 10639 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.285426 10717 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:06.278481 10671 raft_consensus.cc:740] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.088964 10723 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.298607 10717 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:06.300141 10723 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:06.301337 10692 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.302037 10671 consensus_queue.cc:260] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.290206 10695 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.294447 10646 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.294447 10609 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.303789 10671 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:06.274617 10713 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.306061 10695 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.296131 10726 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.307278 10713 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.292110 10753 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.307881 10622 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:06.310323 10717 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.304352 10718 consensus_queue.cc:260] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.311873 10718 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:55:06.215636  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 173607us
W20260709 14:55:06.192236  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.470726 10753 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.471830  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:06.472725  2575 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.244318  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:06.473449  2755 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 14:55:06.351894  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 149118us
I20260709 14:55:06.245750  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:06.256290  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:06.481671  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 225324us
I20260709 14:55:06.482419 10609 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:55:06.483704  2650 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.483551 10609 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.484150 10385 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:06.485105  2650 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.388677  2605 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 28.
W20260709 14:55:06.265399  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 142577us
I20260709 14:55:06.276832  3412 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:55:06.489415  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 212465us
W20260709 14:55:06.400936  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.404054 10367 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.273908 10126 raft_consensus.cc:740] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.406769 10489 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.408211  2756 raft_consensus.cc:1240] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 21. Current term is 22. Ops: []
I20260709 14:55:06.410140 10466 consensus_queue.cc:1059] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:06.419288  2685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1545) took 2188 ms (client timeout 1795 ms). Trace:
I20260709 14:55:06.432471  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.435526 10185 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:06.289517 10501 consensus_queue.cc:1243] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 14:55:06.503293 10185 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:06.439637 10505 consensus_queue.cc:1175] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "56620e612f0549e6abf320aa80398e74" responder_term: 22 status { last_received { term: 22 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 22 index: 15 } }
I20260709 14:55:06.448112 10717 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.454041 10662 consensus_queue.cc:1059] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 14:55:06.508572 10185 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:06.472034 10753 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.509464 10126 consensus_queue.cc:260] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.287075 10605 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.480078 10622 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:06.481848  2609 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 19: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:06.492729  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:06.396304  2608 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:06.363049  2611 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:55:06.511358 10726 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:06.513466 10753 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:06.517884 10574 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:55:06.534993 10466 raft_consensus.cc:3055] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:06.535892 10466 raft_consensus.cc:740] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:55:06.536051  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 104331us
I20260709 14:55:06.539548 10726 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:06.540179 10702 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:55:06.540280 10662 raft_consensus.cc:3055] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:06.544699 10662 raft_consensus.cc:740] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:06.540293 10726 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:06.498690  2685 rpcz_store.cc:269] 0709 14:55:04.230898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:04.231257 (+   359us) service_pool.cc:224] Handling call
0709 14:55:06.418624 (+2187367us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400268254. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:06.418937 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215497088}
I20260709 14:55:06.532027 10574 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.527863 10622 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:06.555497 10728 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.556211 10574 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.557359 10672 raft_consensus.cc:3055] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:55:06.555608 10622 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:06.558022 10672 raft_consensus.cc:740] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:06.547386 10662 consensus_queue.cc:260] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.537741 10466 consensus_queue.cc:260] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.558043 10574 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:06.556525 10728 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.595816  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:06.596935  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:06.598469 10466 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:06.595137  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63529.5us
I20260709 14:55:06.598963 10489 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:06.599522 10489 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:06.599920 10489 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:06.601189  2424 maintenance_manager.cc:419] P 3f6dfe8cfa964684b547b44a4b308ca2: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033364
I20260709 14:55:06.601836  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:06.598140  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:06.603808  2724 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.604620  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.606185 10500 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:06.607172 10741 consensus_queue.cc:1059] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 14:55:06.609140 10619 raft_consensus.cc:3055] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:06.609714 10619 raft_consensus.cc:740] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.611205 10482 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:06.611501 10619 consensus_queue.cc:260] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.613616  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:06.613585  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 58691.6us
I20260709 14:55:06.614771  2606 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:06.616540  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.593065  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.619060 10717 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.620002 10717 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.621590 10662 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:06.622736  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:06.625222  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.627614 10126 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:06.627667  2577 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:06.635840   431 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:06.641520 10728 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:06.647709 10605 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.625005  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:06.581782 10609 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.638911 10717 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:06.662060  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 60458.9us
W20260709 14:55:06.627133 10646 consensus_queue.cc:1175] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "56620e612f0549e6abf320aa80398e74" responder_term: 23 status { last_received { term: 23 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 23 index: 12 } }
W20260709 14:55:06.678582  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74871.9us
I20260709 14:55:06.661290  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:06.683987  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59249.9us
I20260709 14:55:06.688174  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:06.690667  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:06.713115 10517 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:06.713753 10517 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:06.714108 10517 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:06.710894 10505 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.715303 10505 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.717902 10505 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.718452 10185 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:55:06.719862  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:06.720542  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.721537 10741 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.722254  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:06.722335 10741 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.723366 10695 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.722524 10629 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.723452  2686 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:06.755070  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.756085  2611 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:06.755751 10629 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.758371 10629 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.725169 10619 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:06.726333  2612 raft_consensus.cc:1240] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 14:55:06.759042 10695 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.727351 10157 consensus_queue.cc:1059] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:06.735090 10150 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:55:06.735920  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.738246  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:06.719408 10185 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.739409  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:06.742367 10672 consensus_queue.cc:260] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:06.744185  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 116804us
I20260709 14:55:06.706096 10609 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:06.745792  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:06.746762  2650 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:06.755097  2724 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.752442  3412 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:06.756132  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:06.761389 10157 raft_consensus.cc:3055] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:06.779582 10157 raft_consensus.cc:740] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:06.779189 10609 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.779865  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:06.780439 10672 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:06.780925 10157 consensus_queue.cc:260] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.782335 10609 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.785622  2606 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:06.787699  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.782881 10753 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:06.767026  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.752923 10717 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 14:55:06.794149 10157 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:06.799361  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:06.801379  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:06.808634  2752 raft_consensus.cc:1240] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 14:55:06.803061  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.809861  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.803772  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:06.829473  2680 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 14:55:06.789009 10150 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 14:55:06.831030 10702 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.831745 10702 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:06.832361  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 110345us
W20260709 14:55:06.834102  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126227us
I20260709 14:55:06.839249 10605 consensus_queue.cc:1059] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:06.840489 10605 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:06.841240 10605 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.843178 10501 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.791375 10728 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:06.803493  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.844378 10188 raft_consensus.cc:3055] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:06.839273 10518 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:06.850317 10188 raft_consensus.cc:740] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
W20260709 14:55:06.850538  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103834us
I20260709 14:55:06.851316  2607 raft_consensus.cc:1275] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 14:55:06.851643  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.852180  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.851774 10188 consensus_queue.cc:260] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.842615 10605 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.853543 10188 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:06.783809 10622 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:06.854530  2650 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.844921 10501 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.855249  2650 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.857072 10622 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.861265 10501 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.861686 10385 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:06.861730 10728 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.863018 10753 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.863363 10185 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:06.884172 10728 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:06.885037  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 136047us
I20260709 14:55:06.885633 10726 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.886657  2651 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.888361 10385 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:06.888907 10489 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.891120  2754 raft_consensus.cc:1240] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 26. Current term is 27. Ops: []
I20260709 14:55:06.892556  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:06.895488 10154 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:06.896831 10154 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:06.898094 10154 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:06.905639 10319 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.906598 10319 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:06.909161 10517 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.912003  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:06.866550  2756 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:06.869935  2681 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:06.811568 10612 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.861949 10505 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:06.861420 10529 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:06.879133 10723 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:55:06.880497  2758 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 14:55:06.881124 10753 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:06.914600  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:06.840291 10741 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.920588  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:06.867661 10150 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:06.926987  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 81795.2us
I20260709 14:55:06.875571 10629 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.933485  2682 raft_consensus.cc:2410] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 14:55:06.872318 10609 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:06.938512  2611 raft_consensus.cc:2410] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 14:55:06.940495 10150 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:06.941792  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 186867us
W20260709 14:55:06.943102  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:06.909231 10319 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:06.933982 10646 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:06.968602 10646 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:06.970587  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:06.970956  2754 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 14:55:06.974720 10765 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 14:55:06.971024 10629 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:06.964730  2607 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 19.
I20260709 14:55:06.977363 10622 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:55:06.977494 10629 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:06.978121 10622 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:06.980150 10489 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:06.981444  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:06.989600  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:06.991101  2608 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 25: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
W20260709 14:55:06.994752  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 140278us
W20260709 14:55:07.001307  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 63177us
W20260709 14:55:06.995944  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:06.994186 10671 consensus_meta.cc:287] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.602s	user 0.017s	sys 0.004s
I20260709 14:55:07.009387  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.019528  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:07.020301 10673 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:07.020501  2680 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:07.021145 10673 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:07.022223 10702 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:07.023029  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80032.2us
W20260709 14:55:07.024375  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.024906  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:07.034405  2604 pending_rounds.cc:85] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Aborting all ops after (but not including) 10
I20260709 14:55:07.035238  2604 pending_rounds.cc:107] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 14:55:07.035698  2604 raft_consensus.cc:2889] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:07.036765  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.037700  2605 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:07.040004 10726 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:07.040663  2610 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 14:55:07.042850  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:07.026659  2724 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.044354  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:06.999616 10717 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:06.969383 10612 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.063474  2605 raft_consensus.cc:1240] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 20. Current term is 21. Ops: []
W20260709 14:55:07.065074  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53149.8us
I20260709 14:55:07.067996  2680 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:07.072799  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:07.075577 10671 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:55:07.077391 10671 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:07.078802 10150 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:07.080309 10606 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.011s
I20260709 14:55:07.081661 10517 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:07.081856 10695 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.082463 10612 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.083069 10713 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:07.103545 10713 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.085736  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:07.105656  2681 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:07.106488 10753 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.108312  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.109598 10713 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:07.087396  2685 raft_consensus.cc:1240] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:07.110975  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.111964  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.112526  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.113045 10154 consensus_queue.cc:1048] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:07.113317  2752 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:07.113423  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.114624  2685 raft_consensus.cc:2393] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 26: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:07.088845  2683 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 14:55:07.132758  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 61717.4us
I20260709 14:55:07.090323  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.133317 10629 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:07.087419  2756 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 14:55:07.091712  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.135274  2680 raft_consensus.cc:2376] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22. Re-sending same reply.
I20260709 14:55:07.092618  2682 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:07.093905 10622 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.041553 10367 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:07.095541 10574 consensus_meta.cc:287] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.005s
I20260709 14:55:07.068615  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b188cd7e4b5e4d9fb22a3645bdff7da1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.093940  2686 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 22. Current term is 23. Ops: []
I20260709 14:55:07.100848 10673 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:07.101557  2577 leader_election.cc:400] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 18 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 18. Current term is 19.
I20260709 14:55:07.088429 10646 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:07.111966  2758 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:07.146895  2428 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 12 to 13, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:07.148797 10673 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:07.149401  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 131045us
I20260709 14:55:07.149605 10673 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:07.115949 10718 consensus_meta.cc:287] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.536s	user 0.017s	sys 0.002s
I20260709 14:55:07.116770  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.120414 10726 consensus_queue.cc:1059] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:07.122018  2652 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:07.122560  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.123742  2677 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 14:55:07.124516 10619 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:07.128701 10717 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:07.130796  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 104213us
I20260709 14:55:07.131649 10609 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:07.176463  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.115357  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.164026  2757 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 23: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:07.169755  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.163589  2677 pending_rounds.cc:85] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 12
W20260709 14:55:07.178503  3411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:07.178587  2677 pending_rounds.cc:107] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 14:55:07.179200  2677 raft_consensus.cc:2889] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:55:07.176331  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:07.089948  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.179322  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.179736  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:07.185690  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.186512  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.182606  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:07.187747  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.182243  2754 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:07.188958  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.189507  2756 raft_consensus.cc:2393] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:07.190021  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:07.166440  2426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 168) took 53966 ms (client timeout 15000 ms). Trace:
I20260709 14:55:07.192260  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:07.185824  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.197966 10771 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:07.198468  2752 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:07.200381  2755 raft_consensus.cc:2410] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:55:07.200214  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.202425  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.202024  2428 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:07.204708  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:07.204254  2428 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 11 to 14, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:07.205680  2678 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:07.208752  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:07.210084 10185 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 14:55:07.210345 10517 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:07.211328 10517 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:07.211835 10517 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:07.213824 10773 raft_consensus.cc:3055] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 14:55:07.214375 10773 raft_consensus.cc:740] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:07.215027  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.189616 10646 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:07.216848 10646 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.211585  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.219887  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.221222  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:07.222218  2606 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:07.186115 10713 consensus_queue.cc:1059] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 14:55:07.223831 10753 raft_consensus.cc:3055] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 14:55:07.224408 10753 raft_consensus.cc:740] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:07.225845 10753 consensus_queue.cc:260] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.227793 10753 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:07.195536  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:07.231312  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 223561us
I20260709 14:55:07.232494  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.230105  2753 raft_consensus.cc:2410] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 11.
W20260709 14:55:07.239734  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 347354us
I20260709 14:55:07.198640 10771 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.233408  2683 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
W20260709 14:55:07.245380  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 51673.8us
I20260709 14:55:07.238601  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:07.248776  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:07.249836  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:07.252568  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.253482  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:07.252764  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:07.255967  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.257992  2681 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 28.
I20260709 14:55:07.260162  2651 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:07.260941  2680 raft_consensus.cc:2393] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 20: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:07.262127  2757 raft_consensus.cc:2410] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 14:55:07.256714  2754 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:07.247362  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:07.244881 10609 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:07.245380 10771 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:55:07.229665 10612 consensus_queue.cc:1175] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" responder_term: 21 status { last_received { term: 21 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 21 index: 15 } }
I20260709 14:55:07.245924  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.266580  2577 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.255486 10185 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:07.278981  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.280239  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:55:07.281793 10466 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 14:55:07.282213  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 108355us
I20260709 14:55:07.282927  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.283320 10773 consensus_queue.cc:260] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:07.284816 10773 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:07.279587 10367 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:07.286168 10367 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:07.288511  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.288879 10367 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.289284 10646 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:07.263761 10726 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:07.290349 10726 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.277051 10185 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.255473 10150 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 14:55:07.291975 10185 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:07.289768 10701 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:07.196846  2426 rpcz_store.cc:269] 0709 14:54:13.199952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:13.207148 (+  7196us) service_pool.cc:224] Handling call
0709 14:55:07.165964 (+53958816us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:54:58.945589 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:54:58.946539 (+   950us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:54:58.946600 (+    61us) write_op.cc:435] Acquired schema lock
0709 14:54:58.946632 (+    32us) tablet.cc:662] Decoding operations
0709 14:54:59.028504 (+ 81872us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:54:59.028712 (+   208us) write_op.cc:641] Partition lock acquired for write op
0709 14:54:59.028759 (+    47us) tablet.cc:685] Acquiring locks for 24 operations
0709 14:54:59.299992 (+271233us) tablet.cc:701] Row locks acquired
0709 14:54:59.300052 (+    60us) write_op.cc:260] PREPARE: finished
0709 14:54:59.300287 (+   235us) write_op.cc:270] Start()
0709 14:54:59.300535 (+   248us) write_op.cc:276] Timestamp: P: 1783608899300199 usec, L: 0
0709 14:54:59.300582 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 14:54:59.314654 (+ 14072us) log.cc:844] Serialized 9465 byte log entry
0709 14:55:00.567116 (+1252462us) op_driver.cc:464] REPLICATION: finished
0709 14:55:01.237702 (+670586us) write_op.cc:301] APPLY: starting
0709 14:55:01.261277 (+ 23575us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:01.979063 (+717786us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:01.979500 (+   437us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:03.273177 (+1293677us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:03.273661 (+   484us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 14:55:03.273728 (+    67us) write_op.cc:312] APPLY: finished
0709 14:55:03.274353 (+   625us) log.cc:844] Serialized 211 byte log entry
0709 14:55:03.379013 (+104660us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:03.405082 (+ 26069us) write_op.cc:454] Released schema lock
0709 14:55:03.405278 (+   196us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:03.547370 (+142092us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:03.557254 (+  9884us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":848094,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":491652,"apply.run_cpu_time_us":88461,"apply.run_wall_time_us":2270832,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104399,"num_ops":24,"prepare.queue_time_us":940246,"prepare.run_cpu_time_us":29379,"prepare.run_wall_time_us":574587,"raft.queue_time_us":622483,"raft.run_cpu_time_us":64738,"raft.run_wall_time_us":592994,"replication_time_us":1216388,"thread_start_us":2546490,"threads_started":5,"wal-append.queue_time_us":267900,"wal-append.run_cpu_time_us":105303,"wal-append.run_wall_time_us":4143938}]]}
I20260709 14:55:07.291126 10726 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:07.304083  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 43998us
I20260709 14:55:07.305532  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:07.309065  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:07.268075 10673 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:07.268841 10695 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:07.291047 10713 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 14:55:07.320288  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 138804us
I20260709 14:55:07.319089  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.317126  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:07.298617  2426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 169) took 22943 ms (client timeout 15000 ms). Trace:
W20260709 14:55:07.312322  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.314486 10157 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:07.326205 10157 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.326687 10157 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:07.327713 10780 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 14:55:07.328912  2423 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.341s	user 0.021s	sys 0.049s
W20260709 14:55:07.322962  2426 rpcz_store.cc:269] 0709 14:54:44.355345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:07.298229 (+22942884us) service_pool.cc:211] Skipping call since client already timed out
0709 14:55:07.298491 (+   262us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 14:55:07.324756 10646 raft_consensus.cc:3060] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:07.325655  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:07.329597  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.338058  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:07.316605 10701 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:07.311278 10619 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:07.339946  3412 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:07.316504 10695 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.322247 10713 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:07.354748  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:07.355180  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 113824us
W20260709 14:55:07.357070  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48283.9us
I20260709 14:55:07.363114 10702 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:07.339596 10367 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:55:07.370985  2724 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.371512 10150 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:07.371750  2724 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:07.372058 10150 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.372465 10150 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:07.373611  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.374272  2724 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:07.390340  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50532.7us
I20260709 14:55:07.349232 10701 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:07.401520  2678 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:07.353049 10695 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:07.419708  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 57467.8us
I20260709 14:55:07.420416 10622 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:07.427911 10622 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.431568 10622 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:07.377194  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:07.206674  2428 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 11 to 15, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:07.375171 10574 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 14:55:07.449081  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 119062us
W20260709 14:55:07.433435  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 80357.9us
I20260709 14:55:07.437117  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.375981 10713 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:07.349400 10717 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 14:55:07.388760 10517 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.374747 10482 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:07.367350 10367 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:07.442672 10607 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:07.457273  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 54083.4us
I20260709 14:55:07.434531 10701 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:07.375044 10619 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:07.487704  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.483417  2651 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:07.480521  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:07.494671  3409 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.488811  2612 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:07.473758 10319 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:07.508852  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 135304us
W20260709 14:55:07.510102  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:07.511346  2650 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:07.522027  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:07.474473 10188 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 14:55:07.537678  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:07.516544 10673 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:07.536041 10662 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:55:07.599190  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 131525us
I20260709 14:55:07.556537 10319 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:07.616739  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:07.632437 10619 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:07.633805 10764 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 14:55:07.633554 10185 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.560611 10673 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.636677 10673 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.637070 10185 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:07.639927  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 153984us
I20260709 14:55:07.644155 10188 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:07.647614 10783 log.cc:927] Time spent T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 14:55:07.648792   431 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:07.649513  2428 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 9 to 10, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 10 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:07.650961 10775 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
I20260709 14:55:07.656823 10574 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:07.599301  2423 maintenance_manager.cc:643] P 3f6dfe8cfa964684b547b44a4b308ca2: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.997s	user 0.057s	sys 0.056s Metrics: {"bytes_written":3936,"delete_count":0,"lbm_write_time_us":2518,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":9395584,"thread_start_us":165049,"threads_started":1,"update_count":142,"wal-append.queue_time_us":177136}
I20260709 14:55:07.661610 10574 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.663601 10367 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:07.667585 10517 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:07.669452 10785 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 14:55:07.669185  2428 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 14 to 15, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:07.671370 10777 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 14:55:07.671325 10771 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.675313 10798 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:07.676813  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.680861 10157 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:07.687753 10791 log.cc:927] Time spent T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 14:55:07.688992 10789 log.cc:927] Time spent T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 14:55:07.689568 10662 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 14:55:07.690032 10726 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.691181 10185 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:07.694128 10150 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:07.546185 10605 consensus_queue.cc:1059] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 14:55:07.709709  2424 maintenance_manager.cc:419] P 3f6dfe8cfa964684b547b44a4b308ca2: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.171101
W20260709 14:55:07.736447  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 260386us
W20260709 14:55:07.746507  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 272337us
I20260709 14:55:07.761739  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:07.771938  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 193592us
I20260709 14:55:07.708849 10671 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 14:55:07.782516  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:07.789475  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 112766us
I20260709 14:55:07.790719  3409 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:07.787814 10609 consensus_meta.cc:287] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.003s
I20260709 14:55:07.795163  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:07.796810  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 196360us
W20260709 14:55:07.793470  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 328734us
I20260709 14:55:07.759829 10701 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:07.783453 10319 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:07.763995 10185 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:07.794672 10188 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:07.754686 10798 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:07.794575 10605 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:07.813128  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.803977 10609 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:07.827527  2758 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:55:07.798233 10802 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:07.829037 10802 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:07.826406  3409 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:07.831173 10188 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:07.833717  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:07.834560  2612 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 18.
W20260709 14:55:07.841425  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 303875us
I20260709 14:55:07.815618 10574 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:07.866487  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.866475  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:07.847344 10367 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 14:55:07.878155  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 80114.5us
I20260709 14:55:07.887157  2752 raft_consensus.cc:2410] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
W20260709 14:55:07.897696  2652 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:07.882323 10367 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:07.911548 10726 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:07.911999  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51359.9us
I20260709 14:55:07.990821  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:07.891747 10771 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:07.908933 10820 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:07.995076  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 95466.6us
I20260709 14:55:07.999480  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:07.994153 10574 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:07.988412 10765 log.cc:927] Time spent T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.103s	user 0.000s	sys 0.005s
I20260709 14:55:08.016163 10574 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.023031 10821 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:08.023691 10821 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.024780 10574 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.008754 10717 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 14:55:08.004674 10811 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.088s	user 0.000s	sys 0.005s
W20260709 14:55:08.009760 10773 consensus_meta.cc:287] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.724s	user 0.014s	sys 0.007s
I20260709 14:55:08.003620 10489 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.026484 10126 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.016500  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.039215  2751 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:08.022298  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:07.991614 10367 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:08.053345  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.059121  2754 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 25
I20260709 14:55:08.058400  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.060118  2752 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:08.033691 10489 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.061599  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:08.034807 10713 consensus_meta.cc:287] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.556s	user 0.003s	sys 0.012s
W20260709 14:55:08.037735 10753 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.793s	user 0.007s	sys 0.012s
I20260709 14:55:08.072747  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:08.067413  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.086133  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.028954 10701 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.033128 10150 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.029731 10717 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:08.086635  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.102537  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.089172  2608 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 25. Current term is 26. Ops: []
I20260709 14:55:08.102203  2683 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 14:55:08.029754 10798 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.092203  3412 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:55:08.111299  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 267155us
I20260709 14:55:08.087090  2760 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:55:08.101066  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.107509  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.026177 10821 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.116343  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.118067  2760 raft_consensus.cc:2410] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 14:55:08.064723 10726 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.125717  2611 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 14:55:08.066066 10718 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.100848  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:08.129232  2604 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:08.036072 10803 raft_consensus.cc:3055] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 14:55:08.145613  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53520.6us
I20260709 14:55:08.150427  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:08.160025  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 92661.7us
I20260709 14:55:08.067850 10126 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.120054 10574 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.018499 10820 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.129238 10798 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:08.090364 10713 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:08.162165  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:08.169207  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.148183 10803 raft_consensus.cc:740] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:08.176141  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:08.178128  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.158595 10753 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 14:55:08.202559  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.163133 10671 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 14:55:08.214730  2609 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:08.140893 10726 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.217025 10828 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:08.221763  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.232677  2760 raft_consensus.cc:3060] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:08.139806 10718 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.171823 10815 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:08.167944 10188 consensus_queue.cc:1059] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 14:55:08.249444  2684 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:55:08.250278  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:08.255796 10779 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 14:55:08.256357 10824 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 14:55:08.259500 10820 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:08.261533 10819 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 14:55:08.281056 10726 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:08.282868 10646 consensus_meta.cc:287] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.945s	user 0.008s	sys 0.012s
I20260709 14:55:08.283881 10646 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:08.284850 10825 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 14:55:08.286054 10489 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:08.287595 10695 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.834s	user 0.016s	sys 0.006s
I20260709 14:55:08.238337 10828 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.288355 10695 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.294034  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.295467  2608 raft_consensus.cc:2393] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:08.303345 10188 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:08.304313  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 49290us
I20260709 14:55:08.221761 10671 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.307617  2604 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:08.309122  2754 raft_consensus.cc:2468] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:08.319248  2752 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
W20260709 14:55:08.306221  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 151700us
I20260709 14:55:08.333647  3409 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:08.338757  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 116051us
I20260709 14:55:08.181898 10798 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:08.340546  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 238715us
I20260709 14:55:08.183338 10574 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.356528  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.341948 10671 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:08.331292 10154 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:08.369979  2604 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:55:08.350217 10646 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:08.341790 10188 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.360455 10828 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.394816 10810 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.395824 10810 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:08.393244  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.408390  2724 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.414141 10367 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.417153 10803 consensus_queue.cc:260] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.419309  2751 raft_consensus.cc:2468] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:08.386216 10713 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.428086  2608 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:08.345721 10717 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.418213  2758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2154) took 1174 ms. Trace:
I20260709 14:55:08.343314 10815 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.436008  2609 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:08.434620  2758 rpcz_store.cc:276] 0709 14:55:07.244003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:07.270714 (+ 26711us) service_pool.cc:224] Handling call
0709 14:55:08.417874 (+1147160us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400205654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:08.418077 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1160631168}
I20260709 14:55:08.441928  2753 raft_consensus.cc:2368] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24.
I20260709 14:55:08.431442 10830 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.448218 10830 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.449431 10188 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:08.455210  2759 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:08.455942 10367 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.456768  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.457734  2751 raft_consensus.cc:2393] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:08.459052 10815 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.388195 10154 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.462031 10154 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:08.464936  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79624.2us
W20260709 14:55:08.466325  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:08.469980  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:08.473722  2724 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.474666  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.436978 10622 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:08.484858 10367 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:55:08.485486  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.486159  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.486227  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.487066 10817 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.487843 10817 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.488770  2755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2159) took 1162 ms. Trace:
I20260709 14:55:08.490306  2686 raft_consensus.cc:1275] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:55:08.492945  2751 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:08.467410 10802 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.499382 10367 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:08.504366 10798 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.495882 10803 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:08.495654  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.507772  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.511854  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.515121  2686 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:08.496865 10828 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:08.476503 10830 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.521168 10828 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.521543  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:08.523696 10784 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 1.042s	user 0.008s	sys 0.000s
I20260709 14:55:08.527897 10718 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:08.529263  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 43845.6us
I20260709 14:55:08.489390  2755 rpcz_store.cc:276] 0709 14:55:07.326087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:07.438041 (+111954us) service_pool.cc:224] Handling call
0709 14:55:07.461819 (+ 23778us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 14:55:07.492423 (+ 30604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:07.492490 (+    67us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 14:55:07.537016 (+ 44526us) log.cc:844] Serialized 78 byte log entry
0709 14:55:07.774446 (+237430us) raft_consensus.cc:1645] Marking committed up to 16
0709 14:55:07.795355 (+ 20909us) log.cc:844] Serialized 14 byte log entry
0709 14:55:07.880269 (+ 84914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:07.880395 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:08.485270 (+604875us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b1800426e20. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:55:08.488209 (+  2939us) raft_consensus.cc:1685] finished
0709 14:55:08.488372 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:08.488640 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400448,"prepare.queue_time_us":19543,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":94648,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":30497,"spinlock_wait_cycles":4342016,"thread_start_us":246029,"threads_started":2,"wal-append.queue_time_us":267353}
W20260709 14:55:08.533864  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.490732 10817 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:08.535468  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.536187  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:08.541955  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 49337.9us
W20260709 14:55:08.544746  2650 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 118 similar messages]
I20260709 14:55:08.530663 10126 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:08.515303 10802 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.547101  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.549474 10790 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.571272  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.571345 10790 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.572203  2612 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:08.550802 10367 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:08.574606  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.552171  2759 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:08.575465  2682 raft_consensus.cc:2393] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:08.577157  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.577701  2683 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:55:08.577827  2611 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:08.576507  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.579015  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.552990 10801 raft_consensus.cc:3055] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:55:08.574378 10367 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.580029 10801 raft_consensus.cc:740] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:08.581102 10801 consensus_queue.cc:260] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.581880 10367 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.582468 10801 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:08.568775 10609 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:08.588727 10609 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.569192  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.592307  2755 raft_consensus.cc:2393] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:08.570602 10786 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:08.593577  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.553746  2577 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:08.550033  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 376310us
W20260709 14:55:08.595187  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.561911  3411 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:08.595736  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.583277  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.561923 10646 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 14:55:08.562135 10753 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:08.566254  2606 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 14:55:08.602036  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.602932  2684 raft_consensus.cc:2368] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21.
I20260709 14:55:08.603977  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:08.604882  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.602437  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.596997  3411 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:08.587636  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:08.583344  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.595235  2577 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:08.585420  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.609001  2760 raft_consensus.cc:2468] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:08.615170  2683 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:08.616171  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:08.620263 10517 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:08.621248 10713 raft_consensus.cc:2804] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:08.621958 10713 raft_consensus.cc:697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:08.622251  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.623142 10713 consensus_queue.cc:237] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.626438 10367 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:08.627094  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.620622 10629 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:08.633109  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:08.635171  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 99750.5us
W20260709 14:55:08.636111  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.636277  2651 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.608569  2682 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:08.618515  2577 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:08.627832  2679 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:08.638613  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.631441  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.628958  2723 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:08.639753  2612 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:08.643754  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:08.644246 10790 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:08.649422   431 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:08.651875  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:08.652729  2612 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:08.656569  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.613173 10126 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:08.561190 10828 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.663985  2683 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:08.613757 10787 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:55:08.627450 10662 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:08.695046 10662 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.695544 10662 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:08.697934 10695 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.699539 10185 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.703930 10609 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.659852 10820 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 14:55:08.710494  2686 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
I20260709 14:55:08.712560  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.655068  2577 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
W20260709 14:55:08.714103  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 59178us
I20260709 14:55:08.714377  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.658622 10126 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.646347 10833 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:08.668830 10810 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:08.652623  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:08.716179  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63594.8us
I20260709 14:55:08.671964  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.717820  2677 raft_consensus.cc:2368] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24.
I20260709 14:55:08.719640  2577 leader_election.cc:400] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24.
I20260709 14:55:08.720057  2577 leader_election.cc:403] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 14:55:08.722570  2680 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:08.723286  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.681298  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.728703  2752 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:08.632615 10629 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.684168 10574 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.652218 10771 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:08.660108 10489 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:08.653654 10517 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.693253 10671 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.694182 10367 consensus_queue.cc:1048] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:55:08.593708 10786 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.746038  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.711678  2606 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 14:55:08.719172 10126 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:08.723829  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.677845  2612 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:08.683338  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.749997  2678 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 28.
I20260709 14:55:08.749797  2683 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:08.665686 10790 raft_consensus.cc:2804] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:08.729996  2686 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:08.729959  3411 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:08.753252  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:08.744632  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 78181.8us
I20260709 14:55:08.721232 10185 raft_consensus.cc:2764] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 14:55:08.743134  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.755311  2680 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:08.740796  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:08.722154 10787 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:08.754367  2677 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:08.756455 10574 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:08.747223 10517 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.768466  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.732460 10629 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:08.763948  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.739022 10646 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:08.766151  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 120728us
I20260709 14:55:08.742517 10823 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:08.741950 10802 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.768996  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.729871 10838 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:08.769304  2752 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:08.755242  2651 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:08.778730  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:08.725759 10828 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 14:55:08.751636 10701 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24.
I20260709 14:55:08.762090  2679 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:08.776454 10820 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:08.768356 10574 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.765478 10790 raft_consensus.cc:697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:08.789920 10848 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:08.790798 10790 consensus_queue.cc:237] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.766060 10786 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.789840 10574 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:08.795015 10802 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:08.795688 10802 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.790705 10848 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.796288 10802 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:08.796952 10718 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.797581 10718 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.800166 10718 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.800779 10848 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.802352  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.772207 10489 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.809127  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:08.810520  2752 raft_consensus.cc:2410] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:55:08.817405 10517 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:08.808451 10838 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:08.785888  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.771675 10821 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:08.821027 10838 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.821480 10821 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:08.821943 10821 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:08.823490 10671 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:08.828284  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.828779  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.829259  2608 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:55:08.829663  2606 raft_consensus.cc:2368] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:08.775728 10646 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.830345 10517 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:08.831027 10517 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.831523  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.832160 10646 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:08.834810  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.835774  2679 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:55:08.836074 10126 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:08.836935  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:08.837633  2678 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:08.838559  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.839639 10367 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:08.840103 10126 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:08.840737  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60565.1us
W20260709 14:55:08.841652  2724 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.842190  2724 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.841820 10820 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:08.834952 10517 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:08.844852 10662 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:08.784039 10771 raft_consensus.cc:2804] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:08.861340  3409 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:08.899067  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.899857  2606 raft_consensus.cc:1240] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 25. Current term is 26. Ops: []
I20260709 14:55:08.901533  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.909938  2753 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Advancing to term 28
W20260709 14:55:08.911888  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 96577.5us
W20260709 14:55:08.924445  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48586us
I20260709 14:55:08.870891  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:08.926596  2752 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 17 FOLLOWER]: Advancing to term 19
I20260709 14:55:08.926919  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:08.927902  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 66755.7us
I20260709 14:55:08.928021  2611 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 18 FOLLOWER]: Advancing to term 19
I20260709 14:55:08.929774  2575 leader_election.cc:400] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21.
I20260709 14:55:08.930258  2575 leader_election.cc:403] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 14:55:08.930032  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.924989  2652 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:08.935896  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:08.936659  2679 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 14:55:08.936551  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 94942.6us
I20260709 14:55:08.934381  2726 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:08.945815  2575 leader_election.cc:400] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24.
W20260709 14:55:08.949429  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.950225  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:08.950505  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.950994  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.938577 10771 raft_consensus.cc:697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:08.953481 10771 consensus_queue.cc:237] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:08.814219 10828 consensus_queue.cc:1243] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer 56620e612f0549e6abf320aa80398e74 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:55:08.959849  2575 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:08.965221  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:08.845422 10820 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:08.961889  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:08.971407  2604 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:55:08.971453  2726 leader_election.cc:400] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:08.972147  2726 leader_election.cc:403] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
W20260709 14:55:08.969288  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:08.931478 10367 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:08.975666  2724 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:08.976689  2679 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 28.
I20260709 14:55:08.939218 10646 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:08.853489 10489 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:09.044214  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 271692us
I20260709 14:55:09.049672  2723 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:08.962709 10786 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for earlier term 20. Current term is 21.
I20260709 14:55:08.966255 10612 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 14:55:08.969175 10673 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:08.932289 10802 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:09.079177  2752 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:55:08.974485 10848 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:08.938732 10622 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 14:55:08.978628 10828 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:09.042999 10820 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:55:09.108949  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 184047us
I20260709 14:55:09.110625  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:09.073693 10150 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.081013 10157 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:09.064999 10798 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:55:09.116631  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 71376.2us
I20260709 14:55:08.846499 10787 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.078512 10810 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.074395 10612 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:09.067390 10717 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.110405 10367 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 26
W20260709 14:55:09.134331  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 84756.4us
I20260709 14:55:09.079671 10574 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:09.179417 10810 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:09.109174 10673 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:09.128588 10150 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.142030 10612 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:09.146919 10805 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:09.133194 10828 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.148581 10816 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.136729 10619 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 14:55:09.205782 10815 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 14:55:09.206394 10815 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:09.206266 10816 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.150998 10808 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:09.151513 10807 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:09.162302  3409 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:55:09.210165  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 47960us
I20260709 14:55:09.162982 10823 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:09.167006 10717 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.166812  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:09.172066 10609 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:09.123999 10820 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:09.136487 10798 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:09.209136  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 235249us
I20260709 14:55:09.212703  3409 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:09.215598  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 48917.5us
I20260709 14:55:09.217391 10671 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:09.141691 10853 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:09.221817  2651 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:09.129344 10126 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 14:55:09.207592 10815 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.252194  3411 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:09.202744 10805 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:09.253798  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:09.203732 10517 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.253178  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:09.262787  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.264210  2755 raft_consensus.cc:2410] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 14:55:09.216437 10807 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:09.215090 10489 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:09.268395  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:09.218510 10823 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:09.271661  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 67165.3us
I20260709 14:55:09.247223 10619 raft_consensus.cc:3060] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.251354 10671 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:09.251436 10856 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.202116 10828 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:09.278792  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:09.201364 10574 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:09.277086  2760 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 24
I20260709 14:55:09.265951 10805 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:09.261736 10646 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:09.211529 10810 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:09.225899 10848 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 14:55:09.268493 10807 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:09.260114 10662 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:09.284292 10671 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.247543 10821 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.262809 10816 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:09.297554  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:09.289824 10814 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:09.242251 10808 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:09.291514  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:09.285283 10815 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:09.274070 10517 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.343359  2760 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
W20260709 14:55:09.359160 10851 log.cc:927] Time spent T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 14:55:09.359865 10802 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:09.323958 10673 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.334087 10574 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:09.363828 10150 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:09.288174 10801 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 14:55:09.332783 10810 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 14:55:09.341369 10816 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:55:09.367781 10629 consensus_meta.cc:287] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.003s
I20260709 14:55:09.378670 10574 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:09.390069 10574 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:55:09.390328  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:09.391162  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 63482.9us
W20260709 14:55:09.391981  2650 leader_election.cc:336] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.392442  2650 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:09.393750 10838 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:09.379050 10786 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:09.395542 10786 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:09.395946 10852 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
W20260709 14:55:09.397150  2650 leader_election.cc:336] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.397732  2650 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:09.403028  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 268728us
I20260709 14:55:09.408286  2753 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 28.
W20260709 14:55:09.429301  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 460038us
I20260709 14:55:09.331848 10856 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:09.375190  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 230167us
I20260709 14:55:09.405323 10810 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:09.417441 10717 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:09.430279  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1581) took 1358 ms. Trace:
I20260709 14:55:09.420311 10816 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:09.418546 10786 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:09.404318 10629 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:09.417865  2611 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 19.
I20260709 14:55:09.437443  2681 rpcz_store.cc:276] 0709 14:55:08.068626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:08.068809 (+   183us) service_pool.cc:224] Handling call
0709 14:55:08.103094 (+ 34285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:08.682012 (+578918us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b18004cc540. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e1ee4bc 00007fa16b53c48b 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa16e63e060 00007fa16e63ebfb 00007fa16e64964b
0709 14:55:08.682224 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:08.682256 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:08.720606 (+ 38350us) log.cc:844] Serialized 26 byte log entry
0709 14:55:08.851506 (+130900us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:08.851563 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:08.851734 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:09.358302 (+506568us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:09.426937 (+ 68635us) raft_consensus.cc:1685] finished
0709 14:55:09.426974 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:09.427196 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5914,"raft.queue_time_us":109065,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":7608,"spinlock_wait_cycles":77361280,"thread_start_us":218388,"threads_started":2,"wal-append.queue_time_us":352776}
I20260709 14:55:09.419440 10829 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:09.380477 10820 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.428082 10798 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:09.482093  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 228393us
I20260709 14:55:09.531371  2753 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23. Ops: []
I20260709 14:55:09.558580  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:09.448722  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:09.485680  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:09.480235  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:09.578663  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 325557us
W20260709 14:55:09.587311  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.587668  2755 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:09.587984  2760 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 22: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:55:09.480996  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 228891us
W20260709 14:55:09.588299  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139656us
I20260709 14:55:09.596710  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:09.598234  2611 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:09.449800 10612 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.592759  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:09.591468  3409 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:09.601780  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:09.605208  2753 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.564391 10821 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:09.590147 10861 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.614715  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.624403  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:09.625073  2611 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.623696  2726 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:09.594430 10863 raft_consensus.cc:2764] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader election decision vote started in defunct term 27: won
I20260709 14:55:09.604172 10828 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:09.633358 10861 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.664208  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:09.667219  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 54531.3us
W20260709 14:55:09.650236   431 thread.cc:527] Waited for 12000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:09.564733 10810 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:09.673048  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.673954  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:55:09.678378  2753 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:09.622196 10870 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 14:55:09.640924 10864 log.cc:927] Time spent T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 14:55:09.638077 10821 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:09.585937 10816 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:09.644446 10629 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:09.602432 10868 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 14:55:09.660825 10869 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:09.703907 10367 consensus_meta.cc:287] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.001s
I20260709 14:55:09.713182  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:09.651346 10871 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:09.658344 10861 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:09.693998 10870 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:09.703593  2611 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:09.720378  2755 raft_consensus.cc:2468] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:09.721674 10367 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:09.692281 10821 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:09.726873  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:09.687258 10803 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 14:55:09.732605  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.724241 10718 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.738950  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:09.708066 10869 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:09.748592  2679 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:09.691707 10574 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.727772 10821 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.751314  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:09.742359  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:09.729871 10868 raft_consensus.cc:3060] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.765672  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.752635  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:09.766580  2606 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.767252  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.759021  2680 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:09.764842  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.766659  2685 raft_consensus.cc:2410] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 14:55:09.759440  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:09.747757  2576 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.770277  2612 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:09.757344  2577 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:09.771744  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.772579  2679 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:09.772658  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:09.774794  2678 raft_consensus.cc:2393] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 in current term 21: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:09.776474  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:09.777660  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.779084  2679 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:09.780402  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:09.789031  2612 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:55:09.789692  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.778604  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:09.790520  2679 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 27.
I20260709 14:55:09.774309  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:09.740692 10872 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:09.792711  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.791605  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.795243  2677 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 14:55:09.800855  2751 raft_consensus.cc:1275] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:09.746250 10466 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:09.740329 10787 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:09.781085  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.749423 10619 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:09.808204  2612 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:55:09.808346  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:09.809371  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:09.798807 10798 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:09.802567 10713 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:09.765235 10612 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:09.817494  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:09.810426  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.761442 10718 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.835273  2680 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
W20260709 14:55:09.838711  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.839339  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:09.813407 10871 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 14:55:09.823776 10854 log.cc:927] Time spent T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.528s	user 0.002s	sys 0.002s
I20260709 14:55:09.846381  2606 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
W20260709 14:55:09.852576 10671 consensus_meta.cc:287] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.003s
I20260709 14:55:09.855218 10713 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:09.809481 10466 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:09.816653 10870 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:09.815162 10787 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:09.887717  2576 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:09.866135 10871 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:09.859706 10872 raft_consensus.cc:3060] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:09.893463  2679 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
W20260709 14:55:09.891364 10807 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.008s
I20260709 14:55:09.909327 10787 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:09.826748 10861 consensus_queue.cc:1048] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 14:55:09.889026  2576 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:09.917248 10881 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 14:55:09.903402 10150 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:09.872993 10671 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:10.034693  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 146977us
W20260709 14:55:10.036151  2576 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.036890  2576 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:09.928002 10823 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.004s
W20260709 14:55:10.039592  2576 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:10.047358  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.052030  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:10.040093  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 243049us
I20260709 14:55:09.936091 10808 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:09.923753 10807 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:10.063282 10815 consensus_meta.cc:287] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.004s
I20260709 14:55:10.132189  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:10.134306  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 317723us
I20260709 14:55:10.042476 10157 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 14:55:10.134622  2680 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:10.043164 10787 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:10.061321  3412 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:10.159654  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:10.135409  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 88115.9us
I20260709 14:55:10.071034 10790 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:10.156651  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:10.203011  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56455.9us
I20260709 14:55:10.203184  2753 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:55:10.119501 10574 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:10.159148  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 97927.7us
I20260709 14:55:10.055780 10150 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.053328 10823 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.199908  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:10.101788 10871 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.213549  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:10.160935 10815 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.217028  2680 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:10.218132  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:10.200517 10889 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.201182 10869 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.229128  2752 raft_consensus.cc:1240] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:10.220781  2609 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 14:55:10.204170 10810 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:10.226473  2610 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 14:55:10.231400  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.232916  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.235952  2609 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:10.236927 10466 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.229365 10889 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.243692 10671 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:10.243574 10889 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.246784  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:10.227432 10877 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.338s	user 0.002s	sys 0.000s
I20260709 14:55:10.248173  2680 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.256557  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:10.257541  2607 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.245445 10810 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:10.260424  2680 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:10.221711 10886 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 14:55:10.355994 10887 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 14:55:10.224025 10815 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:10.367652  2760 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:55:10.259598 10517 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:10.390069 10805 consensus_meta.cc:287] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 1.070s	user 0.008s	sys 0.004s
I20260709 14:55:10.391273 10810 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:10.396329  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 608877us
I20260709 14:55:10.350589 10862 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.394250 10815 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.357277 10886 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:10.390776 10805 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:10.396021 10877 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 14:55:10.451354  2423 tablet_replica.cc:1406] Time spent applying in-flights took a long time: real 0.287s	user 0.000s	sys 0.000s
I20260709 14:55:10.451736  2607 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:10.453938  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:10.454851  2760 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:10.406337  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:10.444169 10895 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 14:55:10.460450  2576 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:10.465291  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59008.2us
I20260709 14:55:10.466109  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:10.466125  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.448586 10695 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.463925  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:10.468042 10695 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.471581 10823 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.415355 10807 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.447958 10801 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.484936  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:10.467139  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.451416 10886 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:10.490207  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:10.490294  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:10.496709 10805 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:10.501129 10872 consensus_meta.cc:287] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.001s
I20260709 14:55:10.501775 10872 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:10.475142 10157 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:10.478365 10517 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 14:55:10.504438  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.449916 10862 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.510143  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.503178 10157 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.519860 10157 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:55:10.502810 10821 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.702s	user 0.012s	sys 0.000s
W20260709 14:55:10.531181  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.529234 10150 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.527423 10821 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.502871 10807 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:10.536484  2756 raft_consensus.cc:1275] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:55:10.538157 10673 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:10.539953  2755 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:10.542212 10862 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:10.547878 10874 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.688s	user 0.006s	sys 0.009s
I20260709 14:55:10.551340 10718 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:10.553508  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.554592  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.554462  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.555418  2680 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
W20260709 14:55:10.558864  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.559561  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.492837 10801 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:10.563659  2576 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.487258 10713 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 14:55:10.554409 10872 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:10.568881  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:10.558063 10821 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:10.570523  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.559638 10517 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:10.560293 10787 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:10.587488  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.588747  2611 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:10.571501 10872 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:10.565743 10673 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:10.593051 10872 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:10.572506  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.594735  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.614214  2752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1593) took 1360 ms. Trace:
W20260709 14:55:10.598093  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 384630us
W20260709 14:55:10.616724  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 44265.8us
W20260709 14:55:10.617460  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:10.618402  2678 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 14:55:10.614185  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:10.619830  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:10.620335  2678 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:10.577524 10801 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.599136 10888 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:10.599292  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.604074 10889 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.604708 10891 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:10.605154 10157 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.605719  2575 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:10.627043  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:10.604732  2651 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:10.606786 10805 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:10.611213 10821 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:10.643565  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.643965 10891 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.644184 10821 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.625737 10848 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:10.612092  3411 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:10.613202  3409 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:10.647286 10855 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:55:10.618680  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 1.1262e+06us
I20260709 14:55:10.648969 10821 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.650408 10821 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.651018 10862 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.651108 10821 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.651729 10862 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.654135 10862 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.654909 10821 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:10.624090  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.625800 10888 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:10.657124 10888 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.658075 10786 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.626999  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:10.625187  2611 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 21.
I20260709 14:55:10.642395  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.642652  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:10.660014  2610 raft_consensus.cc:2368] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23.
I20260709 14:55:10.642804  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:10.644569  2605 raft_consensus.cc:2393] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 21: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:10.614190  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:55:10.660671  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48697.8us
I20260709 14:55:10.655318  2607 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
W20260709 14:55:10.661324  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47214.2us
I20260709 14:55:10.616932  2752 rpcz_store.cc:276] 0709 14:55:09.254201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:09.254371 (+   170us) service_pool.cc:224] Handling call
0709 14:55:09.254929 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:10.229653 (+974724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:10.613921 (+384268us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54003d5754. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:10.614139 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239899520}
W20260709 14:55:10.662151  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 49064us
I20260709 14:55:10.612579  2608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 2002) took 1023 ms. Trace:
I20260709 14:55:10.660588  2760 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.647522 10848 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.663690  2608 rpcz_store.cc:276] 0709 14:55:09.589580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:09.591223 (+  1643us) service_pool.cc:224] Handling call
0709 14:55:09.591738 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:09.687894 (+ 96156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:09.687936 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:09.691508 (+  3572us) log.cc:844] Serialized 26 byte log entry
0709 14:55:09.861774 (+170266us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:09.861821 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:10.612034 (+750213us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b54003c7b54. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:10.612101 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:10.612166 (+    65us) raft_consensus.cc:1685] finished
0709 14:55:10.612183 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:10.612500 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1937,"raft.queue_time_us":14616,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":1469,"spinlock_wait_cycles":162519936,"thread_start_us":138567,"threads_started":1,"wal-append.queue_time_us":196832}
W20260709 14:55:10.660574  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:10.708097  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:10.741227  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:10.752245  2760 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
W20260709 14:55:10.665521 10868 consensus_meta.cc:287] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.007s
W20260709 14:55:10.672770   431 thread.cc:527] Waited for 13000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:10.714198  2678 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:10.762408  2651 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:10.711347  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45534.4us
W20260709 14:55:10.710950  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 47474.4us
I20260709 14:55:10.666136  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.800546  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:10.772650  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112144us
W20260709 14:55:10.725729  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 63579.7us
I20260709 14:55:10.665493  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.800990  3412 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:10.707088 10673 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:10.659212 10861 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:10.708432 10619 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:10.733644 10891 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.665668 10805 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:10.807632  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:10.817593 10805 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.658735 10786 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.821694 10786 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.825263 10890 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.827464 10574 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:10.828514 10574 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:10.829027 10574 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:10.751307  2726 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:10.733191 10838 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.727638 10815 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.809911  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:10.805415  2650 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.708874 10870 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:10.666240 10718 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 14:55:10.814618  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 148607us
I20260709 14:55:10.844527  2575 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.815547 10713 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:10.808564 10673 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:10.628737 10816 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.177s
I20260709 14:55:10.826165 10890 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.835692  2612 raft_consensus.cc:2393] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 26: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:55:10.831147  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80013.5us
I20260709 14:55:10.838323 10856 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.805956 10868 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:10.809634 10662 raft_consensus.cc:2804] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:10.845176  2752 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:55:10.809969 10466 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.841308 10870 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:10.726382 10872 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 14:55:10.812634 10861 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:10.851137  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 1998) took 2250 ms. Trace:
I20260709 14:55:10.853004 10790 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:10.860900  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.659004 10889 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.836390 10157 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.843734 10718 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:10.801795  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:10.851785 10673 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:10.853140 10814 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.834523 10821 raft_consensus.cc:2804] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:10.877173  2752 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:10.870154 10662 raft_consensus.cc:697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:55:10.886921  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 85962.3us
I20260709 14:55:10.839475 10838 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.842669 10815 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.909991 10815 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:10.916229  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:10.917704  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 111493us
I20260709 14:55:10.894582 10718 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:10.919024 10821 raft_consensus.cc:697] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:10.920702 10829 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:10.920501 10821 consensus_queue.cc:237] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.873483  2684 rpcz_store.cc:276] 0709 14:55:08.596762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:08.640568 (+ 43806us) service_pool.cc:224] Handling call
0709 14:55:08.664173 (+ 23605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:09.194911 (+530738us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 14:55:09.566495 (+371584us) log.cc:844] Serialized 14 byte log entry
0709 14:55:09.673931 (+107436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:09.673999 (+    68us) raft_consensus.cc:1645] Marking committed up to 18
0709 14:55:09.674060 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:10.846587 (+1172527us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540024e5d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:10.846691 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:10.846932 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":441650,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":10255,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":4838,"spinlock_wait_cycles":1219767040,"thread_start_us":435799,"threads_started":2,"wal-append.queue_time_us":80870,"wal-append.run_cpu_time_us":1272,"wal-append.run_wall_time_us":1041764}
I20260709 14:55:10.887811 10810 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.830106 10574 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:10.875675 10861 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.865622 10786 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 14:55:10.871346 10466 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.913275 10802 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:10.893340 10753 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 14:55:10.887669 10801 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 14:55:10.893365  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 91629.5us
W20260709 14:55:10.937968  2724 leader_election.cc:336] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.938697  2724 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:10.940148  2724 leader_election.cc:336] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.929610  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:10.932317  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68851.8us
I20260709 14:55:10.885476  2575 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:10.941649  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 56298.5us
W20260709 14:55:10.902213  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 101764us
I20260709 14:55:10.928351  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.943909  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.944803  2684 raft_consensus.cc:2368] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:10.946182  2724 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:10.947240  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:10.948104  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:10.950006  2685 raft_consensus.cc:3060] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:10.948621  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:10.929917 10786 consensus_queue.cc:1243] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer 7bc44bf9b26c41148142e09b78ccb3c3 log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 14:55:10.924983  2423 maintenance_manager.cc:643] P 3f6dfe8cfa964684b547b44a4b308ca2: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 3.171s	user 0.143s	sys 0.071s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":24382,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":279144,"dirs.run_cpu_time_us":833,"dirs.run_wall_time_us":2431,"lbm_read_time_us":51704,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":20,"lbm_write_time_us":6966,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":8,"peak_mem_usage":249058,"reinsert_count":0,"thread_start_us":188559,"threads_started":2,"update_count":142}
I20260709 14:55:10.903468 10814 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:10.936908 10859 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:10.964740 10859 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.965209 10848 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.965739 10466 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:10.967684  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:10.968650  2612 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 23: Already voted for candidate c1b0e692e4294ee3b8694bbada174ba1 in this term.
I20260709 14:55:10.938345 10802 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:10.973336 10802 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:10.974363 10367 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:10.953715 10662 consensus_queue.cc:237] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.975831  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45634 (request call id 2009) took 1212 ms. Trace:
I20260709 14:55:10.977697  2758 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 14:55:10.976325  2679 rpcz_store.cc:276] 0709 14:55:09.763762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:09.808774 (+ 45012us) service_pool.cc:224] Handling call
0709 14:55:10.975593 (+1166819us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b540022d154. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:10.975740 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169025280}
I20260709 14:55:10.983520  2752 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 14:55:10.983745  2575 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
W20260709 14:55:10.990754  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50664.9us
I20260709 14:55:10.991811  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:10.949944 10823 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:10.992403  2724 leader_election.cc:336] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:10.928696 10810 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:10.990499  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:10.952960 10790 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:55:10.973677 10883 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 1.111s	user 0.006s	sys 0.010s
I20260709 14:55:11.003866  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:11.008031  2724 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.010181  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:11.040516  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 48142.9us
W20260709 14:55:11.041411  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.041889  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.059767  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:11.052767  2754 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:11.060884  2684 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:11.067365  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:11.068312  2752 raft_consensus.cc:2368] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23.
I20260709 14:55:11.068377  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.079476  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:11.084990  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 318862us
I20260709 14:55:10.958798 10870 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.055270 10919 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:11.055826 10790 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:11.098574  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:11.100781  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 117165us
I20260709 14:55:11.076180 10816 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:11.071234 10890 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:11.119361  2684 pending_rounds.cc:85] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 11
I20260709 14:55:11.119902  2684 pending_rounds.cc:107] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 14:55:11.120282  2684 raft_consensus.cc:2889] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 14:55:11.118957  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 141074us
I20260709 14:55:11.102240 10837 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:11.108588 10862 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:11.132431  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 142057us
I20260709 14:55:11.130141  3409 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:11.137770  2685 raft_consensus.cc:2468] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:11.137693  3409 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:11.140384 10790 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:11.143332 10908 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 14:55:11.138785 10837 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.156173 10838 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.111934 10802 consensus_queue.cc:1059] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 14:55:11.155308 10917 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:11.167160 10917 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:11.168298 10917 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:11.169543 10814 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.133387 10918 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:11.171932 10918 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.171900 10911 raft_consensus.cc:3055] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:11.186228 10911 raft_consensus.cc:740] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:11.199061 10911 consensus_queue.cc:260] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.200846 10911 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:11.201061 10888 consensus_meta.cc:287] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.002s
I20260709 14:55:11.201787 10695 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:11.201617 10888 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.186600 10837 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:11.206300  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 68687.8us
I20260709 14:55:11.207809  3409 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:11.228068 10810 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.230877 10918 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:11.232501  2686 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 25: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:55:11.239943  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 87143us
W20260709 14:55:11.253684  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 212284us
I20260709 14:55:11.226231 10816 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:11.232409 10924 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:11.266156  2677 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 14:55:11.273583  2755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2183) took 1541 ms (client timeout 1720 ms). Trace:
W20260709 14:55:11.274339  2755 rpcz_store.cc:269] 0709 14:55:09.732122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:09.732301 (+   179us) service_pool.cc:224] Handling call
0709 14:55:11.273299 (+1540998us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400424fd4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:11.273496 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853479680}
W20260709 14:55:11.281213  2724 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.256525 10810 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 14:55:11.296557  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42488 (request call id 2175) took 1738 ms. Trace:
I20260709 14:55:11.299149  2754 rpcz_store.cc:276] 0709 14:55:09.557840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:09.558161 (+   321us) service_pool.cc:224] Handling call
0709 14:55:11.296196 (+1738035us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000c03d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:11.296449 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3197963520}
I20260709 14:55:11.308120 10888 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:11.312942 10886 consensus_meta.cc:287] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.802s	user 0.007s	sys 0.008s
I20260709 14:55:11.313566 10886 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:11.319370  2651 leader_election.cc:400] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23.
I20260709 14:55:11.319875  2651 leader_election.cc:403] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 14:55:11.322141 10837 consensus_queue.cc:1048] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 14:55:11.324102 10837 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23.
I20260709 14:55:11.259802 10718 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:11.298471  2753 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 14:55:11.280061 10816 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 14:55:11.310349 10861 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.358366  2677 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 14:55:11.371917  2605 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:11.373065 10861 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.374009 10861 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 14:55:11.374512  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:11.361737  3409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:11.378454 10886 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:11.379195  2677 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 14:55:11.328270 10924 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:11.380735 10924 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:11.386880  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 326529us
I20260709 14:55:11.391772 10930 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.421862  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:11.395917  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 144774us
I20260709 14:55:11.328948 10574 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 14:55:11.386880 10673 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.441109  2603 raft_consensus.cc:1240] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 27. Current term is 28. Ops: []
W20260709 14:55:11.436057  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.443598  2724 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.465962 10859 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:11.467557 10889 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.474983 10157 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:11.476049 10941 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.482661  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:11.419755 10925 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.492254  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:11.498723  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.501353  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:11.502189  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 65766.2us
I20260709 14:55:11.509968  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.511638  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.502687  2753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2190) took 1268 ms. Trace:
I20260709 14:55:11.514217  2723 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.508071  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.476207 10919 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:11.395689 10816 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 14:55:11.512708  2755 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:11.515719  2757 raft_consensus.cc:2410] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 14:55:11.514705  2753 rpcz_store.cc:276] 0709 14:55:10.234307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:10.234503 (+   196us) service_pool.cc:224] Handling call
0709 14:55:11.296559 (+1062056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:11.326932 (+ 30373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:11.502374 (+175442us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000c03d0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:11.502595 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440897280}
I20260709 14:55:11.516305  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.516633  2756 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 14:55:11.512856  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 76842.1us
I20260709 14:55:11.521239  2751 raft_consensus.cc:2410] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 14:55:11.518950  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.495571 10871 raft_consensus.cc:3055] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 14:55:11.496731 10612 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:11.522630  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:11.514408  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:11.529367  2752 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:11.442253 10929 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 14:55:11.523751  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:11.520059  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.527305  2723 leader_election.cc:400] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:11.532989  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.540822  2755 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
W20260709 14:55:11.540210  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:11.523401  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.430294 10930 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.551543  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.552538  2753 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:11.509865 10820 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:11.542542  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.518452 10862 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 14:55:11.545722  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.553280  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:11.558357  2755 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:11.558538  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:11.559878  2724 leader_election.cc:336] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.529958 10914 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:11.560539  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.565114  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.565620  2608 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:11.522991 10871 raft_consensus.cc:740] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:11.561718  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:11.566718  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.567265  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
W20260709 14:55:11.562350  2650 leader_election.cc:336] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.570116  2650 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.541373 10942 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:11.572751  2650 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.561635 10673 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.546137 10924 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:11.522866 10612 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.578745  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:11.580804  2755 raft_consensus.cc:2468] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:11.584697  2751 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:55:11.571981  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.580335  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.563216 10367 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:11.557710 10808 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:11.586498  2724 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:11.626662  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.521368 10925 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.578043 10814 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:11.518853 10941 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:11.637293 10941 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:11.638180 10930 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:11.568215 10938 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 14:55:11.630650  2724 leader_election.cc:336] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.633637  2611 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 14:55:11.563621 10820 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:11.631052  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:11.578755 10801 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 14:55:11.627916 10914 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:11.658267  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:11.643525 10934 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
I20260709 14:55:11.659576  2753 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:11.631929 10718 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:11.659708  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.664565  2760 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 26.
I20260709 14:55:11.638190 10612 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:11.662549 10945 log.cc:927] Time spent T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 14:55:11.590003 10942 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:11.717700 10821 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:11.738715  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:11.758960  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.741770 10919 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.741389 10914 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:11.759299 10889 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:11.762066  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:11.784312  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47458.4us
I20260709 14:55:11.785531 10886 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:11.788661  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.781749 10942 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:11.780723 10889 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:11.799985   431 thread.cc:527] Waited for 14000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:11.780223 10914 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:11.798489 10889 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:11.776738 10919 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.801131 10820 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:11.813500 10944 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
I20260709 14:55:11.820210  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:11.821106  2681 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 24.
I20260709 14:55:11.822683  2605 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 14:55:11.823313  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.824009  2603 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:55:11.820878  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.826006  2607 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 14:55:11.806558 10919 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:11.829247  2608 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 14:55:11.829192 10814 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:11.832459  2683 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:55:11.836211 10925 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:11.845536  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:11.825277 10673 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:11.847522  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:11.847434  2681 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 25. Current term is 26. Ops: [25.12-25.12]
I20260709 14:55:11.852373  2605 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:11.824563 10790 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:11.856460  2608 raft_consensus.cc:1240] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
W20260709 14:55:11.868048 10950 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 14:55:11.870419 10157 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:11.871260 10157 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:11.879168 10911 consensus_queue.cc:1059] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 14:55:11.883441 10921 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.740s	user 0.004s	sys 0.002s
I20260709 14:55:11.885536  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.886893  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:11.888036  2612 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:11.888453 10948 log.cc:927] Time spent T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 14:55:11.895107  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:11.896659  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 356413us
I20260709 14:55:11.827234 10933 consensus_queue.cc:1059] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 14:55:11.921099  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.923163  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.902549 10918 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:11.858979 10941 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.931372 10799 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:11.934002  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.936172  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:11.936445 10157 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:11.939306  2608 raft_consensus.cc:2393] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:11.941685 10933 raft_consensus.cc:3055] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:11.942268 10933 raft_consensus.cc:740] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:11.943313 10933 consensus_queue.cc:260] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:11.944320  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.944700 10933 raft_consensus.cc:3060] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:55:11.931459 10937 log.cc:927] Time spent T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.462s	user 0.006s	sys 0.000s
I20260709 14:55:11.951078  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:11.951802  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.944826  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.953745  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:11.977604  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:11.979321  3412 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
W20260709 14:55:11.953421  2724 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:11.980741  2604 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:11.984141  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:11.897737 10910 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:11.833300 10814 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:11.915627 10695 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:11.995012  2612 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
I20260709 14:55:11.863765 10673 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.011473  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.993898 10157 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.030589  2652 leader_election.cc:400] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 22. Current term is 23.
I20260709 14:55:11.997223 10862 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:12.032017  2652 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:11.914090 10790 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:12.032927  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 126062us
I20260709 14:55:12.014497 10695 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:11.998493 10924 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:12.000514 10810 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:12.037714  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 139898us
I20260709 14:55:12.050729  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.043002  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:12.010708 10837 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:12.052480  2723 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:11.977430 10848 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 14:55:11.931459 10941 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:11.996656 10870 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 14:55:12.061518  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:12.021092 10940 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:12.004976 10942 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 14:55:12.008853 10910 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:12.093317 10910 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:12.096688 10790 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:12.056809 10820 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.029938 10629 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 14:55:12.068656 10919 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.081990 10823 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:12.061445 10718 consensus_queue.cc:1059] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 14:55:12.067507 10924 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.064797 10837 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.075774 10940 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:12.046643 10695 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:12.098785  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.035228 10862 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:12.050756 10886 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:12.151319 10862 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:12.094745 10959 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.155869  2751 raft_consensus.cc:1240] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 19. Current term is 20. Ops: []
W20260709 14:55:12.108827  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 82005.6us
W20260709 14:55:12.111022  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 73473.7us
I20260709 14:55:12.157420  2652 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:12.123052  2752 raft_consensus.cc:1275] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:55:12.046172 10912 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 14:55:12.141824 10940 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:12.148554  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.126590 10914 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:12.128086 10823 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:12.179148 10823 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:12.180557 10941 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:12.181315 10837 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:12.116654 10870 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:12.157420  3412 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.160504  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.129287 10911 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:12.149802  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.162074  3411 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.160665 10808 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:12.173574 10367 consensus_meta.cc:287] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.001s
I20260709 14:55:12.174819 10814 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:12.190331  3412 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:12.196571 10810 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:12.221323 10942 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:12.251895 10942 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:12.253129 10942 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.253731 10924 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:12.207490 10718 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:12.263747  2681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:12.216187 10814 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:12.216703 10367 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:12.276276  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.266551  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.306146  2685 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:12.197570 10870 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.205390 10911 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.315621  2752 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:12.212004 10889 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.317395  2685 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 14:55:12.308682  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 106963us
I20260709 14:55:12.319916  2677 raft_consensus.cc:1240] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 25. Current term is 26. Ops: [25.12-25.12]
I20260709 14:55:12.329905  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:12.323992  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 120437us
I20260709 14:55:12.330797  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.256531 10157 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
W20260709 14:55:12.333631  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 143396us
I20260709 14:55:12.335572  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.336308  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.330538  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:12.340490  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.341881  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.344128  2684 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:12.273629 10862 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:12.347597  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.347597  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.350392  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.355150  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.351512  2612 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:55:12.323191 10861 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:12.326998 10673 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:12.360210  2680 raft_consensus.cc:1275] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:12.361315  2677 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 14:55:12.366698  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.362087  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.367522  2686 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
I20260709 14:55:12.353020 10888 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:12.372252  2757 raft_consensus.cc:1240] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:12.265678 10871 consensus_queue.cc:260] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.377583  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.378949  2752 raft_consensus.cc:2393] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:12.337883 10157 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.385835  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:12.382784 10367 consensus_queue.cc:1048] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 14:55:12.402391 10911 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:12.403692 10963 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
I20260709 14:55:12.405884 10889 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.406695  2607 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:55:12.411013 10871 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:12.415102  2605 raft_consensus.cc:2410] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 14:55:12.416800 10157 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.422685  2723 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:12.353981 10718 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 14:55:12.375525 10888 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:12.453202  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:12.462447  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:12.463711  2650 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:12.372643 10861 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.436201 10367 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:12.485292  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:12.487124  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.390972 10848 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 14:55:12.489076  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:12.459638 10888 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:12.493115 10612 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 14:55:12.513368  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:12.511888  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.489881 10960 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:12.523168  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.507994 10889 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:12.508157 10859 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:12.536339  2577 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:12.540088  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.508111 10814 consensus_queue.cc:1048] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 14:55:12.545327  2752 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 14:55:12.533887 10157 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:12.554117  2756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42032 (request call id 1614) took 1019 ms. Trace:
I20260709 14:55:12.554764  2756 rpcz_store.cc:276] 0709 14:55:11.534993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:11.535518 (+   525us) service_pool.cc:224] Handling call
0709 14:55:11.536115 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:11.575700 (+ 39585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:11.575741 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:11.627416 (+ 51675us) log.cc:844] Serialized 26 byte log entry
0709 14:55:11.794984 (+167568us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:55:11.795036 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:12.553526 (+758490us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54001ff4d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:12.553585 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:12.553658 (+    73us) raft_consensus.cc:1685] finished
0709 14:55:12.553676 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:12.554027 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43603,"raft.queue_time_us":13762,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":9710,"spinlock_wait_cycles":59780864,"thread_start_us":133206,"threads_started":1,"wal-append.queue_time_us":186862}
I20260709 14:55:12.557402  2754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42488 (request call id 2196) took 1053 ms. Trace:
I20260709 14:55:12.557688 10695 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.558640 10718 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:12.559556 10968 log.cc:927] Time spent T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 14:55:12.559355 10718 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:12.557940  2754 rpcz_store.cc:276] 0709 14:55:11.504431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:11.505352 (+   921us) service_pool.cc:224] Handling call
0709 14:55:11.506008 (+   656us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 14:55:11.640188 (+134180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:11.640224 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 14:55:11.646135 (+  5911us) log.cc:844] Serialized 52 byte log entry
0709 14:55:11.783974 (+137839us) raft_consensus.cc:1645] Marking committed up to 12
0709 14:55:11.784031 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:12.556891 (+772860us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540040f1d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:12.556960 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:12.557017 (+    57us) raft_consensus.cc:1685] finished
0709 14:55:12.557032 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:12.557321 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":8654,"spinlock_wait_cycles":307277696,"thread_start_us":111666,"threads_started":1,"wal-append.queue_time_us":129182}
I20260709 14:55:12.562630 10790 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.574466 10801 raft_consensus.cc:3055] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:55:12.575160 10801 raft_consensus.cc:740] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:12.576159 10801 consensus_queue.cc:260] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:12.577574 10801 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:12.549448 10157 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:12.579232 10157 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:12.580415 10695 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:12.588645 10978 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 14:55:12.537111 10960 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.592758 10861 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:12.549823 10962 raft_consensus.cc:2804] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:55:12.593575 10960 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.593793 10962 raft_consensus.cc:697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:12.595079 10962 consensus_queue.cc:237] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:12.596686 10976 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 14:55:12.597250 10977 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 14:55:12.599637 10974 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:55:12.607725 10808 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:12.608525 10808 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.611294  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.613834 10823 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.616492 10823 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:12.616662 10718 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:12.619477 10859 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:12.620167 10859 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:12.620493  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.599406  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.632179  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.633677  2759 raft_consensus.cc:1240] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:12.633823  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:12.629401  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 293186us
I20260709 14:55:12.635919  2652 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.637928  2682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da98c08aee5d4d86bce741e08142c1e3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.638463  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.640300  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.650156  2678 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 14:55:12.661211  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:12.662536  2652 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.671095  3409 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.673202  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:12.678216  2652 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.665398 10810 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:12.666682 10870 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.720969  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.722946  2606 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 28: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:12.669883 10837 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.729471  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.682785 10966 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:55:12.733520  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:12.692520 10886 consensus_meta.cc:287] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.542s	user 0.014s	sys 0.000s
I20260709 14:55:12.684931 10959 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 14:55:12.740131 10986 log.cc:927] Time spent T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 14:55:12.739092  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:12.692361 10973 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:12.671927 10816 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 14:55:12.665350 10940 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.713361 10808 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.692703 10975 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:12.764051  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 224045us
W20260709 14:55:12.752264 10978 log.cc:927] Time spent T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 14:55:12.767553  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45291.4us
I20260709 14:55:12.771749  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.777920  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.755632 10886 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:12.726912 10810 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:12.736851 10802 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 14:55:12.784896  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:12.791303  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:12.773568 10924 consensus_queue.cc:1059] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 14:55:12.859206  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:12.796705  2681 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:12.881592  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:12.800660   431 thread.cc:527] Waited for 15000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:12.799412  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:12.783790 10892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.006s
I20260709 14:55:12.782188 10673 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 14:55:12.785923 10861 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 14:55:12.905539  2651 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:12.789019 10799 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.142s
I20260709 14:55:12.867146 10888 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.927794  2681 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 22.
W20260709 14:55:12.906330  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 107050us
W20260709 14:55:12.970331  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 65292.2us
I20260709 14:55:12.972023  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:12.849167 10810 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:12.971637 11004 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:12.959851 10859 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:12.987915  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 316229us
I20260709 14:55:12.971145 10862 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:12.995301  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 126439us
W20260709 14:55:12.902673 10892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.007s	sys 0.007s
W20260709 14:55:12.907667 10993 log.cc:927] Time spent T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 14:55:12.959158 10999 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 14:55:12.950488 10940 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.001714 10801 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:12.907218 10157 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:12.816241 10929 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.188s
I20260709 14:55:13.003400 10859 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:12.907102 10917 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:13.012562  2726 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.016886  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:13.011266 10157 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.020252 10821 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:13.021317 10971 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 14:55:13.022660 11002 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 14:55:13.023229 10367 consensus_queue.cc:1059] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 14:55:13.024487 10367 raft_consensus.cc:3055] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 14:55:13.024890 10367 raft_consensus.cc:740] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:55:13.025610 10989 log.cc:927] Time spent T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.285s	user 0.002s	sys 0.000s
I20260709 14:55:13.028532 10911 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.029124 10911 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.030952  2684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45634 (request call id 2028) took 1200 ms. Trace:
W20260709 14:55:13.031888 10994 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.219s	user 0.000s	sys 0.003s
I20260709 14:55:13.014602  2726 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:13.031490  2684 rpcz_store.cc:276] 0709 14:55:11.830920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:11.831054 (+   134us) service_pool.cc:224] Handling call
0709 14:55:11.846414 (+ 15360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:12.348855 (+502441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:13.030642 (+681787us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b540012c354. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e646e01 00007fa16e64583e 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6
0709 14:55:13.030878 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143994752}
I20260709 14:55:13.026024 10367 consensus_queue.cc:260] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:13.048122  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:13.054914  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:13.055985  2756 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:13.052369  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:13.059514  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:13.047293 10816 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.052327 10862 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:13.059209 10889 consensus_meta.cc:287] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.005s
I20260709 14:55:13.081619  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:13.059405 10941 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 14:55:13.104867  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:13.107437  2612 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:13.114794  2685 raft_consensus.cc:1240] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 26. Current term is 27. Ops: []
I20260709 14:55:13.074913 10890 consensus_queue.cc:1059] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 14:55:13.123803  2652 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:13.089211 10810 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.125100 10911 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.126539  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.090759 10889 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:13.128499  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:13.129547  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:13.129982  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:13.124193  2575 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.099768 10930 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:13.125231 10979 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:13.083887 10816 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.156339 10890 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:13.139880  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:13.116722 10367 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:13.132030 10959 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:13.126718 10888 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.132252 10973 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:13.159865  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:13.199205  2760 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:13.144747 10969 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:13.182775 10997 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 14:55:13.222904  2756 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:13.150086 10673 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 14:55:13.136241 10810 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.153570 10930 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:13.238329  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 78101.4us
I20260709 14:55:13.236035  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:13.246732  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 90687.7us
I20260709 14:55:13.188403 11000 consensus_queue.cc:1059] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.089s
I20260709 14:55:13.278811  3411 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:13.285691  3411 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.290792 10930 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:13.300532 10802 raft_consensus.cc:3055] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:13.300946 10816 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.301394 10802 raft_consensus.cc:740] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:13.304307 10810 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.285060 10942 consensus_queue.cc:1059] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 14:55:13.307030 10802 consensus_queue.cc:260] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.308920 10802 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:13.310657 10890 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:13.311406 10890 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:13.312997 10933 raft_consensus.cc:3055] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:13.312480 10890 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.313453 10933 raft_consensus.cc:740] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:13.314123 10801 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:13.314419 10933 consensus_queue.cc:260] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.315502 10892 raft_consensus.cc:3055] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 14:55:13.315936 10933 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:13.316078 10892 raft_consensus.cc:740] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:13.330545  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:13.332942  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:13.371469  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:13.372341  2604 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:13.374104  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:13.375088  2760 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:13.387810  2681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48344 (request call id 1622) took 1124 ms. Trace:
I20260709 14:55:13.388370  2681 rpcz_store.cc:276] 0709 14:55:12.263122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:12.263370 (+   248us) service_pool.cc:224] Handling call
0709 14:55:13.387569 (+1124199us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400269654. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e64a2fa 00007fa17360bb17 00007fa16cb866d4 00007fa16cb86662 00007fa16cb86591 00007fa16cb8650c 00007fa16cb857c2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:13.387730 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215868672}
W20260709 14:55:13.432947  3412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:13.441150  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:13.354765 11000 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:13.464602  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:13.391180 10988 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:13.411062 10790 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:13.452811 10942 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.490252  2604 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
W20260709 14:55:13.491798  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50805.2us
I20260709 14:55:13.492883 10816 raft_consensus.cc:493] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.499460 11031 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:13.507473  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:13.504659 10975 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:13.558863 11024 log.cc:927] Time spent T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 14:55:13.550693 10871 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 14:55:13.555063 11021 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:13.555753 10967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.013s	sys 0.044s
I20260709 14:55:13.536595 10816 raft_consensus.cc:515] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:13.508865 10892 consensus_queue.cc:260] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:13.537184 11004 consensus_meta.cc:287] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.010s
I20260709 14:55:13.583240 11004 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:13.588944 10816 leader_election.cc:290] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:13.574357 11021 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.476459 10988 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:13.594022 10988 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:13.597968 11021 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:13.548177 10810 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:13.537317  3412 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.565143  2575 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.547362 11031 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:13.571489 10967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.013s	sys 0.045s
I20260709 14:55:13.526983 10942 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.478749 11000 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:13.604907  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 67674.8us
I20260709 14:55:13.607771 11000 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:13.604472 10810 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:13.610020 10942 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:13.613943 11031 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:13.481640 10790 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:13.637032 10790 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:13.651237  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 86269.2us
I20260709 14:55:13.653697  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:13.663146  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:13.664095  2680 raft_consensus.cc:2393] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
W20260709 14:55:13.667411  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 52401.6us
I20260709 14:55:13.685067  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:13.705083  2610 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:13.709303  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:13.712399  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:13.714416  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:13.700328 10367 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:13.709990 10918 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:13.677722 11034 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:13.717624 10918 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.725261  2758 raft_consensus.cc:1275] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 14:55:13.711285  2724 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 106 similar messages]
I20260709 14:55:13.712558  3412 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:13.706411 10821 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:13.768443  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
W20260709 14:55:13.771687  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59232.1us
W20260709 14:55:13.772544  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45477.4us
I20260709 14:55:13.777689  2605 raft_consensus.cc:1240] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 14:55:13.766535  2577 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:13.717144 10367 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.787065  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:13.769657 10862 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 14:55:13.766014 10933 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:13.791527 10912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.004s	sys 0.030s
I20260709 14:55:13.778542 10929 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.769743 10998 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:13.818100  2651 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:13.827116  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 123023us
W20260709 14:55:13.815323 10912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.004s	sys 0.031s
I20260709 14:55:13.794808 10862 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:13.814402 11034 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:13.780663 10821 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:13.891074   431 thread.cc:527] Waited for 16000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:13.823262 11036 consensus_queue.cc:1048] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 14:55:13.837380 11023 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 14:55:13.836149 10997 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:13.903383  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64841.5us
I20260709 14:55:13.822116 10998 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:13.907232 10997 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:13.906904  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:13.910945 11023 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:13.912397  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:13.912667 11038 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 14:55:13.912760 11023 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:13.914036 10918 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:13.915174 10910 raft_consensus.cc:3055] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:13.915658 10910 raft_consensus.cc:740] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:13.916175 11023 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:13.916663 10910 consensus_queue.cc:260] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:13.823004 10929 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:13.919476  2724 leader_election.cc:336] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:13.921208  2756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:13.932950  2724 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:13.912756 11039 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 14:55:13.923346 10910 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:13.924561 10988 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:13.931692 11034 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:13.939875 10820 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:13.974879 11043 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 14:55:13.944382 10918 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:13.994058 10929 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:13.994644 10820 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:13.996021 10718 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:13.935794 10862 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:13.996592 11034 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.018965  2679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48344 (request call id 1642) took 1010 ms. Trace:
I20260709 14:55:14.020083 11034 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:14.019553  2679 rpcz_store.cc:276] 0709 14:55:13.008584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:13.008781 (+   197us) service_pool.cc:224] Handling call
0709 14:55:13.009318 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:13.009945 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 14:55:13.021887 (+ 11942us) log.cc:844] Serialized 14 byte log entry
0709 14:55:13.644450 (+622563us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b18003dc660. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b53be07 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b558c02 00007fa16e1e020b 00007fa16e1e5d23 00007fa16e1e623a 00007fa16e650a2a
0709 14:55:13.788109 (+143659us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:13.788863 (+   754us) log.cc:844] Serialized 26 byte log entry
0709 14:55:13.789077 (+   214us) raft_consensus.cc:1645] Marking committed up to 10
0709 14:55:13.789128 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:13.789238 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:14.018516 (+229278us) raft_consensus.cc:1685] finished
0709 14:55:14.018652 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:14.018877 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21713,"prepare.queue_time_us":7561,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":23342,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":26762,"spinlock_wait_cycles":29456640,"thread_start_us":126159,"threads_started":1,"wal-append.queue_time_us":887833}
I20260709 14:55:13.998996 10988 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.023419  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.044711 10367 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.046064  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.046662  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.031332 10820 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.034061 10821 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:14.037343  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.013502 10718 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.006814 10997 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.043732 10998 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.044384 10862 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
W20260709 14:55:14.055660  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.058652  2724 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:14.059955  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.060364  2724 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:14.062561  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.073220  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.074477  2680 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:14.065600 10820 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 14:55:14.076321  2679 raft_consensus.cc:1240] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Rejecting Update request from peer 56620e612f0549e6abf320aa80398e74 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:14.075337 10924 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.079560 10924 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.080230 10991 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:14.080745 10991 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:14.081674 10991 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.082180 10924 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.082994  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.087311 10991 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:14.063230 10930 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.090447  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.093125  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.097965  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.099040  2680 raft_consensus.cc:2410] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1b0e692e4294ee3b8694bbada174ba1 for term 28 because replica has last-logged OpId of term: 27 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 11.
W20260709 14:55:14.094446  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:14.100060  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.101454 11036 consensus_queue.cc:1059] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 14:55:14.115535  2611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2284) took 1399 ms. Trace:
I20260709 14:55:14.116566 10718 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.119323  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.121606 10988 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.126577  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.129513  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.131007 11021 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:14.116142  2611 rpcz_store.cc:276] 0709 14:55:12.716064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:12.724744 (+  8680us) service_pool.cc:224] Handling call
0709 14:55:13.234481 (+509737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 14:55:13.234780 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 14:55:13.357398 (+122618us) log.cc:844] Serialized 14 byte log entry
0709 14:55:13.475386 (+117988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 14:55:13.475477 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 14:55:13.475554 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:13.475636 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:14.115237 (+639601us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400346050. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e6458ad 00007fa17360a419 00007fa16cb81bd4 00007fa16cb81b62 00007fa16cb81a91 00007fa16cb81a0c 00007fa16cb80cc2 00007fa16c98148c 00007fa16c9808c6 00007fa16c98026f
0709 14:55:14.115413 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42974,"prepare.queue_time_us":13247,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":113437,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":9109,"spinlock_wait_cycles":1171297024,"thread_start_us":227025,"threads_started":3,"wal-append.queue_time_us":91975}
I20260709 14:55:14.138486  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.143213  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.151865  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.144757 11003 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:14.158007  2612 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 14:55:14.159274  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.147940 10718 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.160212  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.155930 11034 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.160979  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.161252 11034 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.146412 10997 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:14.160611  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.146723 10910 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.167989 11034 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.169289  2682 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 14:55:14.171275  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.175011  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 30 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.175938  2760 raft_consensus.cc:1240] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 24. Current term is 25. Ops: []
I20260709 14:55:14.171357  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c0ea3d097a4b27b9367e3b84b6d5fd" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.191025  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:14.192409  2726 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.193352  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.196065  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:14.155459 11003 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.198648  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.201068 10821 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.203687  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.201220 10930 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:14.206008  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.204579  2608 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 29.
I20260709 14:55:14.206776  2606 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 14:55:14.186517 10910 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.213507 10910 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.201038 11015 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 14:55:14.197966 10820 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:14.216799  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.174089 10998 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:14.225520  2652 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:14.235823 10962 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:14.236646 10962 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.190249 10933 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.240530 10933 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.245849  2652 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:14.252177  3412 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.202723 11003 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:14.182250 10997 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.268028  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2e7eae198c34a6fafb697d50f269192" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.273785  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.258416 11020 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:14.258617 10886 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.285282  2680 raft_consensus.cc:2410] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 for term 30 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 14:55:14.285683  3412 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.243785 10930 consensus_queue.cc:1059] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 14:55:14.245127 10998 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.290515  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:14.258924 11015 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 14:55:14.301329  3410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:14.278023 10910 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:14.305022  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:14.293356 10924 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:14.299358 11035 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 14:55:14.319952  2759 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 14:55:14.308537 10925 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:14.328915  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:14.320348 10816 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.298393 10871 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 14:55:14.323271 11035 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.326160 10933 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.348964 11035 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 14:55:14.354732 11052 log.cc:927] Time spent T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 14:55:14.358180 10998 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.360759 10997 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.348109 10962 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.372498  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.373529  2685 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:14.383817 10157 raft_consensus.cc:3055] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:14.379436 10924 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 14:55:14.387660 10801 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.391307 10157 raft_consensus.cc:740] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:14.404474 10157 consensus_queue.cc:260] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.406334 10157 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:14.420609 11035 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.399356 10801 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.425088 10801 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.425029 11035 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 30 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.426151  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.422551 10892 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:14.433418  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:14.438352  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 749572us
I20260709 14:55:14.441746 11035 raft_consensus.cc:493] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:14.442924 10997 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:14.445552  2760 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:55:14.470036 10801 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:14.445245  2723 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:14.474817 10890 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.481074  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:14.489073 10801 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.489302 10890 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.489746 10966 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:14.490271 10966 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:14.490696 10966 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:14.493378  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.494654  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.492120 10890 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.497099 10801 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.480693 10997 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.443081 11035 raft_consensus.cc:515] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.495566  2751 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:14.510421  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.514144 10890 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:14.516335 10890 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.516770 10890 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:14.520498 11017 raft_consensus.cc:3055] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 14:55:14.521539 11017 raft_consensus.cc:740] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:14.520790 11003 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.511343 10924 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.525903 10924 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.528036 11017 consensus_queue.cc:260] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.547978  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.528944 10924 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.529109  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:14.551532  2755 raft_consensus.cc:2393] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:14.551546  2751 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:14.551965  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112066us
I20260709 14:55:14.520936 10997 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:14.530920  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.531538 11003 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.543188 10890 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.573643 11003 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.574463 11003 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.582448  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.585839  2679 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
W20260709 14:55:14.586880  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 801762us
W20260709 14:55:14.588013  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.588518  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:14.588920 10821 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.589457 10157 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.589524 10821 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.594991 10821 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.608498  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.609390  2679 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 28.
I20260709 14:55:14.610055  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:14.605876  3412 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.611629  2677 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Advancing to term 30
I20260709 14:55:14.614706  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:14.615617  2679 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:14.616283  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:14.616987  2611 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Advancing to term 30
I20260709 14:55:14.620800  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.620944  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:14.629194  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.637745  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.640731  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:14.631426  2576 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.642303  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:14.644380  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:14.644809  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:14.647235  2603 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:14.647876  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:14.648590  2608 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:14.637594  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.650580  2751 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:55:14.630309 10912 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:14.618605 11035 leader_election.cc:290] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.656913 10966 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.663571 10890 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:14.629415 10821 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 14:55:14.613560 11017 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:14.680588  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 30.
I20260709 14:55:14.619589 10157 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:14.693166  3411 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.694355  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:14.695215  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.697889  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.703613  2752 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:14.703693  2758 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:14.702077  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:14.706434  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.707294  2679 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:55:14.675477 10892 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:14.711936  2611 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 30.
I20260709 14:55:14.715320  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee6ef23c038421d972947971af9f0cc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:14.721800  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.722707  2684 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 23.
I20260709 14:55:14.723165  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.699667 10821 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.701988 11034 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.743247  2752 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:14.745774  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.744361 11032 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.716966 10911 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:14.750168 11032 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.755432  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.757174 11003 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.757967 11035 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:14.756367 10940 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:14.759073 10940 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.759594 10940 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:14.759985 10911 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:14.725534 10870 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:14.760939 10870 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:14.761317 10911 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:14.762173  3409 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.774569 10718 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:14.776468 10861 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:14.777228  3409 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.777423 10861 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.761376 10870 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:14.783123 10917 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 14:55:14.783879 10917 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 30 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:14.784914 10917 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:14.747099  3411 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.789932 10912 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:14.793776 11034 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.729039 10821 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:14.797668  3409 leader_election.cc:304] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.799223 10718 raft_consensus.cc:2749] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:14.790735 10912 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:14.801143 10912 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.710958 10157 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:14.805119  2607 raft_consensus.cc:2468] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 20.
I20260709 14:55:14.809813  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:14.810798  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 131739us
W20260709 14:55:14.831278  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 205957us
W20260709 14:55:14.837015  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:14.823807  2434 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 6 to 7, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:14.846725  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76527.6us
I20260709 14:55:14.850731 10157 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:14.872566 10940 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:14.899242   431 thread.cc:527] Waited for 17000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:14.899103 10940 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.904815  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:14.905694  2606 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 27
I20260709 14:55:14.908728  2726 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:14.913230  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:14.913844  2677 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:14.919800  2677 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:14.925498  3409 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:14.933102  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.938740 10861 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:14.922214 10673 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:14.943199 10673 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:14.947837  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:14.932987  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.952654  2677 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 27.
I20260709 14:55:14.955057  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:14.967343  2607 raft_consensus.cc:2393] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:14.973824  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:14.976620  2606 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 27.
I20260709 14:55:14.977031  2751 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 20.
I20260709 14:55:14.959275 10910 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:14.923007 10799 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:14.979614  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:14.944290 10966 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:14.931054 10823 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:14.982479 10816 raft_consensus.cc:2804] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:15.010258  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.004025  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:15.010259 10870 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:15.008435 10991 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:15.020905  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:14.987028 10910 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:14.985839 10799 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.030766 11068 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:15.006548 11032 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:15.026950  2428 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 161) took 102259 ms (client timeout 15000 ms). Trace:
I20260709 14:55:15.049396 10673 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.062664 10991 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:15.035602 11069 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:55:15.012567 10816 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.069660 10870 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.127174 10816 raft_consensus.cc:3060] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:15.079186  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:15.131305  2752 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:15.089406 10998 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:15.050405 10821 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.135296 10998 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.137209 10821 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:15.099939  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.138186 10998 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.104421  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.140856  2677 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 25
W20260709 14:55:15.099939  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 89586.6us
I20260709 14:55:15.118983 10823 raft_consensus.cc:2749] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:15.041836 11034 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.140451  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.119623 10799 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.106514 10967 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:15.151310  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.158955  2678 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 14:55:15.159888  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.164914  2677 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:15.166949  2752 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:15.169181  3411 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:15.187505 10718 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:15.189689 10910 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:15.191007 10892 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:15.191651 10892 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.194536  2611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.195739  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.196269  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.195747  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.148484 11069 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:15.158531 10967 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.199276 10967 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:15.199503  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.200350  2612 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 28.
W20260709 14:55:15.203357  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.203595  2678 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 29.
W20260709 14:55:15.205292  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 66220.7us
I20260709 14:55:15.116372 11068 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:15.114403  2428 rpcz_store.cc:269] 0709 14:53:32.768035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:32.768470 (+   435us) service_pool.cc:224] Handling call
0709 14:55:15.023131 (+102254661us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:08.535409 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:08.536374 (+   965us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:08.536483 (+   109us) write_op.cc:435] Acquired schema lock
0709 14:55:08.536515 (+    32us) tablet.cc:662] Decoding operations
0709 14:55:08.537793 (+  1278us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:08.537955 (+   162us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:08.538005 (+    50us) tablet.cc:685] Acquiring locks for 6 operations
0709 14:55:08.539143 (+  1138us) tablet.cc:701] Row locks acquired
0709 14:55:08.539226 (+    83us) write_op.cc:260] PREPARE: finished
0709 14:55:08.539568 (+   342us) write_op.cc:270] Start()
0709 14:55:08.539850 (+   282us) write_op.cc:276] Timestamp: P: 1783608908539446 usec, L: 0
0709 14:55:08.539910 (+    60us) op_driver.cc:348] REPLICATION: starting
0709 14:55:08.541476 (+  1566us) log.cc:844] Serialized 2477 byte log entry
0709 14:55:09.160853 (+619377us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18002a25c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b5593f3 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:55:09.183600 (+ 22747us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:09.541356 (+357756us) op_driver.cc:464] REPLICATION: finished
0709 14:55:09.888584 (+347228us) write_op.cc:301] APPLY: starting
0709 14:55:09.898791 (+ 10207us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:10.354865 (+456074us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:10.354896 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:10.413828 (+ 58932us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:10.414179 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=5,mrs_lookups=0
0709 14:55:10.414239 (+    60us) write_op.cc:312] APPLY: finished
0709 14:55:10.414634 (+   395us) log.cc:844] Serialized 64 byte log entry
0709 14:55:10.456730 (+ 42096us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:10.467350 (+ 10620us) write_op.cc:454] Released schema lock
0709 14:55:10.478985 (+ 11635us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:10.499272 (+ 20287us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:10.499318 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2592381,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":299682,"apply.run_cpu_time_us":24131,"apply.run_wall_time_us":609467,"cfile_cache_hit":28,"cfile_cache_hit_bytes":33322,"cfile_init":1,"lbm_read_time_us":75346,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"num_ops":6,"prepare.queue_time_us":83638,"prepare.run_cpu_time_us":34012,"prepare.run_wall_time_us":238367,"raft.queue_time_us":272930,"raft.run_cpu_time_us":14692,"raft.run_wall_time_us":255483,"replication_time_us":991515,"spinlock_wait_cycles":57759872,"thread_start_us":648907,"threads_started":4,"wal-append.queue_time_us":265302,"wal-append.run_cpu_time_us":53436,"wal-append.run_wall_time_us":2340417}]]}
W20260709 14:55:15.208997  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 170199us
I20260709 14:55:15.209420 11068 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.194813  2608 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 19. Current term is 20. Ops: []
I20260709 14:55:15.227520  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.228770  2751 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:15.213801  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:15.230746  2603 raft_consensus.cc:3060] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:15.234541  2723 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:15.242108  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.243181  2751 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:15.246434  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.244764  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:15.231528  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.247788  2755 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:15.264173  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.265686  2758 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:15.266985  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:15.267328  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:15.268199  2607 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:55:15.270278  2652 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:15.272442  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.274930  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:15.275452  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 14:55:15.259549  2650 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.287676  2650 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.289013  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.289801  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:15.295408  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.296311  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.301366  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.301677  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:15.302721  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.239372 10925 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:15.230922 10718 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:15.299397  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:15.301983  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.317101  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.246387 10823 raft_consensus.cc:2804] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 14:55:15.319635  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.285518 11032 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:15.329128  2685 raft_consensus.cc:2393] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 29: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:15.289274 10157 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:15.298571 10814 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:15.298317 11020 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:15.306813 10941 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:15.303727 10892 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.321204 10960 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:15.303536 10886 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 14:55:15.347554  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 45613.4us
W20260709 14:55:15.365990  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:15.306133 10918 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:15.340015 10157 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.353856 10941 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:15.357429 11073 log.cc:927] Time spent T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.296s	user 0.003s	sys 0.003s
I20260709 14:55:15.331570 11032 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.340260 10814 raft_consensus.cc:493] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.396894  2685 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 14:55:15.328154 10925 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:15.369625 10960 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.382736 10979 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:15.411196  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 109965us
I20260709 14:55:15.422047  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:15.406893  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 100951us
W20260709 14:55:15.410061  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 101637us
I20260709 14:55:15.389179 10157 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:15.404632 11032 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:15.407644 10814 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:15.480134 10991 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:15.428608 10960 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:15.333876 11069 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:15.502130  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.413587 10718 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:15.529870 10816 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:15.440728 10941 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:15.539446 10925 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.559276 10816 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:15.559775 10799 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:15.560504 10799 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.563587 10799 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.568141 11017 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:15.568814 11017 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.571882 11017 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:15.576741  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.579432  2685 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:15.581305  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:15.583190 10941 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:15.583909 10941 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:15.584424 10941 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:15.584576  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 286968us
I20260709 14:55:15.592902  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.594138  2677 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:15.600694  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:15.602074  2677 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:15.603140  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:15.607260  2576 leader_election.cc:336] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.608222  2576 leader_election.cc:304] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.617983  2606 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 25
I20260709 14:55:15.620410  2684 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:55:15.628387  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:15.629992  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:15.631031  2576 leader_election.cc:336] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.631641  2576 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.639410  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.640352  2760 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:15.648331  2760 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:55:15.627305 10892 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:15.646180 11015 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 14:55:15.639586  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:15.657555  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.666142  2755 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:15.664436  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:15.673301  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:15.566686  2603 raft_consensus.cc:2468] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 29.
I20260709 14:55:15.651680 11079 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:15.680620 11079 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.685402 11079 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:15.685709 10967 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.655838 11088 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:15.646063 10673 raft_consensus.cc:2749] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:15.701526 10933 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 14:55:15.691820 10942 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 14:55:15.692266 10871 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:15.730329  2603 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 14:55:15.700454 10967 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.675411 10823 raft_consensus.cc:697] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:15.741083  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:15.730492 10871 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:15.751413  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:15.650027  2606 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:15.750205 10157 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.758601  2612 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 26: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:15.722293 10925 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 14:55:15.733337 10941 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.753334 10871 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:15.769570 10157 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:15.775544  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:15.790452  2755 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
W20260709 14:55:15.790444  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 162191us
I20260709 14:55:15.797505  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.813922  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:15.826368  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.827420  2686 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 14:55:15.839116  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 208127us
I20260709 14:55:15.824985 10823 consensus_queue.cc:237] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.820677 10917 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 14:55:15.808835 10942 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 14:55:15.862516  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:15.874599  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.875341  2576 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:15.889739  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.892617  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:15.893412  2755 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:15.895874 10941 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:15.898530  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.898924 10960 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:15.900067   431 thread.cc:527] Waited for 18000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:15.900463  2678 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:15.878533 11032 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.903927 11032 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:15.899391 11090 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 14:55:15.908846 10960 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:15.891585  2576 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.910676 11032 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:15.911295  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:15.912241  2603 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 26: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:55:15.912475  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.913048  2576 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.913350  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:15.914218 10861 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:15.914419  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:15.914309  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:15.915714  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:15.917357 10960 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 14:55:15.919550  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.920068  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.921077 10861 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:15.921669  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.922297  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.923080 10861 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 14:55:15.923556  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:15.924042  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:15.924861 10861 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:15.929083  2685 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:15.929158  2686 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 28.
I20260709 14:55:15.940737  2751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:15.941499  2751 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:15.941977  2575 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665
I20260709 14:55:15.944862  2679 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 14:55:15.939529  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:15.938429 11088 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:15.952996 11032 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 14:55:15.918493 10814 raft_consensus.cc:515] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.004247 11032 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 28 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:16.005385 11032 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:15.954468 10870 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 14:55:15.955648 11086 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:16.013113  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:16.013521 11086 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.016324 11086 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:15.957942 11095 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 14:55:15.961894 11079 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:16.018766  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:16.019316 10814 leader_election.cc:290] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:16.019407  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:16.020277 11079 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:16.012632  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:16.021627  2606 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:16.021672  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:16.022781 10814 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 14:55:15.997336  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 46138.8us
W20260709 14:55:15.952203  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:16.028252  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 103005us
I20260709 14:55:16.028609  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:16.029388  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:16.029825  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:16.030539  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab522322e3448f18a90387481cab3ff" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
W20260709 14:55:16.031140  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:16.031502  2754 raft_consensus.cc:3060] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:16.031623  2576 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:16.029629  2678 raft_consensus.cc:2368] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 27. Current term is 28.
I20260709 14:55:16.034807  2723 leader_election.cc:400] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 27. Current term is 28.
I20260709 14:55:16.035233  2723 leader_election.cc:403] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:55:16.035347  2755 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:16.040385  2751 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:15.952781 11017 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:16.037038  2577 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:16.028378  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 79694.6us
I20260709 14:55:16.044622  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:16.045324  2680 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:16.047084  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:16.042176  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:16.047894  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:16.050083  2603 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
W20260709 14:55:16.055169  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:16.062320  2606 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:16.064409  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:16.066226 10810 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 14:55:16.067006 11086 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.067924 11086 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.069828 10941 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 28
I20260709 14:55:16.045106 11088 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:16.076857 11017 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:16.078339 11017 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:16.082345  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:16.082084  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 45170us
I20260709 14:55:16.085525  2577 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:16.083014  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:15.982793 11093 log.cc:927] Time spent T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 14:55:16.105194  2754 raft_consensus.cc:2468] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:16.114778  2651 leader_election.cc:304] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:16.117009  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:16.117833  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:16.117571 10157 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:16.120659  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:16.112207  2680 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:16.124056  2575 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:16.041354 10979 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:16.132098  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.062754 11079 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 14:55:16.135270  2608 raft_consensus.cc:2368] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:16.137126  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:16.137495  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.114898 11080 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:16.116050 10861 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:16.114962 11088 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:16.146318  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.124179 10924 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.124907 11087 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:16.123832 11084 raft_consensus.cc:2804] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:16.154418  2608 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:16.153779  2577 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:16.136067 10917 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 14:55:16.139348 11098 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:16.156123 10861 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:16.116465 10871 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:16.154969  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:16.211714 11097 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 14:55:16.214682 10861 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.151645 11086 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:16.193859 10924 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:16.136688 10979 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.169256 11088 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:16.236804 10924 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:16.209612 11084 raft_consensus.cc:697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:16.256294  2758 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
I20260709 14:55:16.256716  2608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.284394  2680 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 24.
W20260709 14:55:16.304000  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 150337us
I20260709 14:55:16.224742 11098 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:55:16.327668  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 187202us
W20260709 14:55:16.328058  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 72703.2us
I20260709 14:55:16.346624  2726 leader_election.cc:400] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091) with higher term. Message: Invalid argument: T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26.
I20260709 14:55:16.350342  2755 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:16.322404 10810 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.368954  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.276803 10997 consensus_queue.cc:1059] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 14:55:16.326154 10910 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.391206  2605 raft_consensus.cc:1240] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Rejecting Update request from peer 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 20. Current term is 21. Ops: []
W20260709 14:55:16.401007  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113205us
I20260709 14:55:16.404321  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:16.409401  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 55107us
I20260709 14:55:16.403911 10910 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:16.388734 10941 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829) with higher term. Message: Invalid argument: T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 27. Current term is 28.
I20260709 14:55:16.407760 10816 raft_consensus.cc:493] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.462219  2680 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 14:55:16.469010 11113 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:16.469720 11113 raft_consensus.cc:493] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:16.470340 11113 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:16.473338 10979 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:16.474893 11108 log.cc:927] Time spent T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 14:55:16.377275 10810 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:16.480811  2606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.488945  2606 raft_consensus.cc:2393] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 25: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:55:16.483297  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 79106.4us
W20260709 14:55:16.458225 11100 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 14:55:16.379822 11084 consensus_queue.cc:237] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:16.497535  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51859.4us
I20260709 14:55:16.379406 11088 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.503094  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:16.475996 10814 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:16.504608  2758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.453497 11098 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.520929 10810 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:16.527554 11120 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 14:55:16.528931  2684 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:16.535784  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:16.483018 10816 raft_consensus.cc:515] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:16.529583 11104 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.319s	user 0.000s	sys 0.003s
I20260709 14:55:16.530480 10871 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:16.516839 10917 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 14:55:16.565595 11019 raft_consensus.cc:493] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:16.603478 11117 log.cc:927] Time spent T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 14:55:16.604049 11019 raft_consensus.cc:515] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:16.617756 10816 leader_election.cc:290] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:16.610338 11019 leader_election.cc:290] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:16.617880 10910 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:16.637904 11113 raft_consensus.cc:515] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.630311 11069 raft_consensus.cc:3055] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 14:55:16.641614 11069 raft_consensus.cc:740] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:16.641907 11113 leader_election.cc:290] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:16.642031 11122 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.091s	user 0.001s	sys 0.001s
I20260709 14:55:16.643337 11069 consensus_queue.cc:260] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.646399 11069 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 14:55:16.652674 11123 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 14:55:16.718281  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:16.740330  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:16.741561  2677 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 26
W20260709 14:55:16.744210  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 394640us
W20260709 14:55:16.740262  2650 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:16.765223  2758 raft_consensus.cc:2368] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26.
I20260709 14:55:16.776432  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:16.806799  2677 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:16.815742  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47a7b94ee76444db9c73a90f520af7c" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.823021  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:16.824352  2759 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:16.859716  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:16.864763 11135 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.866308 11135 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:16.878513  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61107.9us
I20260709 14:55:16.884377 11103 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:16.885860  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:16.885392 11103 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:16.900842   431 thread.cc:527] Waited for 19000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:16.910404  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "750a16eb87de49b4a212064463a9d3a4" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.912386  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.915231  2723 leader_election.cc:304] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:16.918893  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 75327.2us
I20260709 14:55:16.919225  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c52e7cc9c9f460c9c052fe21cfb57d7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:16.923120  2685 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26. Ops: []
I20260709 14:55:16.897894 11103 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:16.925211  2683 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 14:55:16.919268 10823 raft_consensus.cc:2804] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:16.926865 11113 consensus_queue.cc:1059] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:16.927749 11132 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:16.929030 11133 raft_consensus.cc:3055] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 14:55:16.929616 11133 raft_consensus.cc:740] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:16.901640 11135 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:16.939342 11133 consensus_queue.cc:260] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.941648 11133 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:16.951164 11103 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 14:55:16.930672 10823 raft_consensus.cc:697] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:16.933548  2684 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
I20260709 14:55:16.959857  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:16.967512  2683 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 14:55:16.980921 11118 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:16.983230  2753 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:55:16.982820 10823 consensus_queue.cc:237] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:16.990024  2683 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 14:55:17.000538  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:17.005373 11097 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 14:55:17.018748  2759 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 26.
W20260709 14:55:17.016108  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:17.039994  3411 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:17.040840  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:17.041654  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:17.044361  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 56681.1us
I20260709 14:55:17.044530 11130 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:17.049678  2609 raft_consensus.cc:1275] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 14:55:17.052440  2610 raft_consensus.cc:1240] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Rejecting Update request from peer 4f1347e243fd46a5bb7cee5b2e23b665 for earlier term 25. Current term is 26. Ops: []
I20260709 14:55:17.088035 10890 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:17.089162 10890 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:17.097890  3411 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:17.099017  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194ae28527c14d78817238308fb284c6" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:17.100107  2612 raft_consensus.cc:3060] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:17.117357 10933 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:17.122181 10890 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:17.122901  2686 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 14:55:17.123631  2686 pending_rounds.cc:85] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting all ops after (but not including) 12
I20260709 14:55:17.124007  2686 pending_rounds.cc:107] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 14:55:17.124420  2686 raft_consensus.cc:2889] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:17.127265  2760 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 14:55:17.146272  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:17.148903  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:17.150411  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 53883.2us
I20260709 14:55:17.150606 10890 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:17.150633 10940 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:17.153867  3409 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:17.154101  2607 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 14:55:17.154937  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 87918.9us
I20260709 14:55:17.157737 10991 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 14:55:17.155972 10890 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 14:55:17.163730  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 242270us
I20260709 14:55:17.176120  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:17.186580  2612 raft_consensus.cc:2468] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:17.173594  3411 leader_election.cc:304] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:17.220232 10967 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:17.221599  2651 leader_election.cc:400] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26.
I20260709 14:55:17.222024  2651 leader_election.cc:403] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 14:55:17.163478 10890 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:17.223863 10890 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:17.225255 11098 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:17.226941 11098 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 14:55:17.238673  2758 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 14:55:17.246225 11127 raft_consensus.cc:2749] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277) with higher term. Message: Invalid argument: T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 for earlier term 25. Current term is 26.
I20260709 14:55:17.247309  2680 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 14:55:17.248374  2759 raft_consensus.cc:1275] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 14:55:17.251314  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:17.269942 10942 raft_consensus.cc:2749] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:17.280104 11097 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 14:55:17.289676  2603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:17.294406 11079 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:55:17.335012  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 113927us
I20260709 14:55:17.340746 11020 consensus_queue.cc:1048] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:17.342650  3411 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 189315us
I20260709 14:55:17.344237  3411 leader_election.cc:304] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:17.345510 10718 raft_consensus.cc:2749] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:55:17.386787  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 110031us
I20260709 14:55:17.406924  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:17.415103  3412 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:17.415637  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:17.499384  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 77328.5us
W20260709 14:55:17.520205  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:17.526741 11145 log.cc:927] Time spent T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.074s	user 0.000s	sys 0.004s
I20260709 14:55:17.765856 11015 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:17.783712 11148 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
W20260709 14:55:17.786762 11151 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 14:55:17.787457 11163 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:17.788614 11163 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:17.803020 11163 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:17.837757 11088 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:17.850085 11088 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:17.853899 11088 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:17.855263  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:17.867848 11153 log.cc:927] Time spent T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 14:55:17.873673 11144 log.cc:927] Time spent T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.583s	user 0.011s	sys 0.010s
W20260709 14:55:17.877758 11154 log.cc:927] Time spent T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 14:55:17.879670  3411 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:17.882387 10718 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:17.901649   431 thread.cc:527] Waited for 20000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:17.908396  2756 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 14:55:17.925639  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:17.931401 11166 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:17.932274 11166 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:17.947634  2607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "947b4a5caeb243ef87d3a535eddda080" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:17.948668 11166 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:17.957234  2680 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 14:55:17.969269 10870 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:18.039975  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 50341.3us
I20260709 14:55:18.097604 11079 raft_consensus.cc:493] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:18.097637  2679 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 14:55:18.104851  2725 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 14:55:18.098572 11079 raft_consensus.cc:515] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:18.115000  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 179888us
I20260709 14:55:18.093060 10940 raft_consensus.cc:493] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:18.145447  2760 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 14:55:18.146181  2760 pending_rounds.cc:85] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting all ops after (but not including) 12
I20260709 14:55:18.157038  2760 pending_rounds.cc:107] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 14:55:18.157382  2760 raft_consensus.cc:2889] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:55:18.192898  2610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:18.186653  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48904.1us
W20260709 14:55:18.229547  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 216672us
I20260709 14:55:18.114202 10823 consensus_queue.cc:1048] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
W20260709 14:55:18.277971  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45894us
I20260709 14:55:18.141577 10940 raft_consensus.cc:515] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:18.313380 11175 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 14:55:18.316825 11079 leader_election.cc:290] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:18.314060 11170 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 14:55:18.331974  2605 raft_consensus.cc:1275] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 14:55:18.335479 11158 log.cc:927] Time spent T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 14:55:18.286630 11133 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:18.423796 10940 leader_election.cc:290] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:18.427856  2604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55870 (request call id 2341) took 1258 ms. Trace:
I20260709 14:55:18.428571  2604 rpcz_store.cc:276] 0709 14:55:17.170013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:17.172590 (+  2577us) service_pool.cc:224] Handling call
0709 14:55:17.176675 (+  4085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 14:55:17.260166 (+ 83491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 14:55:17.260306 (+   140us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 14:55:17.272225 (+ 11919us) log.cc:844] Serialized 26 byte log entry
0709 14:55:17.724344 (+452119us) raft_consensus.cc:1645] Marking committed up to 11
0709 14:55:17.724413 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 14:55:17.724534 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 14:55:18.427315 (+702781us) raft_consensus.cc:1685] finished
0709 14:55:18.427347 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 14:55:18.427557 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68241,"raft.queue_time_us":75690,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":4583,"thread_start_us":451556,"threads_started":1,"wal-append.queue_time_us":717609}
I20260709 14:55:18.439172 11133 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.442312 11133 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:18.450851  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 171359us
W20260709 14:55:18.471232  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 135985us
W20260709 14:55:18.471987  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 138398us
I20260709 14:55:18.478513  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:18.500943  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 319405us
W20260709 14:55:18.513063  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 211155us
I20260709 14:55:18.543167 11188 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 14:55:18.555804  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:18.568698 11184 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:18.569679 11184 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.585465 11193 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:18.586393 11193 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.589308 11193 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:18.590366 11193 raft_consensus.cc:493] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:18.579180 11118 raft_consensus.cc:493] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:18.591234 11118 raft_consensus.cc:515] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:18.605674 11198 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:18.606521 11198 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:18.607158  2752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea5acf8ab374c8fa3009782429fd3ed" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.600382  2433 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 7 to 18, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:18.613705 11198 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:18.619380  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.623590 11184 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:18.626057 11171 log.cc:927] Time spent T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.339s	user 0.005s	sys 0.004s
I20260709 14:55:18.630398  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:18.631170  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 69349.1us
W20260709 14:55:18.634922  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.635293  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.639503 11193 raft_consensus.cc:515] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:18.646447 11118 leader_election.cc:290] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:18.648819 11118 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
I20260709 14:55:18.649542 11118 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.654539  2433 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 15 to 17, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:18.674376  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:18.671988  2680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f95289d39644bead88ff8640742871" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:18.676819  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:18.679472  3411 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:18.680851  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:18.680989  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:18.681641  2605 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 23.
I20260709 14:55:18.682407  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d27004b985a45779b0f86fcb7273dbc" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.685052  2575 leader_election.cc:304] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:18.686518 11098 raft_consensus.cc:2749] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 14:55:18.686704  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9cf6350d3b499497e3ace2d1dea988" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.661468 11193 leader_election.cc:290] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:18.687866  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6340ce5f0bc04ded8681c8573a0910c1" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.690240  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:18.660025 11118 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:18.695209  2433 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 14 to 15, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:18.700992  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:18.706975  2575 leader_election.cc:304] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:18.707701  2726 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:18.711689 11102 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:18.710537  2575 leader_election.cc:304] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
W20260709 14:55:18.703704  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:18.715114 10823 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:18.723280 10823 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:18.723732 10823 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:18.725555 11198 raft_consensus.cc:2749] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:18.719529 11196 consensus_queue.cc:1048] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 14:55:18.728205  3411 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:18.720232  2576 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 130 similar messages]
I20260709 14:55:18.731711 11135 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 14:55:18.734210  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.735002  2576 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:18.735948 11118 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:18.721220 11192 raft_consensus.cc:2749] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:18.739308 10975 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:18.739696  3411 leader_election.cc:304] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:18.727636 11175 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:18.756898  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:18.765323  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:18.760203 11103 raft_consensus.cc:2749] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:18.783083  2434 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 97) took 173050 ms (client timeout 15000 ms). Trace:
I20260709 14:55:18.787490  2433 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 13 to 16, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:18.824738  2576 leader_election.cc:336] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.831938  2576 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:18.839751  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.840339  2576 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:18.789431  2434 rpcz_store.cc:269] 0709 14:52:25.711703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:25.712248 (+   545us) service_pool.cc:224] Handling call
0709 14:55:18.744228 (+173031980us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:15.089661 (+     0us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16b53c0ac 00007fa174396ab9 00007fa16b55869b 00007fa16b55693f 00007fa16b556455 00007fa17207b16c
0709 14:55:15.433521 (+343860us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:15.466932 (+ 33411us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:15.467003 (+    71us) write_op.cc:435] Acquired schema lock
0709 14:55:15.467037 (+    34us) tablet.cc:662] Decoding operations
0709 14:55:15.467603 (+   566us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:15.467738 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:15.467796 (+    58us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:55:15.468078 (+   282us) tablet.cc:701] Row locks acquired
0709 14:55:15.468114 (+    36us) write_op.cc:260] PREPARE: finished
0709 14:55:15.468345 (+   231us) write_op.cc:270] Start()
0709 14:55:15.468677 (+   332us) write_op.cc:276] Timestamp: P: 1783608915468262 usec, L: 0
0709 14:55:15.468706 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 14:55:15.469778 (+  1072us) log.cc:844] Serialized 540 byte log entry
0709 14:55:15.981963 (+512185us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b4400022b4c. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16b3e1db1 00007fa16b47fc85 00007fa16b4806a2 00007fa16e1f21db 00007fa16e1edaaf 00007fa16e1e186e 00007fa16e1e0bb6 00007fa16e1e07d8 00007fa16e1ea221 00007fa16e1ea1d9 00007fa16e1ea169
0709 14:55:16.496371 (+514408us) op_driver.cc:464] REPLICATION: finished
0709 14:55:16.645810 (+149439us) write_op.cc:301] APPLY: starting
0709 14:55:16.645941 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:16.647368 (+  1427us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:16.647391 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:16.648349 (+   958us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:16.648606 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 14:55:16.648671 (+    65us) write_op.cc:312] APPLY: finished
0709 14:55:16.648944 (+   273us) log.cc:844] Serialized 24 byte log entry
0709 14:55:16.649818 (+   874us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:16.649979 (+   161us) write_op.cc:454] Released schema lock
0709 14:55:16.650062 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:16.650624 (+   562us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:16.650668 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2479151,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":145800,"apply.run_cpu_time_us":3861,"apply.run_wall_time_us":4711,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":376865,"prepare.run_cpu_time_us":43182,"prepare.run_wall_time_us":416404,"raft.queue_time_us":244478,"raft.run_cpu_time_us":40566,"raft.run_wall_time_us":221070,"replication_time_us":994694,"spinlock_wait_cycles":139582720,"thread_start_us":866464,"threads_started":4,"wal-append.queue_time_us":378101,"wal-append.run_cpu_time_us":18320,"wal-append.run_wall_time_us":1941505}]]}
I20260709 14:55:18.842934 11188 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:18.851574 11192 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:18.820204  2433 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 15 to 16, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:18.881716  3412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 59653.3us
I20260709 14:55:18.892037  2433 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 12 to 16, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:18.902475   431 thread.cc:527] Waited for 21000ms trying to join with heartbeat (tid 3581)
W20260709 14:55:18.913655 11206 log.cc:927] Time spent T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 14:55:18.922049  2576 leader_election.cc:336] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:18.922667  2576 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:18.923549 11192 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:18.943351 10823 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.948318  2755 raft_consensus.cc:1240] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Rejecting Update request from peer c1b0e692e4294ee3b8694bbada174ba1 for earlier term 23. Current term is 24. Ops: []
I20260709 14:55:18.959204 11103 consensus_queue.cc:1059] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:18.960557 11103 raft_consensus.cc:3055] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 14:55:18.960983 11103 raft_consensus.cc:740] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:18.962010 11103 consensus_queue.cc:260] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:18.964210 11103 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 14:55:18.973110  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:18.978310  2433 catalog_manager.cc:5697] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 12 to 15, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:18.982630  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:18.988870 10823 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:18.992610  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:19.000874  2433 catalog_manager.cc:5697] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 10 to 17, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:19.003865  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:19.015260  2724 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:19.016462  2433 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.019260 11172 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 14:55:19.047827  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:19.052850  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:19.092607  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:19.153865 11212 raft_consensus.cc:493] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:19.154575 11212 raft_consensus.cc:515] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:19.157251 11212 leader_election.cc:290] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:19.159627  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:19.160702  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343553f3139e4da4ba16879fae174c5a" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:19.162974 11167 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:19.166133 11167 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:19.168978 11167 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:19.172606  3409 leader_election.cc:304] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:19.174748  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:19.178411  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57a97528f31400dbc0900e76292b021" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:19.179558 11167 raft_consensus.cc:2749] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:19.182380  3412 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:19.188179 11103 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:19.235323 11103 raft_consensus.cc:493] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:19.236073 11103 raft_consensus.cc:515] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:19.240068  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:19.242946 11103 leader_election.cc:290] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:19.242806  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e31634415994959847625e41eecf9b3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 27 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:19.263772  3411 leader_election.cc:304] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:19.268298 11137 raft_consensus.cc:2749] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 14:55:19.353856  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:19.383912  2433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 100) took 78519 ms (client timeout 15000 ms). Trace:
W20260709 14:55:19.384999  2433 rpcz_store.cc:269] 0709 14:54:00.862474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:00.862799 (+   325us) service_pool.cc:224] Handling call
0709 14:55:19.381932 (+78519133us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:19.107617 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:19.108437 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:19.108496 (+    59us) write_op.cc:435] Acquired schema lock
0709 14:55:19.108515 (+    19us) tablet.cc:662] Decoding operations
0709 14:55:19.123693 (+ 15178us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:19.123856 (+   163us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:19.123892 (+    36us) tablet.cc:685] Acquiring locks for 9 operations
0709 14:55:19.124923 (+  1031us) tablet.cc:701] Row locks acquired
0709 14:55:19.124978 (+    55us) write_op.cc:260] PREPARE: finished
0709 14:55:19.125213 (+   235us) write_op.cc:270] Start()
0709 14:55:19.125410 (+   197us) write_op.cc:276] Timestamp: P: 1783608919125124 usec, L: 0
0709 14:55:19.125459 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 14:55:19.126662 (+  1203us) log.cc:844] Serialized 3641 byte log entry
0709 14:55:19.201499 (+ 74837us) op_driver.cc:464] REPLICATION: finished
0709 14:55:19.245549 (+ 44050us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b5400110ea8. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174a029e1 00007fa174a0164c 00007fa17207d845 00007fa172081d61 00007fa172081d19 00007fa172081ca9 00007fa172081c71 00007fa172080e9d 0000000000396ad4 0000000000396919 00007fa16b559475
0709 14:55:19.245674 (+   125us) write_op.cc:301] APPLY: starting
0709 14:55:19.245786 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:19.278393 (+ 32607us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:19.278448 (+    55us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:19.318893 (+ 40445us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:19.319282 (+   389us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 14:55:19.319364 (+    82us) write_op.cc:312] APPLY: finished
0709 14:55:19.319826 (+   462us) log.cc:844] Serialized 88 byte log entry
0709 14:55:19.330594 (+ 10768us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.331098 (+   504us) write_op.cc:454] Released schema lock
0709 14:55:19.331200 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:19.333622 (+  2422us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.335106 (+  1484us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":295299,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":2044,"apply.run_cpu_time_us":18518,"apply.run_wall_time_us":127848,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41729,"num_ops":9,"prepare.queue_time_us":25774,"prepare.run_cpu_time_us":13142,"prepare.run_wall_time_us":80646,"raft.queue_time_us":9329,"raft.run_cpu_time_us":1698,"raft.run_wall_time_us":2235,"replication_time_us":75835,"spinlock_wait_cycles":4978944,"thread_start_us":66304,"threads_started":4,"wal-append.queue_time_us":33627}]]}
I20260709 14:55:19.407307  2434 catalog_manager.cc:5697] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.416095  2434 catalog_manager.cc:5697] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 21 to 27, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:19.425374  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:19.424877  2434 catalog_manager.cc:5697] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 22. New cstate: current_term: 22 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.428606  2434 catalog_manager.cc:5697] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.433467  2434 catalog_manager.cc:5697] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 22, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:19.438107  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:19.441677  2434 catalog_manager.cc:5697] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 20 to 28. New cstate: current_term: 28 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.455905  2434 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 20 to 27, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 27 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:19.460971  2434 catalog_manager.cc:5697] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 20 to 25, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 25 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:19.463104  2434 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.465454  2434 catalog_manager.cc:5697] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 21, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.467661  2434 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.474048  2434 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:19.475852  2434 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.485288  2434 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 16 to 23. New cstate: current_term: 23 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:19.490072  2434 catalog_manager.cc:5697] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 22. New cstate: current_term: 22 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.491886  2434 catalog_manager.cc:5697] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.493611  2434 catalog_manager.cc:5697] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.495290  2434 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 20, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:19.496788  2434 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 10 to 19, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.514331  2434 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:19.516230  2434 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.518110  2434 catalog_manager.cc:5697] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 16 to 24, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.520146  2434 catalog_manager.cc:5697] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 22, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 22 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:19.522287  2434 catalog_manager.cc:5697] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 24, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.538589  2434 catalog_manager.cc:5697] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:19.540475  2434 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.542937  2434 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 24, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.553714  2434 catalog_manager.cc:5697] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 7 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
W20260709 14:55:19.551880  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:19.556797  2434 catalog_manager.cc:5697] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 14 to 20, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:19.567139  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:19.644539  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:19.650396  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:19.737820  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:19.849289  2434 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50338 (request call id 170) took 13130 ms (client timeout 15000 ms). Trace:
W20260709 14:55:19.849973  2434 rpcz_store.cc:269] 0709 14:55:06.719649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:18.842792 (+12123143us) service_pool.cc:224] Handling call
0709 14:55:19.849175 (+1006383us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:19.611695 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:19.612309 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:19.612356 (+    47us) write_op.cc:435] Acquired schema lock
0709 14:55:19.612378 (+    22us) tablet.cc:662] Decoding operations
0709 14:55:19.616395 (+  4017us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:19.616456 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:19.616483 (+    27us) tablet.cc:685] Acquiring locks for 29 operations
0709 14:55:19.621475 (+  4992us) tablet.cc:701] Row locks acquired
0709 14:55:19.621501 (+    26us) write_op.cc:260] PREPARE: finished
0709 14:55:19.621837 (+   336us) write_op.cc:270] Start()
0709 14:55:19.621988 (+   151us) write_op.cc:276] Timestamp: P: 1783608919621750 usec, L: 0
0709 14:55:19.622013 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 14:55:19.623132 (+  1119us) log.cc:844] Serialized 11401 byte log entry
0709 14:55:19.626785 (+  3653us) op_driver.cc:464] REPLICATION: finished
0709 14:55:19.627406 (+   621us) write_op.cc:301] APPLY: starting
0709 14:55:19.627500 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:19.661141 (+ 33641us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:19.661245 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:19.709156 (+ 47911us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:19.709659 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 14:55:19.709717 (+    58us) write_op.cc:312] APPLY: finished
0709 14:55:19.710437 (+   720us) log.cc:844] Serialized 251 byte log entry
0709 14:55:19.710695 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.711496 (+   801us) write_op.cc:454] Released schema lock
0709 14:55:19.711573 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:19.712030 (+   457us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.712060 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34718,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":329,"apply.run_cpu_time_us":36949,"apply.run_wall_time_us":84576,"cfile_cache_hit":116,"cfile_cache_hit_bytes":121162,"num_ops":29,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":7768,"prepare.run_wall_time_us":11774,"replication_time_us":4553}]]}
I20260709 14:55:19.845815  2430 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 12 to 18, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:19.875623  2429 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 160) took 138338 ms (client timeout 15000 ms). Trace:
W20260709 14:55:19.877924  2429 rpcz_store.cc:269] 0709 14:53:01.537066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:01.537479 (+   413us) service_pool.cc:224] Handling call
0709 14:55:19.875503 (+138338024us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1612572,"reported_tablets":43}
I20260709 14:55:19.876636  2430 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 14 to 15, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:19.880438  2430 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:19.891808  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:19.903173   431 thread.cc:527] Waited for 22000ms trying to join with heartbeat (tid 3581)
W20260709 14:55:19.976018  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:19.977945  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:19.987000 11103 raft_consensus.cc:493] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:19.987773 11103 raft_consensus.cc:515] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:19.993325 11103 leader_election.cc:290] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:20.009166  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:20.065659  2430 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 163) took 90121 ms (client timeout 15000 ms). Trace:
W20260709 14:55:20.068586  2430 rpcz_store.cc:269] 0709 14:53:49.944788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:53:49.945110 (+   322us) service_pool.cc:224] Handling call
0709 14:55:20.065552 (+90120442us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:19.919731 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:19.920324 (+   593us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:19.920363 (+    39us) write_op.cc:435] Acquired schema lock
0709 14:55:19.920384 (+    21us) tablet.cc:662] Decoding operations
0709 14:55:19.920981 (+   597us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:19.921049 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:19.921080 (+    31us) tablet.cc:685] Acquiring locks for 3 operations
0709 14:55:19.921438 (+   358us) tablet.cc:701] Row locks acquired
0709 14:55:19.921455 (+    17us) write_op.cc:260] PREPARE: finished
0709 14:55:19.921630 (+   175us) write_op.cc:270] Start()
0709 14:55:19.921762 (+   132us) write_op.cc:276] Timestamp: P: 1783608919921576 usec, L: 0
0709 14:55:19.921779 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 14:55:19.922595 (+   816us) log.cc:844] Serialized 1316 byte log entry
0709 14:55:19.925372 (+  2777us) op_driver.cc:464] REPLICATION: finished
0709 14:55:19.925986 (+   614us) write_op.cc:301] APPLY: starting
0709 14:55:19.926107 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:19.928205 (+  2098us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:19.928318 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:19.930312 (+  1994us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:19.930555 (+   243us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 14:55:19.930611 (+    56us) write_op.cc:312] APPLY: finished
0709 14:55:19.941036 (+ 10425us) log.cc:844] Serialized 40 byte log entry
0709 14:55:19.941960 (+   924us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.942080 (+   120us) write_op.cc:454] Released schema lock
0709 14:55:19.942131 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:19.948040 (+  5909us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:19.948082 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2912368,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":320,"apply.run_cpu_time_us":5826,"apply.run_wall_time_us":16566,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12534,"num_ops":3,"prepare.queue_time_us":7583,"prepare.run_cpu_time_us":3280,"prepare.run_wall_time_us":11404,"replication_time_us":3407}]]}
I20260709 14:55:20.093190  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baf9a66ae7e41df9d5d2079bebb7381" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:20.098129  3409 leader_election.cc:304] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:20.097812 11219 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:20.100018 11219 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.099819 11103 raft_consensus.cc:2749] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:20.103003 11219 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:20.105257  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:20.120966  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e0e0c146c54411901808c4bbf22280" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:20.128958  3412 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:20.136765 11167 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:20.142926  2429 catalog_manager.cc:5697] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:20.150787  2429 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 20 to 28. New cstate: current_term: 28 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.186043  2429 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 26. New cstate: current_term: 26 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.193011  2429 catalog_manager.cc:5697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 11 to 25, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.196957  2429 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 30, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 30 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:20.206460 11220 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:20.209383 11220 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.211828 11220 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:20.213842  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:20.207645  2429 catalog_manager.cc:5697] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.218195  2429 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 16 to 26. New cstate: current_term: 26 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.220080 11187 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:20.220795 11187 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.222673  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:20.224092  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:20.225011  2755 raft_consensus.cc:2393] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 24: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:20.227587  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:20.229313 11167 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:20.238480 11187 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:20.241078  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:20.241596  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:20.242792 11192 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:20.220067  2429 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 14 to 22, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 22 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:20.281000  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:20.254637  2429 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 24 to 26, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.287213  2429 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 28, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 28 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:20.288717  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:20.299904  2432 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 98) took 143375 ms (client timeout 15000 ms). Trace:
I20260709 14:55:20.308037 11192 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:20.305251  2432 rpcz_store.cc:269] 0709 14:52:56.923986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:52:57.008441 (+ 84455us) service_pool.cc:224] Handling call
0709 14:55:20.299635 (+143291194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":168829,"reported_tablets":45}
I20260709 14:55:20.288980  2429 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.315507  2429 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.317718  2429 catalog_manager.cc:5697] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 22 to 26, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.319797  2429 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:20.322011  2429 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 16 to 25, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:20.323822  2429 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 24 to 26, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.325984  2429 catalog_manager.cc:5697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 9 to 24. New cstate: current_term: 24 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.327787  2429 catalog_manager.cc:5697] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 17 to 25, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 25 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.329519  2429 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 24 to 25, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:20.331621  2429 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.333925  2429 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 21 to 22, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 22 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:20.352735  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:20.335952  2429 catalog_manager.cc:5697] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 15 to 25, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 25 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:20.371685  2429 catalog_manager.cc:5697] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 19 to 26, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.375712  2429 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:20.377620  2429 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } }
I20260709 14:55:20.379460  2429 catalog_manager.cc:5697] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:20.381495  2429 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 18 to 21, leader changed from 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
W20260709 14:55:20.386307  2785 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:39575 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:39575 timed out after 15.000s (SENT)
W20260709 14:55:20.428272  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:20.502486  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:20.529388 11167 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:20.530122 11167 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.532646 11167 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:20.536749  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:20.537701  2605 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:20.538192  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:20.539124  2755 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 24.
I20260709 14:55:20.541626  3412 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:20.543828 11220 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:20.544323 11220 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:20.544693 11220 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:20.599596 11220 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.617063  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:20.617950  2755 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:20.629761  2755 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
I20260709 14:55:20.631120  3411 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1; no voters: 
I20260709 14:55:20.632685 11219 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 14:55:20.633329 11219 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Becoming Leader. State: Replica: c1b0e692e4294ee3b8694bbada174ba1, State: Running, Role: LEADER
I20260709 14:55:20.634328 11219 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:20.644065 11220 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:20.645619  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:20.646469  2604 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:20.658708  2604 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b0e692e4294ee3b8694bbada174ba1 in term 25.
W20260709 14:55:20.679687  3409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 51207.6us
W20260709 14:55:20.732398  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:20.894423  2431 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 12 to 15, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:20.905081  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:20.905135   431 thread.cc:527] Waited for 23000ms trying to join with heartbeat (tid 3581)
I20260709 14:55:20.951545  2431 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 18 to 21, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:20.956050  2431 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 18 to 20, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:20.976253  2431 catalog_manager.cc:5697] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:20.978756  2431 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:20.981935  2431 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 14 to 20, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:20.985081  2431 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 17 to 20, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.018311  2431 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 15 to 19, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:21.031327  2431 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 reported cstate change: term changed from 15 to 16, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:21.148934  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:21.154668  2612 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 14:55:21.157315 11103 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 14:55:21.163475  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 45933.9us
I20260709 14:55:21.227391  2760 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer c1b0e692e4294ee3b8694bbada174ba1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 14:55:21.229712 11137 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 14:55:21.254313  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 61577.7us
W20260709 14:55:21.282910  2431 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 101) took 44311 ms (client timeout 15000 ms). Trace:
W20260709 14:55:21.283517  2431 rpcz_store.cc:269] 0709 14:54:36.971602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:36.972170 (+   568us) service_pool.cc:224] Handling call
0709 14:55:21.282782 (+44310612us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:21.038498 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:21.039203 (+   705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:21.039254 (+    51us) write_op.cc:435] Acquired schema lock
0709 14:55:21.039275 (+    21us) tablet.cc:662] Decoding operations
0709 14:55:21.040430 (+  1155us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:21.040508 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:21.040544 (+    36us) tablet.cc:685] Acquiring locks for 9 operations
0709 14:55:21.041409 (+   865us) tablet.cc:701] Row locks acquired
0709 14:55:21.041426 (+    17us) write_op.cc:260] PREPARE: finished
0709 14:55:21.041674 (+   248us) write_op.cc:270] Start()
0709 14:55:21.041820 (+   146us) write_op.cc:276] Timestamp: P: 1783608921041606 usec, L: 0
0709 14:55:21.041841 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 14:55:21.042710 (+   869us) log.cc:844] Serialized 3645 byte log entry
0709 14:55:21.071746 (+ 29036us) op_driver.cc:464] REPLICATION: finished
0709 14:55:21.073353 (+  1607us) write_op.cc:301] APPLY: starting
0709 14:55:21.073496 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:21.082395 (+  8899us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:21.082535 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:21.150249 (+ 67714us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:21.150541 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 14:55:21.150600 (+    59us) write_op.cc:312] APPLY: finished
0709 14:55:21.162807 (+ 12207us) log.cc:844] Serialized 88 byte log entry
0709 14:55:21.163094 (+   287us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.163258 (+   164us) write_op.cc:454] Released schema lock
0709 14:55:21.163331 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:21.163767 (+   436us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.163798 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3682,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":1342,"apply.run_cpu_time_us":15183,"apply.run_wall_time_us":90388,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37602,"num_ops":9,"prepare.queue_time_us":190,"prepare.run_cpu_time_us":4666,"prepare.run_wall_time_us":39941,"replication_time_us":29706}]]}
I20260709 14:55:21.276180  2427 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 14 to 20, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:21.292533  2427 catalog_manager.cc:5697] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 16 to 20, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:21.314199  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:21.336048 11137 raft_consensus.cc:493] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:21.336760 11137 raft_consensus.cc:515] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:21.343813  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:21.364799 11137 leader_election.cc:290] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:21.369110  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91eefb5789d41f0b4cd60e9f83fef3f" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:21.371579  3411 leader_election.cc:304] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:21.384967 11167 raft_consensus.cc:2749] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:21.388684  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:21.418993  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51777.1us
W20260709 14:55:21.433094  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:21.465142 11225 raft_consensus.cc:493] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:21.465955 11225 raft_consensus.cc:515] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:21.471139  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:21.471956 11225 leader_election.cc:290] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:21.473131  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1826b730f9f4ae4b0e05570574563c3" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:21.474985  3412 leader_election.cc:304] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:21.479712 11137 raft_consensus.cc:2749] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:21.492177 11137 raft_consensus.cc:493] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:21.492967 11137 raft_consensus.cc:515] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:21.497519 11137 leader_election.cc:290] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:21.500926  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:21.503410  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5cca18a56d54a9aae32ff20c48ef0f8" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:21.505504  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:21.507278  3411 leader_election.cc:304] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:21.512586 11137 raft_consensus.cc:2749] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:21.487741 11225 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:21.514230 11225 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:21.516425  2426 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 15 to 22, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 22 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.518637 11225 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:21.521749  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:21.522354  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac95ecab0524f35a894d205dac97a44" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:21.525243  3411 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:21.526746 11225 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:21.526072  2426 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 20 to 21, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:55:21.538126  2427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 163) took 41634 ms (client timeout 15000 ms). Trace:
I20260709 14:55:21.540334  2426 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.545734  2426 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 16 to 17, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.555361  2426 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 14 to 21, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.566478  2426 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 10 to 16, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:21.538934  2427 rpcz_store.cc:269] 0709 14:54:39.903341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:53.113338 (+13209997us) service_pool.cc:224] Handling call
0709 14:55:21.538010 (+28424672us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:21.329917 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:21.330595 (+   678us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:21.330633 (+    38us) write_op.cc:435] Acquired schema lock
0709 14:55:21.330653 (+    20us) tablet.cc:662] Decoding operations
0709 14:55:21.331187 (+   534us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:21.331258 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:21.331299 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 14:55:21.331719 (+   420us) tablet.cc:701] Row locks acquired
0709 14:55:21.331737 (+    18us) write_op.cc:260] PREPARE: finished
0709 14:55:21.331927 (+   190us) write_op.cc:270] Start()
0709 14:55:21.332082 (+   155us) write_op.cc:276] Timestamp: P: 1783608921331863 usec, L: 0
0709 14:55:21.332104 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 14:55:21.340167 (+  8063us) log.cc:844] Serialized 928 byte log entry
0709 14:55:21.351363 (+ 11196us) op_driver.cc:464] REPLICATION: finished
0709 14:55:21.351851 (+   488us) write_op.cc:301] APPLY: starting
0709 14:55:21.351968 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:21.353596 (+  1628us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:21.353710 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:21.359985 (+  6275us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18003607c0. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16b559522 00007fa16b55b7c1 00007fa16b55b779 00007fa16b55b709 00007fa16b55b6d1 00007fa16b55a9cd 0000000000396ad4 0000000000396919 00007fa16b53c83a
0709 14:55:21.360563 (+   578us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:21.360764 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 14:55:21.360814 (+    50us) write_op.cc:312] APPLY: finished
0709 14:55:21.361073 (+   259us) log.cc:844] Serialized 32 byte log entry
0709 14:55:21.361288 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.361400 (+   112us) write_op.cc:454] Released schema lock
0709 14:55:21.361452 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:21.361917 (+   465us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.361960 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6530,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":138,"apply.run_cpu_time_us":4950,"apply.run_wall_time_us":9972,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8356,"num_ops":2,"prepare.queue_time_us":165,"prepare.run_cpu_time_us":3556,"prepare.run_wall_time_us":18189,"replication_time_us":16619,"spinlock_wait_cycles":5526528}]]}
W20260709 14:55:21.636065  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:21.642565  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:21.685575  2426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 165) took 25604 ms (client timeout 15000 ms). Trace:
I20260709 14:55:21.694350  2428 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.698601  2428 catalog_manager.cc:5697] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 21 to 22, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:55:21.695765  2426 rpcz_store.cc:269] 0709 14:54:56.081857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:07.412730 (+11330873us) service_pool.cc:224] Handling call
0709 14:55:21.685461 (+14272731us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:21.594573 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:21.595315 (+   742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:21.595360 (+    45us) write_op.cc:435] Acquired schema lock
0709 14:55:21.595384 (+    24us) tablet.cc:662] Decoding operations
0709 14:55:21.596338 (+   954us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:21.596415 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:21.596450 (+    35us) tablet.cc:685] Acquiring locks for 6 operations
0709 14:55:21.597110 (+   660us) tablet.cc:701] Row locks acquired
0709 14:55:21.597132 (+    22us) write_op.cc:260] PREPARE: finished
0709 14:55:21.597325 (+   193us) write_op.cc:270] Start()
0709 14:55:21.597484 (+   159us) write_op.cc:276] Timestamp: P: 1783608921597265 usec, L: 0
0709 14:55:21.597509 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 14:55:21.598633 (+  1124us) log.cc:844] Serialized 2481 byte log entry
0709 14:55:21.601643 (+  3010us) op_driver.cc:464] REPLICATION: finished
0709 14:55:21.602242 (+   599us) write_op.cc:301] APPLY: starting
0709 14:55:21.602355 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:21.605960 (+  3605us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:21.606112 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:21.612890 (+  6778us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:21.613125 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 14:55:21.613170 (+    45us) write_op.cc:312] APPLY: finished
0709 14:55:21.613472 (+   302us) log.cc:844] Serialized 64 byte log entry
0709 14:55:21.627064 (+ 13592us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.627225 (+   161us) write_op.cc:454] Released schema lock
0709 14:55:21.627298 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:21.627874 (+   576us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.627911 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31103,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":349,"apply.run_cpu_time_us":10064,"apply.run_wall_time_us":25583,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25068,"num_ops":6,"prepare.queue_time_us":702,"prepare.run_cpu_time_us":4389,"prepare.run_wall_time_us":16513,"replication_time_us":3954}]]}
I20260709 14:55:21.705523  2428 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 21 to 23, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.708563  2428 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.722551 11137 raft_consensus.cc:493] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:55:21.723222  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:21.723310 11137 raft_consensus.cc:515] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:21.725920 11137 leader_election.cc:290] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:21.728063  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:21.735291  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb844e4a36641d79afdab73422e06f1" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:21.737419  3409 leader_election.cc:304] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:21.740236  2428 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 16 to 17, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.742638 11225 raft_consensus.cc:2749] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 14:55:21.748771  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:21.843300 11137 raft_consensus.cc:493] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:21.843940 11137 raft_consensus.cc:515] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:21.848979  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:21.853188  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f87ae033ca94d58bdc3a087236657b7" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:21.855574  3412 leader_election.cc:304] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:21.856878 11103 raft_consensus.cc:2749] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:21.851263 11137 leader_election.cc:290] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:21.877928  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:21.879238  2428 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 164) took 16497 ms (client timeout 15000 ms). Trace:
W20260709 14:55:21.891176  2428 rpcz_store.cc:269] 0709 14:55:05.382185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:15.299658 (+9917473us) service_pool.cc:224] Handling call
0709 14:55:15.927164 (+627506us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54001130d4. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5b0090 00007fa16e64ef3a 00007fa1744051cc 00007fa17450337e 00007fa1729c42a4 00007fa1729c4232 00007fa1729c4161 00007fa1729c40dc 00007fa1729c3392 00007fa16c98148c 00007fa16c9808c6
0709 14:55:21.879113 (+5951949us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:21.757504 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:21.758142 (+   638us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:21.758177 (+    35us) write_op.cc:435] Acquired schema lock
0709 14:55:21.758198 (+    21us) tablet.cc:662] Decoding operations
0709 14:55:21.758958 (+   760us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:21.759027 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:21.759059 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 14:55:21.759643 (+   584us) tablet.cc:701] Row locks acquired
0709 14:55:21.759663 (+    20us) write_op.cc:260] PREPARE: finished
0709 14:55:21.759829 (+   166us) write_op.cc:270] Start()
0709 14:55:21.759961 (+   132us) write_op.cc:276] Timestamp: P: 1783608921759772 usec, L: 0
0709 14:55:21.759981 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 14:55:21.760705 (+   724us) log.cc:844] Serialized 2093 byte log entry
0709 14:55:21.763391 (+  2686us) op_driver.cc:464] REPLICATION: finished
0709 14:55:21.763893 (+   502us) write_op.cc:301] APPLY: starting
0709 14:55:21.764004 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:21.767319 (+  3315us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:21.767438 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:21.771229 (+  3791us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:21.771478 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 14:55:21.771538 (+    60us) write_op.cc:312] APPLY: finished
0709 14:55:21.771845 (+   307us) log.cc:844] Serialized 56 byte log entry
0709 14:55:21.772932 (+  1087us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.773057 (+   125us) write_op.cc:454] Released schema lock
0709 14:55:21.773166 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:21.773606 (+   440us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:21.773638 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":82,"reported_tablets":44,"spinlock_wait_cycles":189904256,"child_traces":[["op",{"apply.queue_time_us":280,"apply.run_cpu_time_us":8520,"apply.run_wall_time_us":9668,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20890,"num_ops":5,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":3595,"prepare.run_wall_time_us":13580,"replication_time_us":3233}]]}
W20260709 14:55:21.910399   431 thread.cc:527] Waited for 24000ms trying to join with heartbeat (tid 3581)
W20260709 14:55:21.918807  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:21.959470  2434 catalog_manager.cc:5697] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 22 to 25, leader changed from 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.962513  2434 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:55:21.977546  2435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 164) took 63407 ms (client timeout 15000 ms). Trace:
W20260709 14:55:21.978227  2435 rpcz_store.cc:269] 0709 14:54:18.569866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:54:18.570058 (+   192us) service_pool.cc:224] Handling call
0709 14:55:21.977274 (+63407216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28411,"reported_tablets":44}
I20260709 14:55:21.980472  2434 catalog_manager.cc:5697] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 20 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.984303  2434 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.986857  2434 catalog_manager.cc:5697] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 23 to 25, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.989744  2434 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 21 to 27, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 27 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } }
I20260709 14:55:21.991704  2434 catalog_manager.cc:5697] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 15 to 20, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.994004  2434 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 17 to 20, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:21.995921  2434 catalog_manager.cc:5697] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:21.997875  2434 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:22.000562  2434 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:55:22.107563  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:22.113380  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:22.115777  2434 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 166) took 7044 ms. Trace:
I20260709 14:55:22.116925  2434 rpcz_store.cc:276] 0709 14:55:15.071828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:19.865271 (+4793443us) service_pool.cc:224] Handling call
0709 14:55:22.115666 (+2250395us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:22.007411 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:22.008001 (+   590us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:22.008091 (+    90us) write_op.cc:435] Acquired schema lock
0709 14:55:22.008103 (+    12us) tablet.cc:662] Decoding operations
0709 14:55:22.009170 (+  1067us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:22.009232 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:22.009262 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 14:55:22.012231 (+  2969us) tablet.cc:701] Row locks acquired
0709 14:55:22.012261 (+    30us) write_op.cc:260] PREPARE: finished
0709 14:55:22.012716 (+   455us) write_op.cc:270] Start()
0709 14:55:22.012882 (+   166us) write_op.cc:276] Timestamp: P: 1783608922012389 usec, L: 0
0709 14:55:22.012908 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 14:55:22.013802 (+   894us) log.cc:844] Serialized 4421 byte log entry
0709 14:55:22.020470 (+  6668us) op_driver.cc:464] REPLICATION: finished
0709 14:55:22.021972 (+  1502us) write_op.cc:301] APPLY: starting
0709 14:55:22.022074 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:22.029857 (+  7783us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:22.029973 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:22.046945 (+ 16972us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:22.047217 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 14:55:22.047269 (+    52us) write_op.cc:312] APPLY: finished
0709 14:55:22.047904 (+   635us) log.cc:844] Serialized 104 byte log entry
0709 14:55:22.048835 (+   931us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:22.049011 (+   176us) write_op.cc:454] Released schema lock
0709 14:55:22.049071 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:22.076600 (+ 27529us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:22.076633 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41727,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1273,"apply.run_cpu_time_us":15337,"apply.run_wall_time_us":54535,"cfile_cache_hit":44,"cfile_cache_hit_bytes":45958,"num_ops":11,"prepare.queue_time_us":122,"prepare.run_cpu_time_us":4755,"prepare.run_wall_time_us":21094,"replication_time_us":7412}]]}
W20260709 14:55:22.139909  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:22.188105  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:22.194643 11103 raft_consensus.cc:493] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:22.195458 11103 raft_consensus.cc:515] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:22.204821  2432 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
W20260709 14:55:22.220085  2433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52794 (request call id 102) took 14397 ms (client timeout 15000 ms). Trace:
W20260709 14:55:22.220593  2433 rpcz_store.cc:269] 0709 14:55:07.822295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:19.418967 (+11596672us) service_pool.cc:224] Handling call
0709 14:55:22.220010 (+2801043us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7404,"reported_tablets":45}
I20260709 14:55:22.229640 11103 leader_election.cc:290] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:22.231212  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:22.239352  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4837b30553bb4eac8840db77c7ffca53" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:22.249670   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:55:22.252104   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.254112   431 raft_consensus.cc:2243] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.255532   431 raft_consensus.cc:2272] T f01d4b3ec58f4ca3b994c5e66179174a P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.259943   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.264338   431 raft_consensus.cc:2243] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.265277   431 raft_consensus.cc:2272] T 80e0e0c146c54411901808c4bbf22280 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.266297  3412 leader_election.cc:304] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:22.271175   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.275938 11225 raft_consensus.cc:2749] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:22.288365   431 raft_consensus.cc:2243] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.291240   431 raft_consensus.cc:2272] T eeb3e91a07fc4383bd55f034f23c3cea P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.295573   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.298349  2434 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "c1b0e692e4294ee3b8694bbada174ba1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } }
I20260709 14:55:22.312525   431 raft_consensus.cc:2243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.313795   431 pending_rounds.cc:70] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 1 pending ops.
I20260709 14:55:22.318501   431 pending_rounds.cc:77] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 19 index: 9 } timestamp: 7305661967383199744 op_type: NO_OP noop_request { }
I20260709 14:55:22.323328   431 raft_consensus.cc:2889] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.323911   431 raft_consensus.cc:2272] T e7c0ea3d097a4b27b9367e3b84b6d5fd P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.327921   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.344295   431 raft_consensus.cc:2243] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.334334  2432 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50348 (request call id 165) took 1889 ms. Trace:
I20260709 14:55:22.350220  2432 rpcz_store.cc:276] 0709 14:55:20.444698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:20.445808 (+  1110us) service_pool.cc:224] Handling call
0709 14:55:22.334230 (+1888422us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:22.219892 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:22.220482 (+   590us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:22.220519 (+    37us) write_op.cc:435] Acquired schema lock
0709 14:55:22.220541 (+    22us) tablet.cc:662] Decoding operations
0709 14:55:22.221073 (+   532us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:22.221138 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:22.221165 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:55:22.221376 (+   211us) tablet.cc:701] Row locks acquired
0709 14:55:22.221395 (+    19us) write_op.cc:260] PREPARE: finished
0709 14:55:22.221596 (+   201us) write_op.cc:270] Start()
0709 14:55:22.221721 (+   125us) write_op.cc:276] Timestamp: P: 1783608922221540 usec, L: 0
0709 14:55:22.221737 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 14:55:22.222432 (+   695us) log.cc:844] Serialized 540 byte log entry
0709 14:55:22.224733 (+  2301us) op_driver.cc:464] REPLICATION: finished
0709 14:55:22.225251 (+   518us) write_op.cc:301] APPLY: starting
0709 14:55:22.225344 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:22.226358 (+  1014us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:22.226479 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:22.227336 (+   857us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:22.227537 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 14:55:22.227584 (+    47us) write_op.cc:312] APPLY: finished
0709 14:55:22.227816 (+   232us) log.cc:844] Serialized 24 byte log entry
0709 14:55:22.228011 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:22.228107 (+    96us) write_op.cc:454] Released schema lock
0709 14:55:22.228153 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:22.228620 (+   467us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:22.228649 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":284,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":293,"apply.run_cpu_time_us":3319,"apply.run_wall_time_us":3320,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":515,"prepare.run_cpu_time_us":2881,"prepare.run_wall_time_us":2879,"replication_time_us":2863}]]}
I20260709 14:55:22.355612   431 raft_consensus.cc:2272] T a594c31352514fb685540e1082f8ee7d P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.374773   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.391677   431 raft_consensus.cc:2243] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.393231   431 raft_consensus.cc:2272] T 1c902c27b99b4cc0a5186bc8117246a7 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.397576   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.431774   431 raft_consensus.cc:2243] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.442236   431 raft_consensus.cc:2272] T 75e6d6e528724dcba0bbaf89427c37a9 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.445940   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.447265   431 raft_consensus.cc:2243] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.448040   431 raft_consensus.cc:2272] T 750a16eb87de49b4a212064463a9d3a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:22.454684  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:22.481858  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:22.506865   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.512276   431 raft_consensus.cc:2243] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.513230   431 pending_rounds.cc:70] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 2 pending ops.
I20260709 14:55:22.520026   431 pending_rounds.cc:77] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305662010326192128 op_type: NO_OP noop_request { }
I20260709 14:55:22.520519   431 raft_consensus.cc:2889] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.520862   431 pending_rounds.cc:77] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305661962647027712 op_type: NO_OP noop_request { }
I20260709 14:55:22.527199   431 raft_consensus.cc:2889] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.527765   431 raft_consensus.cc:2272] T a2e7eae198c34a6fafb697d50f269192 P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.532392   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.568610   431 raft_consensus.cc:2243] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.569536   431 raft_consensus.cc:2272] T 343553f3139e4da4ba16879fae174c5a P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.572290 11167 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:22.572992 11167 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:22.575728 11167 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:22.577726  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:22.580120  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9427dea4aea4238bc56533da5d2720e" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 28 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:22.583381  3412 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:22.585047 11225 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 14:55:22.604187  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:22.621548   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.623070   431 raft_consensus.cc:2243] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.641784   431 pending_rounds.cc:70] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 1 pending ops.
I20260709 14:55:22.642241   431 pending_rounds.cc:77] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305661991322902528 op_type: NO_OP noop_request { }
I20260709 14:55:22.642712   431 raft_consensus.cc:2889] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.643249   431 raft_consensus.cc:2272] T dee6ef23c038421d972947971af9f0cc P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:22.647210  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 68306us
I20260709 14:55:22.651607   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.653162   431 raft_consensus.cc:2243] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.658106   431 raft_consensus.cc:2272] T ff9fe3d5f686477aa2bc32f628dcfb08 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.662532   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.678257   431 raft_consensus.cc:2243] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.679855   431 raft_consensus.cc:2272] T c1bae386d4ba412d86a21df33a9fa819 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.709555   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.718940   431 raft_consensus.cc:2243] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.735471   431 raft_consensus.cc:2272] T 7a9cf6350d3b499497e3ace2d1dea988 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.739285   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.755853   431 raft_consensus.cc:2243] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.756898   431 raft_consensus.cc:2272] T e9427dea4aea4238bc56533da5d2720e P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.773568   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.775250   431 raft_consensus.cc:2243] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.776255   431 raft_consensus.cc:2272] T 4837b30553bb4eac8840db77c7ffca53 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:22.786669  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:22.787099   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.796357   431 raft_consensus.cc:2243] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.797158 11167 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:22.807266 11167 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:22.810011 11167 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:22.797147  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:22.811393  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:22.812155  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd47a5b34fb04ee887e263844b637438" candidate_uuid: "c1b0e692e4294ee3b8694bbada174ba1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:22.813988  3412 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1b0e692e4294ee3b8694bbada174ba1; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:22.814914 11167 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:22.816354   431 raft_consensus.cc:2272] T 8e31634415994959847625e41eecf9b3 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.821678   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.822942   431 raft_consensus.cc:2243] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.824577   431 raft_consensus.cc:2272] T 7abc2eae2c0540e4bc1c7141e81bf63c P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.828316   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.830806   431 raft_consensus.cc:2243] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.832176   431 raft_consensus.cc:2272] T c051e44d64d14770b36c5012be981196 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.836030   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.837461   431 raft_consensus.cc:2243] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.839092   431 raft_consensus.cc:2272] T 100d7ba791034203ab40e147a55e2183 P c1b0e692e4294ee3b8694bbada174ba1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.843075   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.844441   431 raft_consensus.cc:2243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.846074   431 raft_consensus.cc:2272] T 59c97f578e0e4d3f9c754fb31c0b4c6a P c1b0e692e4294ee3b8694bbada174ba1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.862432   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.863727   431 raft_consensus.cc:2243] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:22.865631   431 raft_consensus.cc:2272] T 5d0f08b71bff4091b9f08d398c3f894d P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.881453   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.882746   431 raft_consensus.cc:2243] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.885548   431 raft_consensus.cc:2272] T b47a7b94ee76444db9c73a90f520af7c P c1b0e692e4294ee3b8694bbada174ba1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.907684   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.909118   431 raft_consensus.cc:2243] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.915184   431 pending_rounds.cc:70] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 1 pending ops.
I20260709 14:55:22.915761   431 pending_rounds.cc:77] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305661986720915456 op_type: NO_OP noop_request { }
I20260709 14:55:22.916282   431 raft_consensus.cc:2889] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.916780   431 raft_consensus.cc:2272] T b5cca18a56d54a9aae32ff20c48ef0f8 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.926102   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.934510   431 raft_consensus.cc:2243] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.941322   431 raft_consensus.cc:2272] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.955415   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.964929   431 raft_consensus.cc:2243] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.965863   431 pending_rounds.cc:70] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 2 pending ops.
I20260709 14:55:22.966238   431 pending_rounds.cc:77] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 22 index: 12 } timestamp: 7305662031371677696 op_type: NO_OP noop_request { }
I20260709 14:55:22.966696   431 raft_consensus.cc:2889] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.967108   431 pending_rounds.cc:77] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 20 index: 11 } timestamp: 7305661986555830272 op_type: NO_OP noop_request { }
I20260709 14:55:22.967547   431 raft_consensus.cc:2889] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:22.967983   431 raft_consensus.cc:2272] T e91eefb5789d41f0b4cd60e9f83fef3f P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.978440   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.987677   431 raft_consensus.cc:2243] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:22.988574   431 raft_consensus.cc:2272] T da98c08aee5d4d86bce741e08142c1e3 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:22.992069   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:22.998332   431 raft_consensus.cc:2243] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.007401   431 raft_consensus.cc:2272] T 1f87ae033ca94d58bdc3a087236657b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.011634   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.019767   431 raft_consensus.cc:2243] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.020875   431 raft_consensus.cc:2272] T 8fb844e4a36641d79afdab73422e06f1 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.052615   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.059800   431 raft_consensus.cc:2243] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.061044   431 raft_consensus.cc:2272] T 947b4a5caeb243ef87d3a535eddda080 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.065295   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.083092   431 raft_consensus.cc:2243] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:23.081655  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:23.084076   431 raft_consensus.cc:2272] T a1826b730f9f4ae4b0e05570574563c3 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.087670   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.095695   431 raft_consensus.cc:2243] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.098706   431 raft_consensus.cc:2272] T f32f10706d04497089f6159669dc04a4 P c1b0e692e4294ee3b8694bbada174ba1 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:23.108850  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:23.110284   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.111420   431 raft_consensus.cc:2243] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.112685   431 raft_consensus.cc:2272] T 8b56ad927e054a56a77c811620dda543 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:23.105430  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:23.132426   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.133575   431 raft_consensus.cc:2243] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.134356   431 raft_consensus.cc:2272] T 4ac95ecab0524f35a894d205dac97a44 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.138311   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.155407   431 raft_consensus.cc:2243] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.156909   431 raft_consensus.cc:2272] T 5e1073216da14f9c91bcc3f31f935620 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.160399   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.161398   431 raft_consensus.cc:2243] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.162797   431 raft_consensus.cc:2272] T 406cc4479d48441398435d7dc1850d64 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.182425   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.183458   431 raft_consensus.cc:2243] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.184247   431 raft_consensus.cc:2272] T 7584380efde04ee289b42acaea2e4742 P c1b0e692e4294ee3b8694bbada174ba1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.187842   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.188805   431 raft_consensus.cc:2243] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.189540   431 pending_rounds.cc:70] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Trying to abort 2 pending ops.
I20260709 14:55:23.189844   431 pending_rounds.cc:77] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305661975402487808 op_type: NO_OP noop_request { }
I20260709 14:55:23.190263   431 raft_consensus.cc:2889] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:23.190596   431 pending_rounds.cc:77] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1: Aborting op as it isn't in flight: id { term: 19 index: 10 } timestamp: 7305661942506450944 op_type: NO_OP noop_request { }
I20260709 14:55:23.190987   431 raft_consensus.cc:2889] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:23.191391   431 raft_consensus.cc:2272] T a57a97528f31400dbc0900e76292b021 P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.195153   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.196156   431 raft_consensus.cc:2243] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.197714   431 raft_consensus.cc:2272] T 9a8bd118baa7481399ce15d422463083 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.201428   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.202387   431 raft_consensus.cc:2243] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.203061   431 raft_consensus.cc:2272] T 0d27004b985a45779b0f86fcb7273dbc P c1b0e692e4294ee3b8694bbada174ba1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.206434   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.207340   431 raft_consensus.cc:2243] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.207986   431 raft_consensus.cc:2272] T 6baf9a66ae7e41df9d5d2079bebb7381 P c1b0e692e4294ee3b8694bbada174ba1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.211302   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.212143   431 raft_consensus.cc:2243] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.213469   431 raft_consensus.cc:2272] T 31e4166c53c7459b808e9e6841d90c56 P c1b0e692e4294ee3b8694bbada174ba1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.216919   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.217875   431 raft_consensus.cc:2243] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.219280   431 raft_consensus.cc:2272] T 963a4155717b4274b0121341056b7cb8 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.223397   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.224359   431 raft_consensus.cc:2243] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:23.225205   431 raft_consensus.cc:2272] T dd47a5b34fb04ee887e263844b637438 P c1b0e692e4294ee3b8694bbada174ba1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:23.229238   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:55:23.229470 11189 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.230062   431 raft_consensus.cc:2243] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 14:55:23.230284 11189 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.231274   431 raft_consensus.cc:2272] T acb30692225e4daebaa7e02e9cb291b7 P c1b0e692e4294ee3b8694bbada174ba1 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:23.241923  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:23.247045  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:23.249251  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.250682 11239 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:23.252053 11189 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:23.261597  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:23.270171  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:23.402878 11238 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.403682 11238 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:23.411814  2576 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:23.414551  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:23.415760 11238 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.416777  2575 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.420652 11238 raft_consensus.cc:2749] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:55:23.432788  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:23.452230 11241 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.452960 11241 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:23.461967  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:23.469832  2724 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:23.470456  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.474344 11241 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:23.477339 11231 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:23.493059  2576 connection.cc:573] client connection to 127.0.107.193:37831 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 14:55:23.594579 11231 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.595513 11231 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:23.599735 11231 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:23.602218  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:23.619223  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.619791  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.631083 11231 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:23.633175 11189 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.634004 11189 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.643584  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:23.651223  2650 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.651870  2650 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.663290 11238 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.669106 11238 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:23.666759 11239 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:23.686163 11189 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:23.701853  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:23.702912  2760 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 27.
I20260709 14:55:23.704492  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:23.706106 11192 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:23.706671 11192 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.707230 11192 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:23.723656 11238 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:23.726900  2576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111) [suppressed 264 similar messages]
I20260709 14:55:23.751106 11192 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:23.752912  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.754812 11189 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.756198 11189 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.757197 11248 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.757902 11248 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.762290 11189 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:23.767575  2650 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.764797 11192 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:23.769479  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 112435us
W20260709 14:55:23.782811  2576 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.784664 11239 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.785481 11239 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:23.786008 11248 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.789039 11239 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:23.794129  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:23.799620  2650 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.802708  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:23.805560  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:23.806352  2677 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:55:23.809473 11248 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.810235 11248 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.810639  2651 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.811614 11245 raft_consensus.cc:2749] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:23.812749 11248 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.818535  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:23.819027  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.823413 11238 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:23.819360  2677 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 22.
I20260709 14:55:23.823853 11238 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.824303 11238 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:23.822615  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 61217.4us
I20260709 14:55:23.827509  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff9fe3d5f686477aa2bc32f628dcfb08" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:23.829074  2755 raft_consensus.cc:3060] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:23.838418  2755 raft_consensus.cc:2468] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 28.
I20260709 14:55:23.827000  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:23.842171  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:23.842736 11192 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:23.843698  2609 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:23.844252 11192 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.844655 11192 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
W20260709 14:55:23.843089  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.843519 11231 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.846815 11231 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:23.848487  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:23.853708 11247 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.854461 11247 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.856850 11247 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:23.856984  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.857637 11231 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:23.860749  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:23.840468  2577 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.861610  2677 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
W20260709 14:55:23.861548  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:23.862802  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.864058  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.865080 11241 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:23.865538 11241 raft_consensus.cc:493] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.865906 11189 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:23.865934 11241 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:23.866557 11189 raft_consensus.cc:493] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.867136 11189 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:23.869186  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:23.873978 11238 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.902798 11189 raft_consensus.cc:515] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.862538 11192 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.882730 11252 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.880867 11250 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:23.891245  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 51168.3us
I20260709 14:55:23.881542 11249 raft_consensus.cc:2804] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 14:55:23.885423  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:23.907648 11252 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:23.913792 11252 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:23.907941 11250 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.906793  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb3e91a07fc4383bd55f034f23c3cea" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:23.908478 11249 raft_consensus.cc:697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 28 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:23.916741  2612 raft_consensus.cc:3060] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:23.917554  2576 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:23.901716  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.918850  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.920238 11192 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.921411  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c902c27b99b4cc0a5186bc8117246a7" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:23.922281  2677 raft_consensus.cc:3060] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:23.924774 11247 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.925366 11247 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.928948 11247 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:23.929201  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:23.930056 11249 consensus_queue.cc:237] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:23.936321  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:23.936547  2612 raft_consensus.cc:2468] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:55:23.945580 11250 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.947537  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:23.950251 11247 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:23.943413  2677 raft_consensus.cc:2468] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
W20260709 14:55:23.955695  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:23.956542  2576 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.957067  2576 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.957680 11247 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.959209 11247 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.958148  2575 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:23.958356 11250 raft_consensus.cc:2749] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:23.960578 11192 raft_consensus.cc:2804] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 14:55:23.960865  2576 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.961429 11192 raft_consensus.cc:697] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:23.962733 11192 consensus_queue.cc:237] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:23.964289 11238 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:23.972007  2609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:23.972987  2609 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
I20260709 14:55:23.907343 11189 leader_election.cc:290] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:23.978920  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62710.8us
I20260709 14:55:23.979388  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:23.979931 11247 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:23.980726 11247 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 14:55:23.981153 11247 raft_consensus.cc:493] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:23.981602 11247 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:23.984668  2652 leader_election.cc:304] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:23.985864 11241 raft_consensus.cc:515] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:23.990723  2650 leader_election.cc:336] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:23.992722  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:23.992684  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a594c31352514fb685540e1082f8ee7d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:23.994186  2678 raft_consensus.cc:3060] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:23.992599  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01d4b3ec58f4ca3b994c5e66179174a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:23.994920  2677 raft_consensus.cc:3060] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:23.993460  2650 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:23.996193  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:23.996974  2755 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:23.991910 11245 raft_consensus.cc:2804] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:24.011088  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.013010 11238 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 14:55:24.013541 11238 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.013952 11238 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:24.021097 11239 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 14:55:24.022074 11245 raft_consensus.cc:697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
W20260709 14:55:24.029258  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:24.037397 11245 consensus_queue.cc:237] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:24.040127  2650 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.061856  2677 raft_consensus.cc:2468] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 23.
I20260709 14:55:24.064873  2678 raft_consensus.cc:2468] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
I20260709 14:55:24.066614  2723 leader_election.cc:304] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.066938 11241 leader_election.cc:290] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.067699 11241 raft_consensus.cc:2804] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:24.068204 11241 raft_consensus.cc:697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:24.069231 11241 consensus_queue.cc:237] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.073684  2575 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.085664 11249 raft_consensus.cc:2804] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:55:24.086324 11249 raft_consensus.cc:697] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:24.087409 11249 consensus_queue.cc:237] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.097982 11192 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.098889 11192 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.105258 11192 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.099730  2432 catalog_manager.cc:5697] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 27 to 28, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 28 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:24.107493  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:24.108273  2724 leader_election.cc:336] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.116384  2576 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.116997  2576 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.072611 11238 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.129973 11253 raft_consensus.cc:2749] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:24.130412 11254 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.131424 11231 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.149416 11238 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.150586  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e6d6e528724dcba0bbaf89427c37a9" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:24.151331  2755 raft_consensus.cc:3060] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Advancing to term 21
I20260709 14:55:24.151151 11231 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.159528 11231 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.161296 11189 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.162366 11189 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:24.168710  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.143384 11254 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.173072 11254 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.176733  2434 catalog_manager.cc:5697] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 21 to 22, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:24.186918 11257 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 14:55:24.189334  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:24.192077  2433 catalog_manager.cc:5697] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 26 to 27, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 27 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:24.195116 11192 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.195979 11192 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.196619 11249 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.193254 11189 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.197305 11249 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.204103 11247 raft_consensus.cc:515] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.210172 11253 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.211117 11253 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:24.201866  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.214275  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:24.205147 11249 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.208844  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:24.215745  2760 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 19.
I20260709 14:55:24.216472  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bae386d4ba412d86a21df33a9fa819" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:24.217527  2605 raft_consensus.cc:3060] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Advancing to term 23
I20260709 14:55:24.200739  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:24.220613 11192 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.220769  2755 raft_consensus.cc:2468] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 21.
I20260709 14:55:24.221735  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:24.224417 11192 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 14:55:24.224893 11192 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.225234 11192 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:24.226060  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.227144  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:24.227952  2677 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 30.
I20260709 14:55:24.219396 11253 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:24.229148  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.229652  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.229871  2612 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:24.231040 11249 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 14:55:24.231473 11249 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.231854 11249 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 30 FOLLOWER]: Advancing to term 31
I20260709 14:55:24.224025  2577 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:24.234797 11253 raft_consensus.cc:2804] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 14:55:24.235504 11253 raft_consensus.cc:697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:24.236873 11253 consensus_queue.cc:237] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.241844 11238 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.242712 11238 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:24.243201  2576 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.245287  2650 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.246011  2650 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.246912 11189 raft_consensus.cc:2749] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:24.248106  2650 leader_election.cc:336] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.245394 11238 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:24.249330  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:24.250115  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:24.249958 11192 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.251199  2604 raft_consensus.cc:2393] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 20: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
I20260709 14:55:24.254246 11241 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.255204 11192 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.257620  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c97f578e0e4d3f9c754fb31c0b4c6a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:24.258414  2754 raft_consensus.cc:3060] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Advancing to term 20
I20260709 14:55:24.259943  2605 raft_consensus.cc:2468] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:24.221719 11247 leader_election.cc:290] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.262920  2652 leader_election.cc:304] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.264106  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.264987 11189 raft_consensus.cc:2804] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 14:55:24.265443 11192 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.265564 11189 raft_consensus.cc:697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:24.265920 11192 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.266301 11192 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.266633 11189 consensus_queue.cc:237] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.269199 11254 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.269905 11254 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.255146 11241 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.275594 11192 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.277967  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.279227 11250 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.280047 11250 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.280707 11254 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.281876 11231 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.282263 11250 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.282358 11231 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.282970 11231 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.283201  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:24.277475 11241 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.284034  2678 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:24.283149  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:24.288424  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.287968  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.290184 11241 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.290656 11241 raft_consensus.cc:493] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.291646 11241 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:24.295804  2724 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.296463  2724 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.297529 11255 raft_consensus.cc:2749] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:24.300525  2755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:24.300627  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.301523  2754 raft_consensus.cc:2468] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 20.
I20260709 14:55:24.307013 11255 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 14:55:24.311529 11255 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.312332 11255 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.313531 11254 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.314213  2577 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:24.314296 11254 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.315760 11248 raft_consensus.cc:2804] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 14:55:24.316393 11248 raft_consensus.cc:697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:24.317500 11254 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.317395 11248 consensus_queue.cc:237] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.321825 11192 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:24.324425 11255 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.329876  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc2eae2c0540e4bc1c7141e81bf63c" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
W20260709 14:55:24.330188  2576 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.330776  2677 raft_consensus.cc:3060] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.330807  2576 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.335063 11241 raft_consensus.cc:515] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:24.336125  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.336903  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.339241  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c051e44d64d14770b36c5012be981196" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:24.340031  2678 raft_consensus.cc:3060] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:24.341053  2576 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.342002  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.342661  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.343811  2576 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.344636  2677 raft_consensus.cc:2468] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:24.345340 11238 raft_consensus.cc:2749] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:24.352167  2575 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.353449 11251 raft_consensus.cc:2804] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:24.354055 11251 raft_consensus.cc:697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:24.353654 11231 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.355134 11251 consensus_queue.cc:237] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:24.358385  2724 leader_election.cc:336] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.361984  2678 raft_consensus.cc:2468] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:24.365485  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.366379  2604 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:24.367655  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.368389  2605 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:24.350471 11241 leader_election.cc:290] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.372519 11249 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.381901 11249 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:24.363981  2723 leader_election.cc:304] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.378458 11231 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.382680  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100d7ba791034203ab40e147a55e2183" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:24.383539  2678 raft_consensus.cc:3060] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 14:55:24.388949 11255 raft_consensus.cc:2804] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:24.389715 11255 raft_consensus.cc:697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:24.390883 11255 consensus_queue.cc:237] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.392197 11254 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.392848 11254 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:24.378252 11264 log.cc:927] Time spent T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 14:55:24.404438  2678 raft_consensus.cc:2468] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 31.
I20260709 14:55:24.407482 11254 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:24.412822  2575 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:24.413313  2724 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.414045  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.415704 11252 raft_consensus.cc:2804] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 14:55:24.416370 11252 raft_consensus.cc:697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 31 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
W20260709 14:55:24.416535  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:24.417387 11252 consensus_queue.cc:237] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.419018  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:24.413669  2576 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.422904  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.406798 11231 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.424352  2576 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.426040 11254 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 14:55:24.430025  2576 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.430661 11252 raft_consensus.cc:2749] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:24.432057  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.433578  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:24.435026  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:24.435139  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:24.436021  2605 raft_consensus.cc:3060] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.439056 11254 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.447228 11241 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.447978 11241 raft_consensus.cc:493] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.448216  2605 raft_consensus.cc:2468] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:24.448589 11241 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:24.452011  2576 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.453747  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:24.453864  2760 raft_consensus.cc:1275] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 14:55:24.428504 11231 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.463972  2726 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.463243 11270 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.447644 11254 raft_consensus.cc:493] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.462785 11249 consensus_queue.cc:1048] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 14:55:24.492280 11255 raft_consensus.cc:2804] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:24.496073 11255 raft_consensus.cc:697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:55:24.495127  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.499820  2678 raft_consensus.cc:1275] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 14:55:24.501946 11248 consensus_queue.cc:1048] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 14:55:24.505312  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:24.506476 11189 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:24.507609  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:24.507092 11189 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:24.509054   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 3572)
I20260709 14:55:24.476291 11254 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.515027 11189 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.519966 11231 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.510933 11270 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.530325 11270 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.551003  2677 raft_consensus.cc:1275] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 14:55:24.552017  2650 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.554970  2605 raft_consensus.cc:1275] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 14:55:24.556243 11248 consensus_queue.cc:1048] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:24.563720  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:24.528299 11255 consensus_queue.cc:237] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.550397 11241 raft_consensus.cc:515] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.550506 11239 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.569651  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:24.573602  2760 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:24.580271  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.581246 11189 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.581727 11189 raft_consensus.cc:493] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.582149 11189 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:24.588953  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 116987us
I20260709 14:55:24.599733  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.600242 11241 leader_election.cc:290] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.600602  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 25.
I20260709 14:55:24.584115 11247 consensus_queue.cc:1048] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 14:55:24.606803  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:24.607981  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:24.612368 11189 raft_consensus.cc:515] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.617578  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:24.618161  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:24.620852  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b56ad927e054a56a77c811620dda543" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:24.621816  2612 raft_consensus.cc:3060] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.622406 11189 leader_election.cc:290] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.646220  2612 raft_consensus.cc:2468] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:24.648867  2726 leader_election.cc:304] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:24.584354 11239 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.632157 11245 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.674568 11189 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.675408 11189 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.632848 11250 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.685194 11189 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.687179  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
W20260709 14:55:24.692731  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 105632us
I20260709 14:55:24.712590  2678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:24.713681  2678 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:24.719573  2604 raft_consensus.cc:1275] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 14:55:24.722223 11189 consensus_queue.cc:1048] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 14:55:24.726341  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406cc4479d48441398435d7dc1850d64" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:24.727262  2754 raft_consensus.cc:3060] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.733280  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:24.683213 11245 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.746018 11277 raft_consensus.cc:2804] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:24.746902 11277 raft_consensus.cc:697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:24.747982 11277 consensus_queue.cc:237] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.752578 11276 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:24.753160 11276 raft_consensus.cc:493] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.753589 11276 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:24.755852 11239 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:24.766548  2754 raft_consensus.cc:2468] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 26.
W20260709 14:55:24.769071  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 14:55:24.688696 11250 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.724547 11252 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:24.779129  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:24.780074 11287 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:24.780891 11280 log.cc:927] Time spent T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 14:55:24.781481 11271 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:24.778649 11252 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:24.789848  2651 leader_election.cc:304] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:24.800932 11245 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:24.802254  2604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74" is_pre_election: true
I20260709 14:55:24.803747 11276 raft_consensus.cc:515] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.809691  2612 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
W20260709 14:55:24.818358  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 78345.4us
I20260709 14:55:24.805302 11254 raft_consensus.cc:515] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:24.848822  2604 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 27.
W20260709 14:55:24.879899  2724 leader_election.cc:336] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:24.881160  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:24.883785  2754 raft_consensus.cc:2393] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 27: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:24.895723 11288 raft_consensus.cc:2804] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:24.896672 11288 raft_consensus.cc:697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:24.897804 11288 consensus_queue.cc:237] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:24.839481 11271 raft_consensus.cc:493] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
W20260709 14:55:24.877405 11257 log.cc:927] Time spent T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 14:55:24.912220  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 263486us
I20260709 14:55:24.909225 11271 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:24.835301 11287 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:25.051146  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 76317.4us
I20260709 14:55:25.052620  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32f10706d04497089f6159669dc04a4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:25.053771  2677 raft_consensus.cc:3060] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:25.058255  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 72089.2us
I20260709 14:55:25.060905 11250 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:25.079015  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a8bd118baa7481399ce15d422463083" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:25.080503  2605 raft_consensus.cc:3060] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:25.062089 11254 leader_election.cc:290] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:25.092495 11287 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:25.095134  2650 leader_election.cc:336] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.096721  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.099646  2677 raft_consensus.cc:2468] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
I20260709 14:55:25.107208  2605 raft_consensus.cc:2468] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:25.075881 11276 leader_election.cc:290] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:25.120710 11300 log.cc:927] Time spent T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 14:55:25.123075 11299 log.cc:927] Time spent T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 14:55:25.123950  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.124701  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.125205  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:25.125648  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:25.113214  2726 leader_election.cc:304] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: 
I20260709 14:55:25.144557  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:25.144922  2679 raft_consensus.cc:1275] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 14:55:25.145478  2683 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 25.
I20260709 14:55:25.133052 11254 raft_consensus.cc:2804] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:25.150460 11254 raft_consensus.cc:697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:25.155315 11254 consensus_queue.cc:237] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:25.159284  2686 raft_consensus.cc:1275] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 14:55:25.181370 11252 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:25.182646  2432 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50322 (request call id 169) took 1090 ms. Trace:
I20260709 14:55:25.152153 11271 raft_consensus.cc:515] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:25.186308  2760 raft_consensus.cc:1275] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 14:55:25.198207  2754 raft_consensus.cc:1275] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 14:55:25.204387  2723 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:25.183195  2432 rpcz_store.cc:276] 0709 14:55:24.092403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 14:55:24.092588 (+   185us) service_pool.cc:224] Handling call
0709 14:55:25.182581 (+1089993us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 14:55:24.230260 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 14:55:24.234955 (+  4695us) write_op.cc:432] Acquiring schema lock in shared mode
0709 14:55:24.234995 (+    40us) write_op.cc:435] Acquired schema lock
0709 14:55:24.235020 (+    25us) tablet.cc:662] Decoding operations
0709 14:55:24.235462 (+   442us) write_op.cc:620] Acquiring the partition lock for write op
0709 14:55:24.235549 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 14:55:24.235584 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 14:55:24.235824 (+   240us) tablet.cc:701] Row locks acquired
0709 14:55:24.235851 (+    27us) write_op.cc:260] PREPARE: finished
0709 14:55:24.236024 (+   173us) write_op.cc:270] Start()
0709 14:55:24.236164 (+   140us) write_op.cc:276] Timestamp: P: 1783608924235971 usec, L: 0
0709 14:55:24.236186 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 14:55:24.237011 (+   825us) log.cc:844] Serialized 540 byte log entry
0709 14:55:24.455458 (+218447us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5400110420. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa16e5af571 00007fa16e5c4343 00007fa16e5c7d75 00007fa16e5c7d09 00007fa16e5c7c99 00007fa16e5c7c61 00007fa16e5c6f5d 0000000000396ad4 0000000000396919 00007fa16b559475 00007fa16b55b7c1
0709 14:55:24.455733 (+   275us) op_driver.cc:464] REPLICATION: finished
0709 14:55:24.587573 (+131840us) write_op.cc:301] APPLY: starting
0709 14:55:24.590521 (+  2948us) tablet.cc:1367] starting BulkCheckPresence
0709 14:55:24.632169 (+ 41648us) tablet.cc:1370] finished BulkCheckPresence
0709 14:55:24.632202 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 14:55:25.116208 (+484006us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b18001fc060. stack: 00007fa16b527343 00007fa16b527240 00007fa16aeb61d4 00007fa174bf4304 00007fa174bf4299 00007fa174421de1 00007fa17442e5b1 00007fa16e3850ec 00007fa16e3d1c0a 00007fa16e3d14f7 00007fa16e3d1490 00007fa16e388f48 00007fa17209d1bb 00007fa1720e45d4 00007fa171f908da 00007fa171f933cb
0709 14:55:25.117173 (+   965us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 14:55:25.117431 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 14:55:25.117492 (+    61us) write_op.cc:312] APPLY: finished
0709 14:55:25.117766 (+   274us) log.cc:844] Serialized 24 byte log entry
0709 14:55:25.137267 (+ 19501us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:25.142513 (+  5246us) write_op.cc:454] Released schema lock
0709 14:55:25.142564 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 14:55:25.143152 (+   588us) write_op.cc:489] Releasing partition, row and schema locks
0709 14:55:25.143182 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38630,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":107914,"apply.run_cpu_time_us":10666,"apply.run_wall_time_us":566628,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":106047,"prepare.run_cpu_time_us":11533,"prepare.run_wall_time_us":90920,"raft.queue_time_us":336280,"raft.run_cpu_time_us":57542,"raft.run_wall_time_us":202369,"replication_time_us":219410,"spinlock_wait_cycles":21949952,"thread_start_us":316366,"threads_started":4,"wal-append.queue_time_us":87979}]]}
I20260709 14:55:25.211891 11192 consensus_queue.cc:1048] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 14:55:25.212954 11277 raft_consensus.cc:2804] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:25.215463 11275 consensus_queue.cc:1048] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:25.221258 11277 raft_consensus.cc:697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:25.227929  2685 raft_consensus.cc:1275] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 14:55:25.244706  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.246233  2650 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.245591  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.247524  2576 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.250017 11271 leader_election.cc:290] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:25.252003  2724 leader_election.cc:336] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.253265  2724 leader_election.cc:336] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.258599 11289 consensus_queue.cc:1048] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 14:55:25.264340  2759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:25.265260  2759 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 28.
I20260709 14:55:25.265406 11277 consensus_queue.cc:237] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:25.246789  2650 leader_election.cc:304] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:25.282330  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.283313  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.282387  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:25.290275 11245 raft_consensus.cc:2749] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 14:55:25.294170  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.294945  2724 leader_election.cc:336] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.298753  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.302201 11189 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:25.302666 11189 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:25.302752  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1073216da14f9c91bcc3f31f935620" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:25.303105 11189 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:25.303386  2605 raft_consensus.cc:3060] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
I20260709 14:55:25.304108  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.304415  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.297689  2433 catalog_manager.cc:5697] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 22 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:25.309779  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 14:55:25.319563  2679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:25.325344  2683 raft_consensus.cc:1275] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:55:25.300015 11252 consensus_queue.cc:1048] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 14:55:25.336586 11317 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 14:55:25.337393 11317 raft_consensus.cc:493] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:25.337843 11317 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 14:55:25.341849  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:25.342672  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 14:55:25.340099 11290 consensus_queue.cc:1048] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 14:55:25.313793  2433 catalog_manager.cc:5697] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 22 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:25.357070  2605 raft_consensus.cc:2468] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:25.359045  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.364917  2433 catalog_manager.cc:5697] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:25.367096  2433 catalog_manager.cc:5697] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 22 to 23, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } }
I20260709 14:55:25.372749 11189 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:25.379556 11189 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:25.376946  2726 leader_election.cc:304] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.381400  2605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:25.384899  2604 raft_consensus.cc:1275] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 14:55:25.387421 11270 raft_consensus.cc:2804] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 14:55:25.388389  2609 raft_consensus.cc:1275] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 14:55:25.395742 11305 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 14:55:25.396476 11305 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:25.396888 11305 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Advancing to term 26
W20260709 14:55:25.397770  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 48551us
I20260709 14:55:25.372361 11239 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 14:55:25.413695 11239 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader c1b0e692e4294ee3b8694bbada174ba1)
I20260709 14:55:25.414410 11239 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:25.425122 11270 raft_consensus.cc:697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:25.426563 11270 consensus_queue.cc:237] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:25.442270 11313 consensus_queue.cc:1048] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
W20260709 14:55:25.445350 11319 log.cc:927] Time spent T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 14:55:25.427147 11254 consensus_queue.cc:1048] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 14:55:25.441453  2434 catalog_manager.cc:5697] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:25.460736  2434 catalog_manager.cc:5697] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:25.462810  2434 catalog_manager.cc:5697] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:25.464773  2434 catalog_manager.cc:5697] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:25.467814 11318 log.cc:927] Time spent T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: Append to log took a long time: real 0.071s	user 0.000s	sys 0.003s
W20260709 14:55:25.473472  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.477845  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.478538  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 89531.9us
W20260709 14:55:25.479084  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:25.485098  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:25.499017 11254 consensus_queue.cc:1048] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 14:55:25.508764  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.511394  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:25.515089   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:25.519505 11323 log.cc:927] Time spent T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: Append to log took a long time: real 0.092s	user 0.003s	sys 0.000s
I20260709 14:55:25.548964 11305 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:25.550532  2605 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 26: Already voted for candidate 56620e612f0549e6abf320aa80398e74 in this term.
W20260709 14:55:25.560304  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:25.561894  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.565852 11306 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:55:25.579231 11317 raft_consensus.cc:515] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:25.582093  2679 raft_consensus.cc:2393] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 28: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:25.596724  2612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e4166c53c7459b808e9e6841d90c56" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "56620e612f0549e6abf320aa80398e74"
I20260709 14:55:25.597751  2612 raft_consensus.cc:3060] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Advancing to term 28
I20260709 14:55:25.598349  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:25.607414  2684 raft_consensus.cc:2393] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 56620e612f0549e6abf320aa80398e74 in current term 26: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:25.610294  2575 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.597011 11305 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:25.615962  2753 raft_consensus.cc:1275] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 14:55:25.585510 11317 leader_election.cc:290] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:25.629376  2612 raft_consensus.cc:2468] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 28.
I20260709 14:55:25.631531  2652 leader_election.cc:304] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:25.636175 11284 consensus_queue.cc:1048] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 14:55:25.637296 11322 raft_consensus.cc:2804] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 14:55:25.638019 11322 raft_consensus.cc:697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:25.632229 11253 raft_consensus.cc:2749] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:25.607210 11239 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:25.666630  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.671622  2576 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:25.675179 11297 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 14:55:25.671245 11322 consensus_queue.cc:237] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:25.692905  2757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb30692225e4daebaa7e02e9cb291b7" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:25.693715  2757 raft_consensus.cc:3060] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:25.700564 11239 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 election: Requested vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:25.717599  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:25.731619  2612 raft_consensus.cc:1275] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 14:55:25.746152  2650 leader_election.cc:336] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:25.749161  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:25.751688  2757 raft_consensus.cc:2468] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 29.
I20260709 14:55:25.758428  2651 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:25.759439 11322 raft_consensus.cc:2804] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 14:55:25.760054 11322 raft_consensus.cc:697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:25.761119 11322 consensus_queue.cc:237] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:25.766852 11241 consensus_queue.cc:1048] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 14:55:25.789270  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:25.854415  2685 raft_consensus.cc:1275] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 14:55:25.858806 11294 consensus_queue.cc:1048] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 14:55:25.870149  2650 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:25.880421  2604 raft_consensus.cc:1275] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 14:55:25.883019  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:25.890672 11294 consensus_queue.cc:1048] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
W20260709 14:55:25.939044  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 14:55:25.992080  2432 catalog_manager.cc:5697] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 20 to 21, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:25.994031  2432 catalog_manager.cc:5697] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:25.995810  2432 catalog_manager.cc:5697] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 19 to 20, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:26.000991  2432 catalog_manager.cc:5697] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 30 to 31, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 31 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:26.026513  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:26.032627  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:26.041183  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:26.069666  2433 catalog_manager.cc:5697] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 26 to 27, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 27 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:26.072091  2433 catalog_manager.cc:5697] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 25 to 26, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 26 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:26.121658  2434 catalog_manager.cc:5697] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 27 to 28, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 28 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:26.124228  2434 catalog_manager.cc:5697] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 28 to 29, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 29 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:26.143914  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 14:55:26.178135  2612 raft_consensus.cc:1275] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 14:55:26.181648 11287 consensus_queue.cc:1048] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:26.187897  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 14:55:26.242440  2759 raft_consensus.cc:1275] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 14:55:26.253686  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:26.259618 11287 consensus_queue.cc:1048] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 14:55:26.292426  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:26.466041  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:26.515723   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:26.530478  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:26.681640  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:26.716225  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:26.849661  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:26.878901  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:27.038139  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.042541  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.065424  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:27.107321  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:27.109086  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 55940.3us
W20260709 14:55:27.124290  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:27.150738  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:27.177807  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 62983us
W20260709 14:55:27.185935  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:27.216143  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.217181  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:27.337504  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.388855  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:27.428637  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:27.449313  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.454568  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:27.516669   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 3572)
I20260709 14:55:27.550628 11308 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:27.551971 11308 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:27.572597  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:27.573546  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:27.575245  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:27.576625  2685 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 26.
I20260709 14:55:27.578809  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:27.581314 11350 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:27.582191 11350 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:27.582720 11350 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:27.583218 11308 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:27.593508  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 14:55:27.638139 11350 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:27.641043 11350 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Requested vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:27.645704  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "963a4155717b4274b0121341056b7cb8" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:27.648782  2685 raft_consensus.cc:3060] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:27.650259  2576 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:27.657881  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.662155  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:27.666113  2685 raft_consensus.cc:2468] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56620e612f0549e6abf320aa80398e74 in term 27.
I20260709 14:55:27.682415  2575 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:27.684163 11308 raft_consensus.cc:2804] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:27.691177 11308 raft_consensus.cc:697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 27 LEADER]: Becoming Leader. State: Replica: 56620e612f0549e6abf320aa80398e74, State: Running, Role: LEADER
I20260709 14:55:27.692606 11308 consensus_queue.cc:237] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:27.696162  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:27.720294  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 14:55:27.756994  2432 catalog_manager.cc:5697] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 reported cstate change: term changed from 25 to 27, leader changed from c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193) to 56620e612f0549e6abf320aa80398e74 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "56620e612f0549e6abf320aa80398e74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:27.778466  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.798964  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.926299  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:27.999339  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:28.049178  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:28.109193  2677 raft_consensus.cc:1275] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 56620e612f0549e6abf320aa80398e74: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 14:55:28.111855 11308 consensus_queue.cc:1048] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:28.185557  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:28.304018  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:28.345183  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:28.379405  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:28.392088  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:28.431378  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:28.461215  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:28.517287  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:28.517962   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:28.741442  2724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111) [suppressed 432 similar messages]
W20260709 14:55:28.776125  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:28.819032  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:28.819000  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:28.978438  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:28.999212  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:29.052044  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:29.085408  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:29.111816  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:29.192003  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:29.299479  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.307087  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:29.359757  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 109277us
W20260709 14:55:29.406351  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:29.440536  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.510686  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:29.518688   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:29.527386  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:29.647149  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:29.647918  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.694279  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.846149  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:29.878646  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:29.904487  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.906327  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:29.934893  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:29.970818  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:29.983588  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:30.122337  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:30.135833  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:30.136475  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.180769  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.226904  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:30.235060  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.277058  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.466146  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:30.513372  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.519296   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:30.601153  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.601756  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 47086.3us
W20260709 14:55:30.693384  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.714079  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.714893  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:30.734745  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.786623  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.825480  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:30.857966  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:30.986979  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:31.207116  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:31.251411  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:31.308290  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:31.336146  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:31.419999  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:31.472054  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:31.504652  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:31.525188  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:31.531292  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:31.532275   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:31.554960  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:31.600561  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:31.772562  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:31.910673  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:31.968142  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.087320  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.114656  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:32.131722  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.189764  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:32.189124  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.286008  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:32.333082  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:32.535248   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:32.547009  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.581771  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:32.600970  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.620260  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44827.2us
W20260709 14:55:32.642055  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 60342.5us
W20260709 14:55:32.643041  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:32.643748  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:32.644284  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:32.646170  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:32.650885  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.677943  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:32.701284  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.744055  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:32.746932  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.765873  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:32.779842  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:32.832028  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.153505  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:33.265616  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:33.270671  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.400490  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.439663  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:33.461781  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.469111  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:33.535792   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:33.547112  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.565793  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08002d3ec0 after lost signal to thread 434
W20260709 14:55:33.567716  2781 debug-util.cc:398] Leaking SignalData structure 0x7b0800125c20 after lost signal to thread 2469
W20260709 14:55:33.569423  2781 debug-util.cc:398] Leaking SignalData structure 0x7b0800242280 after lost signal to thread 2636
W20260709 14:55:33.576210  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.606261  2781 debug-util.cc:398] Leaking SignalData structure 0x7b080031c180 after lost signal to thread 2710
W20260709 14:55:33.608207  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7860 after lost signal to thread 2784
W20260709 14:55:33.636613  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:33.734217  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:33.746904  2650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111) [suppressed 447 similar messages]
W20260709 14:55:33.819255  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:33.956517  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:34.002642  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.105512  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.172705  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:34.214696  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:34.212327  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:34.211648  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.287156  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.317313  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:34.366937  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.465236  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.503809  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.536500   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:34.677253  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.689473  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.748026  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:34.765388  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:34.775988  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:34.799049  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.799845  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:34.823444  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.871657  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:34.905539  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.912040  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.946471  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:34.981741  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:34.999934  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:35.006623  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:35.090378  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:35.204573  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:35.254578  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:35.286597  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:35.537146   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:35.581224  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 223737us
W20260709 14:55:35.639045  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:35.639863  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:35.689770  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:35.716670  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:35.782173  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.020780  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:36.029747  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.043138  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.109706  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.125737  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 14:55:36.137043  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:36.176756  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.332816  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.397959  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:36.403925  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:36.421281  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:36.479219  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:36.500852  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:36.537802   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:36.634048  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:36.677702  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:36.859220  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:36.910735  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:36.932353  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:36.944962  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 44269.2us
W20260709 14:55:36.958077  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:36.996165  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.123217  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.164553  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:37.166469  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.325305  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:37.396690  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:37.467813  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:37.486624  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:37.478554  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 59684.6us
W20260709 14:55:37.507529  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.538695   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:37.557086  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:37.566159  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:37.569721  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.572046  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.572953  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:37.573518  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:37.611441  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:37.612628  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.616349  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.675921  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:37.865098  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:37.919039  2575 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 109198us
W20260709 14:55:37.983970  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:38.017073  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:38.038460  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:38.085887  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:38.103129  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:38.104238  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:38.472257  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:38.512768  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:38.520052  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:38.532407  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:38.540594   431 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:38.623346  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:38.750699  2724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111) [suppressed 437 similar messages]
W20260709 14:55:38.770709  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:38.836580  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 14:55:38.915959  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 50594.8us
W20260709 14:55:38.964938  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:39.128705  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:39.146750  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:39.154266  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:39.164480  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:39.165556  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:39.164783  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.390651  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.411932  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:39.470150  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.497905  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:39.534373  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:39.541230   431 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:39.578084  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:39.630509  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.657166  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:39.691850  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:39.912194  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:39.916172  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.942802  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:39.943608  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.056651  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:40.075827  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.076745  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.077615  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:40.085748  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:40.116643  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:40.130388  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:40.153123  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.160248  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:40.191620  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:40.192337  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:40.468919  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:40.541921   431 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:40.559618  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:40.574899  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.577898  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 64361.1us
W20260709 14:55:40.616744  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:40.732843  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:40.784253  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08002d3380 after lost signal to thread 434
W20260709 14:55:40.785976  2633 debug-util.cc:398] Leaking SignalData structure 0x7b080004ff20 after lost signal to thread 2469
W20260709 14:55:40.787787  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800358980 after lost signal to thread 2636
W20260709 14:55:40.789494  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08001749e0 after lost signal to thread 2710
W20260709 14:55:40.791970  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce1a0 after lost signal to thread 2784
W20260709 14:55:40.863600  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:40.864300  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:41.065831  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:41.175877  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:41.316821  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:41.391326  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:41.422364  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:41.462809  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:41.518941  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:41.542526   431 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:41.618674  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:41.655126  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:41.694660  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:41.698935  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 14:55:41.779713  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.367372  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 53043.1us
W20260709 14:55:42.381637  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:42.414634  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:42.431757  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:42.437371  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.438539  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:42.442657  2577 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 58322.9us
W20260709 14:55:42.496800  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:42.511451  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:42.512343  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.513048  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:42.513691  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:42.529742  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:42.543823   431 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:42.548554  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.804472  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:42.863715  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:42.865751  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:42.889473  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.904963  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.902539  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:42.906874  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.907753  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:42.946421  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:42.994477  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:43.261264  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3900 after lost signal to thread 434
W20260709 14:55:43.263039  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08002c3980 after lost signal to thread 2469
W20260709 14:55:43.264905  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08001989e0 after lost signal to thread 2636
W20260709 14:55:43.266891  2781 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1dc0 after lost signal to thread 2710
W20260709 14:55:43.268603  2781 debug-util.cc:398] Leaking SignalData structure 0x7b080013e280 after lost signal to thread 2784
W20260709 14:55:43.332419  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:43.333280  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:43.357375  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:43.422176  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 14:55:43.431838  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:43.435523  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:43.444808  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:43.462149  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:43.520700  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 58619.2us
W20260709 14:55:43.521319  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:43.546654   431 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:43.596263  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 112890us
W20260709 14:55:43.640789  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08001aefe0 after lost signal to thread 434
W20260709 14:55:43.642442  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08003def60 after lost signal to thread 2469
W20260709 14:55:43.644269  2466 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2ae0 after lost signal to thread 2636
W20260709 14:55:43.647265  2466 debug-util.cc:398] Leaking SignalData structure 0x7b0800336b40 after lost signal to thread 2710
W20260709 14:55:43.649117  2466 debug-util.cc:398] Leaking SignalData structure 0x7b080023da60 after lost signal to thread 2784
W20260709 14:55:43.759639  2576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111) [suppressed 394 similar messages]
W20260709 14:55:43.866914  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:43.886435  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:43.933188  2650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 74113us
W20260709 14:55:44.054549  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.068574  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:44.236617  2576 consensus_peers.cc:597] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:44.339879  2576 consensus_peers.cc:597] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 14:55:44.411345  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.435756  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.436601  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:44.442358  2651 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:46277, user_credentials={real_user=slave}} blocked reactor thread for 98371.6us
W20260709 14:55:44.451532  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 80432us
W20260709 14:55:44.458029  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:44.508383  2576 consensus_peers.cc:597] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:44.547353   431 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:44.576411  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:44.600088  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.585062  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.632375  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08003386a0 after lost signal to thread 434
W20260709 14:55:44.634255  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800275f20 after lost signal to thread 2469
W20260709 14:55:44.636441  2633 debug-util.cc:398] Leaking SignalData structure 0x7b08001e0980 after lost signal to thread 2636
W20260709 14:55:44.638310  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800201260 after lost signal to thread 2710
W20260709 14:55:44.640245  2633 debug-util.cc:398] Leaking SignalData structure 0x7b0800337720 after lost signal to thread 2784
W20260709 14:55:44.680982  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:44.694798  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:44.706262  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:44.863729  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:44.879586  2576 consensus_peers.cc:597] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:45.000713  2576 consensus_peers.cc:597] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:45.021401  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 14:55:45.036763  2576 consensus_peers.cc:597] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.087405  2576 consensus_peers.cc:597] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:45.116384  2576 consensus_peers.cc:597] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:45.145462  2576 consensus_peers.cc:597] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:45.158347  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:45.188295  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.256968  2576 consensus_peers.cc:597] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.290927  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:45.301658  2576 consensus_peers.cc:597] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.323402  2576 consensus_peers.cc:597] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:45.328328  2576 consensus_peers.cc:597] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.432420  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.485021  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:45.486347  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:45.491326  2576 consensus_peers.cc:597] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.496232  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:45.548038   431 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 3572)
W20260709 14:55:45.565296  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:45.773674  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:45.788329  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:45.842221  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:45.916792  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:45.934218  2576 consensus_peers.cc:597] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 14:55:45.999730  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:46.097121  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:46.138908   431 tablet_server.cc:196] TabletServer@127.0.107.193:37831 shutdown complete.
W20260709 14:55:46.315609  2576 consensus_peers.cc:597] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:46.343477   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 14:55:46.396682  2652 consensus_peers.cc:597] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:46.409514  2726 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:46.417045  2652 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:46.429189  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:46.442149  2652 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.448081   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:55:46.449851   431 tablet_replica.cc:333] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.450989   431 raft_consensus.cc:2243] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.453011   431 raft_consensus.cc:2272] T dd47a5b34fb04ee887e263844b637438 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.456625   431 tablet_replica.cc:333] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.457828   431 raft_consensus.cc:2243] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.459188   431 raft_consensus.cc:2272] T f231033e78bf42ad8d6dbdf1f54d21be P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.461838   431 tablet_replica.cc:333] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.462592   431 raft_consensus.cc:2243] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.464003   431 raft_consensus.cc:2272] T 947b4a5caeb243ef87d3a535eddda080 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.467424   431 tablet_replica.cc:333] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.468261   431 raft_consensus.cc:2243] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.469583   431 raft_consensus.cc:2272] T f01d4b3ec58f4ca3b994c5e66179174a P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.473388   431 tablet_replica.cc:333] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.474416   431 raft_consensus.cc:2243] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.475801   431 raft_consensus.cc:2272] T 75e6d6e528724dcba0bbaf89427c37a9 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.478590   431 tablet_replica.cc:333] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.479418   431 raft_consensus.cc:2243] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.480621   431 raft_consensus.cc:2272] T 7abc2eae2c0540e4bc1c7141e81bf63c P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.483491   431 tablet_replica.cc:333] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.484543   431 raft_consensus.cc:2243] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.486233   431 raft_consensus.cc:2272] T 1c902c27b99b4cc0a5186bc8117246a7 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.489547   431 tablet_replica.cc:333] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.490386   431 raft_consensus.cc:2243] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.491135   431 raft_consensus.cc:2272] T 6340ce5f0bc04ded8681c8573a0910c1 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.494026   431 tablet_replica.cc:333] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.495222   431 raft_consensus.cc:2243] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.496258   431 raft_consensus.cc:2272] T e5f95289d39644bead88ff8640742871 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.499680   431 tablet_replica.cc:333] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.500893   431 raft_consensus.cc:2243] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Raft consensus shutting down.
W20260709 14:55:46.501317  2652 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.502343   431 raft_consensus.cc:2272] T 0d27004b985a45779b0f86fcb7273dbc P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:46.506052  2726 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.506219   431 tablet_replica.cc:333] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.507407   431 raft_consensus.cc:2243] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.509215   431 raft_consensus.cc:2272] T 963a4155717b4274b0121341056b7cb8 P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.512434   431 tablet_replica.cc:333] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.513514   431 raft_consensus.cc:2243] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.514302   431 raft_consensus.cc:2272] T 194ae28527c14d78817238308fb284c6 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.517939   431 tablet_replica.cc:333] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.519012   431 raft_consensus.cc:2243] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.519963   431 raft_consensus.cc:2272] T 31e4166c53c7459b808e9e6841d90c56 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.523047   431 tablet_replica.cc:333] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.524056   431 raft_consensus.cc:2243] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 24 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.525503   431 raft_consensus.cc:2272] T 92641606b7ec403d843b2ee872402de3 P 56620e612f0549e6abf320aa80398e74 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.529054   431 tablet_replica.cc:333] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.530038   431 raft_consensus.cc:2243] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Raft consensus shutting down.
W20260709 14:55:46.530093  2576 consensus_peers.cc:597] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 14:55:46.531553   431 raft_consensus.cc:2272] T c39462fb47b74715a7e5498d49eaab87 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:46.534047  2726 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.535472   431 tablet_replica.cc:333] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.536325   431 raft_consensus.cc:2243] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.537525   431 raft_consensus.cc:2272] T 1b2b04c469b2404c908d67b076cf2ba4 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.540709   431 tablet_replica.cc:333] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.541510   431 raft_consensus.cc:2243] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.542701   431 raft_consensus.cc:2272] T a57a97528f31400dbc0900e76292b021 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.545583   431 tablet_replica.cc:333] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.546423   431 raft_consensus.cc:2243] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.547686   431 raft_consensus.cc:2272] T a2e7eae198c34a6fafb697d50f269192 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.551376   431 tablet_replica.cc:333] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.552430   431 raft_consensus.cc:2243] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.554033   431 raft_consensus.cc:2272] T d87a2c3e021741a28db500c36ebdc494 P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.557653   431 tablet_replica.cc:333] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.558502   431 raft_consensus.cc:2243] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.559548   431 raft_consensus.cc:2272] T eeb3e91a07fc4383bd55f034f23c3cea P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:46.561371  2726 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.563197   431 tablet_replica.cc:333] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.564363   431 raft_consensus.cc:2243] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.566313   431 raft_consensus.cc:2272] T 80e0e0c146c54411901808c4bbf22280 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.570575   431 tablet_replica.cc:333] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.571856   431 raft_consensus.cc:2243] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.573238   431 raft_consensus.cc:2272] T 4837b30553bb4eac8840db77c7ffca53 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.577786   431 tablet_replica.cc:333] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.578989   431 raft_consensus.cc:2243] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.579872   431 raft_consensus.cc:2272] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.583575   431 tablet_replica.cc:333] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.584583   431 raft_consensus.cc:2243] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.585358   431 raft_consensus.cc:2272] T 9a8bd118baa7481399ce15d422463083 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.588544   431 tablet_replica.cc:333] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.589619   431 raft_consensus.cc:2243] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.590503   431 raft_consensus.cc:2272] T 8b56ad927e054a56a77c811620dda543 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.593758   431 tablet_replica.cc:333] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.594630   431 raft_consensus.cc:2243] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.595474   431 raft_consensus.cc:2272] T a1826b730f9f4ae4b0e05570574563c3 P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.598784   431 tablet_replica.cc:333] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.599716   431 raft_consensus.cc:2243] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.600482   431 raft_consensus.cc:2272] T 7ea5acf8ab374c8fa3009782429fd3ed P 56620e612f0549e6abf320aa80398e74 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:46.600564  2726 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.604079   431 tablet_replica.cc:333] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.605085   431 raft_consensus.cc:2243] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.606695   431 raft_consensus.cc:2272] T 4ac95ecab0524f35a894d205dac97a44 P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:46.608717  2726 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:46.610203   431 tablet_replica.cc:333] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.611276   431 raft_consensus.cc:2243] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.612166   431 raft_consensus.cc:2272] T 8fb844e4a36641d79afdab73422e06f1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.615733   431 tablet_replica.cc:333] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.616604   431 raft_consensus.cc:2243] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.617904   431 raft_consensus.cc:2272] T da98c08aee5d4d86bce741e08142c1e3 P 56620e612f0549e6abf320aa80398e74 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.620836   431 tablet_replica.cc:333] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.621510   431 raft_consensus.cc:2243] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.622150   431 raft_consensus.cc:2272] T 1f87ae033ca94d58bdc3a087236657b7 P 56620e612f0549e6abf320aa80398e74 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.624984   431 tablet_replica.cc:333] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.625880   431 raft_consensus.cc:2243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 LEADER]: Raft consensus shutting down.
I20260709 14:55:46.627650   431 raft_consensus.cc:2272] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 56620e612f0549e6abf320aa80398e74 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:46.630990   431 tablet_replica.cc:333] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:46.631801   431 raft_consensus.cc:2243] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:46.632807   431 raft_consensus.cc:2272] T 5e1073216da14f9c91bcc3f31f935620 P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:47.301095  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 45910us
W20260709 14:55:47.294545  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 331380us
W20260709 14:55:47.327894  2726 consensus_peers.cc:597] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:47.341836  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 14:55:47.390460   431 tablet_replica.cc:333] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.402169   431 raft_consensus.cc:2243] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 28 LEADER]: Raft consensus shutting down.
W20260709 14:55:47.411782  2652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 74462.1us
W20260709 14:55:47.421062  2724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 78568.7us
I20260709 14:55:47.432018   431 raft_consensus.cc:2272] T 498c3de2b5414c968242f7f8fc2ca7a1 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:47.432973  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:47.436910 11446 raft_consensus.cc:493] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:47.438165 11446 raft_consensus.cc:515] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
W20260709 14:55:47.440157  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:47.441880  2726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40091, user_credentials={real_user=slave}} blocked reactor thread for 108463us
I20260709 14:55:47.449225  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7584380efde04ee289b42acaea2e4742" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.452919 11480 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.454161 11480 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.462216 11480 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
W20260709 14:55:47.462451  2723 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:41829, user_credentials={real_user=slave}} blocked reactor thread for 122820us
I20260709 14:55:47.465356  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:47.465996 11446 leader_election.cc:290] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:47.471563  2576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:37831, user_credentials={real_user=slave}} blocked reactor thread for 56094.3us
W20260709 14:55:47.485100  2652 consensus_peers.cc:597] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:47.486490 11475 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.487434 11446 raft_consensus.cc:493] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:47.492412 11446 raft_consensus.cc:515] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.487615 11475 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:47.500344   431 tablet_replica.cc:333] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
W20260709 14:55:47.502296  2576 leader_election.cc:336] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.503072  2576 leader_election.cc:304] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.503257  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.508812 11457 raft_consensus.cc:2749] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:47.506337  2652 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:47.511471  2652 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:47.507323 11475 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.516512 11446 leader_election.cc:290] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:47.517221  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1591def49c3c47edac03ade007ebc91a" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:47.520236  2577 leader_election.cc:304] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:47.522316 11446 raft_consensus.cc:2749] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:47.523468   431 raft_consensus.cc:2243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:47.523844  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:47.527812  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 14:55:47.536046 11462 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.537155 11462 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.538218   431 raft_consensus.cc:2272] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.540575 11462 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:47.530184  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:47.550201  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:47.552310  2652 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:47.542615   431 tablet_replica.cc:333] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
W20260709 14:55:47.553580  2652 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:47.553745 11446 raft_consensus.cc:493] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4f1347e243fd46a5bb7cee5b2e23b665)
W20260709 14:55:47.554368  2652 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.554721 11446 raft_consensus.cc:515] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.555199  2652 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:47.556430 11475 raft_consensus.cc:2749] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:47.558143 11446 leader_election.cc:290] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:47.558936  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f08b71bff4091b9f08d398c3f894d" candidate_uuid: "56620e612f0549e6abf320aa80398e74" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
W20260709 14:55:47.558063  2652 consensus_peers.cc:597] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:47.560402  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.562172   431 raft_consensus.cc:2243] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:47.562562  2726 consensus_peers.cc:597] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:47.567057  2726 consensus_peers.cc:597] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:47.571084  2726 consensus_peers.cc:597] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:47.563424   431 raft_consensus.cc:2272] T 1591def49c3c47edac03ade007ebc91a P 56620e612f0549e6abf320aa80398e74 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:47.552767  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 14:55:47.562706  2652 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:47.583796  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:47.585160   431 tablet_replica.cc:333] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
W20260709 14:55:47.585307  2576 leader_election.cc:336] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.585985  2576 leader_election.cc:304] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56620e612f0549e6abf320aa80398e74; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:47.585266  2650 consensus_peers.cc:597] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 14:55:47.586886 11446 raft_consensus.cc:2749] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:47.586897   431 raft_consensus.cc:2243] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:47.590920   431 raft_consensus.cc:2272] T 6cf25c436e92407284806ff0a6ae57b0 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:47.591121  2726 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.591753  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.592783 11462 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:47.595161  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.595783   431 tablet_replica.cc:333] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.596999   431 raft_consensus.cc:2243] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:47.595870  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.597967   431 raft_consensus.cc:2272] T 7584380efde04ee289b42acaea2e4742 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.598667 11475 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:47.602312   431 tablet_replica.cc:333] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.603667   431 raft_consensus.cc:2243] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 28 LEADER]: Raft consensus shutting down.
I20260709 14:55:47.606398   431 raft_consensus.cc:2272] T ff9fe3d5f686477aa2bc32f628dcfb08 P 56620e612f0549e6abf320aa80398e74 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.610749   431 tablet_replica.cc:333] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.611971   431 raft_consensus.cc:2243] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:47.612968   431 raft_consensus.cc:2272] T 7ab522322e3448f18a90387481cab3ff P 56620e612f0549e6abf320aa80398e74 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.617383   431 tablet_replica.cc:333] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.618251 11462 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.618490   431 raft_consensus.cc:2243] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 31 LEADER]: Raft consensus shutting down.
I20260709 14:55:47.619323 11462 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.622120   431 raft_consensus.cc:2272] T 100d7ba791034203ab40e147a55e2183 P 56620e612f0549e6abf320aa80398e74 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.622370 11462 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.627147   431 tablet_replica.cc:333] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.628477   431 raft_consensus.cc:2243] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:47.629786  2724 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:47.630504  2726 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.629874   431 raft_consensus.cc:2272] T c1bae386d4ba412d86a21df33a9fa819 P 56620e612f0549e6abf320aa80398e74 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.631044  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.631999 11462 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:47.635807   431 tablet_replica.cc:333] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.636668   431 raft_consensus.cc:2243] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:47.637439   431 raft_consensus.cc:2272] T 5d0f08b71bff4091b9f08d398c3f894d P 56620e612f0549e6abf320aa80398e74 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.640944   431 tablet_replica.cc:333] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74: stopping tablet replica
I20260709 14:55:47.641669   431 raft_consensus.cc:2243] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:47.644013   431 raft_consensus.cc:2272] T e9427dea4aea4238bc56533da5d2720e P 56620e612f0549e6abf320aa80398e74 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:47.661854 11475 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.662789 11475 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:47.665783 11475 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.669256 11475 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.670194 11475 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:47.672894 11469 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.673269 11475 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.673921 11469 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:47.677043 11469 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:47.678659  2652 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.682199 11462 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.683162 11462 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.689474  2652 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:47.690959  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.688352  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.695784 11462 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.700318 11474 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:47.693225  2652 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.698203  2650 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.702548 11474 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:47.703799  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.704351  2650 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.705164 11469 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:47.705694 11474 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:47.707037  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.707587  2650 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.708549 11478 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:47.709939 11475 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 14:55:47.714203  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:47.717577  2726 leader_election.cc:336] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.718362  2726 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:47.719375 11474 raft_consensus.cc:2749] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 14:55:47.723874  2726 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.724421  2726 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.725306 11474 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:47.751092 11474 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.761410 11462 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.762362 11462 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.765033 11462 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.767832  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
W20260709 14:55:47.768445  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:47.781076  2650 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:47.783396 11462 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.784263 11462 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.788055  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.752156 11474 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.792891 11474 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.808238 11462 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.810596 11469 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:47.815012  2650 consensus_peers.cc:597] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:47.815474 11469 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.811964  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.823056  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:47.823998  2753 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:55:47.821511 11469 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:47.821053 11462 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.828516  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:47.832494 11462 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.834213 11475 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 14:55:47.834816 11475 raft_consensus.cc:493] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.835126 11462 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.835309 11475 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:47.837133  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.838006  2677 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
W20260709 14:55:47.837025  2726 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.839874  2726 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:47.840304  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
W20260709 14:55:47.840150  2652 leader_election.cc:336] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.838997 11478 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.841578 11478 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.841977  2726 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.844709  2726 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:47.842578 11462 raft_consensus.cc:2749] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:47.845713 11474 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:47.847100 11474 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.847527 11474 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:47.850037  2652 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.844167 11469 raft_consensus.cc:493] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:47.849967  2726 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 14:55:47.851620  2726 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.851624 11469 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.855051 11478 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:47.855929 11469 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:47.857611  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:47.858448  2753 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 26.
I20260709 14:55:47.859457  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:47.860584  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:47.861357 11458 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:47.862449 11458 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.863123 11458 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.863530 11478 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:47.864166 11478 raft_consensus.cc:493] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.864703 11478 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:47.865964 11458 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.866634 11456 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
W20260709 14:55:47.867672  2726 connection.cc:769] client connection to 127.0.107.194:40091 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 14:55:47.867391 11456 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.860733 11462 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:47.870381 11456 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:47.879725  2652 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 14:55:47.887056  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.888043 11462 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.888728 11462 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.890094  2724 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.895488 11462 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.898504  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:47.899590  2677 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 26.
I20260709 14:55:47.902680  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:47.903759 11462 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 14:55:47.904182 11462 raft_consensus.cc:493] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.904592 11462 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:47.921546  2726 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:47.917263 11475 raft_consensus.cc:515] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.924320 11456 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.925151 11456 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:47.926137 11474 raft_consensus.cc:515] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:47.928938 11460 raft_consensus.cc:493] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.929608  2754 raft_consensus.cc:2393] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 27: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
I20260709 14:55:47.929718 11460 raft_consensus.cc:515] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:47.933585  2651 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
I20260709 14:55:47.934278  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92641606b7ec403d843b2ee872402de3" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:47.935256  2753 raft_consensus.cc:3060] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Advancing to term 25
I20260709 14:55:47.935447 11467 raft_consensus.cc:3060] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:47.936345 11460 leader_election.cc:290] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:47.936751 11456 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:47.943105  2726 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:47.943784  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:47.946053  2726 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:47.946751  2726 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.933061 11475 leader_election.cc:290] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:47.949472  2753 raft_consensus.cc:2468] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 25.
I20260709 14:55:47.956092 11481 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 14:55:47.956264  2726 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:47.948009 11456 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:47.962529 11467 raft_consensus.cc:2749] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:47.964164  2651 leader_election.cc:304] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:47.951370 11462 raft_consensus.cc:515] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.966470  2726 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:47.966776 11475 raft_consensus.cc:2804] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 14:55:47.973408  2707 debug-util.cc:398] Leaking SignalData structure 0x7b0800168020 after lost signal to thread 434
I20260709 14:55:47.973523 11475 raft_consensus.cc:697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:47.978716 11475 consensus_queue.cc:237] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:47.982445  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080031e3a0 after lost signal to thread 2469
W20260709 14:55:47.983578  2707 debug-util.cc:398] Leaking SignalData structure 0x7b080024a420 after lost signal to thread 2636
W20260709 14:55:47.986090  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08000f78e0 after lost signal to thread 2710
W20260709 14:55:47.988073  2724 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:47.990837 11474 leader_election.cc:290] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:47.991906 11458 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:47.989013  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:47.992522  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2b04c469b2404c908d67b076cf2ba4" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:47.992715 11458 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:48.002568  2707 debug-util.cc:398] Leaking SignalData structure 0x7b08003a3e00 after lost signal to thread 2784
W20260709 14:55:48.007961  2726 leader_election.cc:336] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.009179 11456 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:48.011714 11458 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:48.017108 11460 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.017962 11460 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.018651  2432 catalog_manager.cc:5697] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 24 to 25, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 25 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:48.019919  2652 leader_election.cc:336] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:48.019789  2650 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:48.023976  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:48.024823  2753 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 21.
I20260709 14:55:48.025646  2685 raft_consensus.cc:2468] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
I20260709 14:55:48.027273  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:48.029510 11467 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 14:55:48.029647 11460 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.030052 11467 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.030661 11467 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 14:55:48.030959  2652 leader_election.cc:336] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:48.032014  2726 leader_election.cc:336] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.031126 11475 raft_consensus.cc:493] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.033505  2652 leader_election.cc:304] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:48.033841 11475 raft_consensus.cc:515] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:48.029526  2723 leader_election.cc:304] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:48.036422 11475 leader_election.cc:290] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:48.037110 11475 raft_consensus.cc:2749] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:48.037350 11456 raft_consensus.cc:2804] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:48.038453 11456 raft_consensus.cc:697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:48.040005 11456 consensus_queue.cc:237] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.044960  2650 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 14:55:48.044842  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f231033e78bf42ad8d6dbdf1f54d21be" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:48.045733  2685 raft_consensus.cc:3060] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 14:55:48.045765  2650 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 14:55:48.055696 11478 raft_consensus.cc:515] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.044234  2726 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.059031 11478 leader_election.cc:290] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.037482 11462 leader_election.cc:290] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:48.061234  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c39462fb47b74715a7e5498d49eaab87" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:48.062043  2753 raft_consensus.cc:3060] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Advancing to term 27
I20260709 14:55:48.064162 11467 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.065452  2685 raft_consensus.cc:2468] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 27.
W20260709 14:55:48.062451  2650 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:48.068240  2754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2c3e021741a28db500c36ebdc494" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:48.068240  2723 leader_election.cc:304] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:48.069183  2754 raft_consensus.cc:3060] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Advancing to term 22
I20260709 14:55:48.069911 11462 raft_consensus.cc:2804] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:48.070706 11462 raft_consensus.cc:697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
W20260709 14:55:48.071178  2652 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.071996 11462 consensus_queue.cc:237] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.072599  2652 leader_election.cc:336] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.073244  2652 leader_election.cc:304] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:48.083530  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:48.084885  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:48.086059 11456 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:48.085402 11467 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.096009  2434 catalog_manager.cc:5697] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 26 to 27, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 27 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:48.102336 11478 raft_consensus.cc:2749] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:48.106518  2754 raft_consensus.cc:2468] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 22.
W20260709 14:55:48.107760  2652 leader_election.cc:336] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.107470  2753 raft_consensus.cc:2468] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 27.
I20260709 14:55:48.109179  2651 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:48.110034 11478 raft_consensus.cc:2804] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 14:55:48.110136  2651 leader_election.cc:304] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:48.110636 11478 raft_consensus.cc:697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:48.111423 11467 raft_consensus.cc:2804] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 14:55:48.111984 11467 raft_consensus.cc:697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:48.112170 11478 consensus_queue.cc:237] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.113054 11467 consensus_queue.cc:237] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.119196  2652 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.214512  2432 catalog_manager.cc:5697] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 26 to 27, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 27 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:48.221107  2434 catalog_manager.cc:5697] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 21 to 22, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:48.223495  2434 catalog_manager.cc:5697] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 26 to 27, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 27 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:48.321904  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:48.424093  2753 raft_consensus.cc:1275] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 14:55:48.426237 11467 consensus_queue.cc:1048] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:48.448768  2652 consensus_peers.cc:597] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:48.488054  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 14:55:48.500944  2650 consensus_peers.cc:597] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 14:55:48.501638  2650 consensus_peers.cc:597] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:48.512971  2685 raft_consensus.cc:1275] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 14:55:48.514997 11458 consensus_queue.cc:1048] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:48.547009  2726 consensus_peers.cc:597] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:48.548939  2753 raft_consensus.cc:1275] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 14:55:48.550902 11467 consensus_queue.cc:1048] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:48.552937  2650 consensus_peers.cc:597] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:48.565855  2652 consensus_peers.cc:597] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:48.593607  2685 raft_consensus.cc:1275] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 14:55:48.596269 11456 consensus_queue.cc:1048] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 14:55:48.606412  2726 consensus_peers.cc:597] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:48.629779  2724 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:48.687798  2650 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 14:55:48.702656  2754 raft_consensus.cc:1275] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 14:55:48.706772 11467 consensus_queue.cc:1048] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 14:55:48.734521 11474 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.735217 11474 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:48.736889  2652 consensus_peers.cc:597] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:48.737622 11474 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:48.764319  2652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111) [suppressed 494 similar messages]
W20260709 14:55:48.765632  2726 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.766728 11474 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.768949 11474 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.776683  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:48.777318  2724 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
W20260709 14:55:48.778996  2650 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 14:55:48.780576 11456 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 14:55:48.784803 11467 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.786069 11460 raft_consensus.cc:493] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.786746 11460 raft_consensus.cc:515] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:48.788487 11475 raft_consensus.cc:493] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.789141 11475 raft_consensus.cc:515] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:48.792196 11475 leader_election.cc:290] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:48.795261  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:48.797859 11474 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
W20260709 14:55:48.801895  2650 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:48.803578  2650 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:48.792228 11460 leader_election.cc:290] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:48.806277  2726 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.804152 11467 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.811618  2685 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 28.
I20260709 14:55:48.812917 11474 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.813558 11474 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.814245  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:48.812737 11475 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.815644 11456 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 14:55:48.815416 11475 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.815956 11474 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:48.816143 11456 raft_consensus.cc:493] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.816659 11456 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Advancing to term 29
I20260709 14:55:48.817996 11467 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.818195 11475 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.823019  2760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:48.824064  2760 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 23.
I20260709 14:55:48.824528  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" is_pre_election: true
I20260709 14:55:48.826798  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:48.828562 11467 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 14:55:48.829022 11467 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:48.829409 11467 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:48.831437  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" is_pre_election: true
I20260709 14:55:48.839023 11456 raft_consensus.cc:515] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.841125  2753 raft_consensus.cc:2393] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7bc44bf9b26c41148142e09b78ccb3c3 in current term 29: Already voted for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in this term.
W20260709 14:55:48.839740  2652 leader_election.cc:336] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.843572  2652 leader_election.cc:304] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:48.844534 11475 raft_consensus.cc:2749] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 14:55:48.837105  2726 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.847059  2677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498c3de2b5414c968242f7f8fc2ca7a1" candidate_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "7bc44bf9b26c41148142e09b78ccb3c3"
I20260709 14:55:48.847675 11456 leader_election.cc:290] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:48.847808  2677 raft_consensus.cc:3060] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 14:55:48.845948  2652 leader_election.cc:336] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.851085  2652 leader_election.cc:304] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:48.855134 11475 raft_consensus.cc:2749] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 14:55:48.858484  2677 raft_consensus.cc:2468] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1347e243fd46a5bb7cee5b2e23b665 in term 29.
I20260709 14:55:48.866393  2723 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 
I20260709 14:55:48.867501 11456 raft_consensus.cc:2804] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 14:55:48.868520 11456 raft_consensus.cc:697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 LEADER]: Becoming Leader. State: Replica: 4f1347e243fd46a5bb7cee5b2e23b665, State: Running, Role: LEADER
I20260709 14:55:48.869853 11456 consensus_queue.cc:237] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.872148  2652 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.874552  2652 leader_election.cc:304] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 4f1347e243fd46a5bb7cee5b2e23b665, 56620e612f0549e6abf320aa80398e74
W20260709 14:55:48.874897  2726 leader_election.cc:336] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.875595 11475 raft_consensus.cc:2749] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 14:55:48.877249  2652 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:48.889381  2650 consensus_peers.cc:597] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:48.867590 11467 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:48.893051  2685 raft_consensus.cc:2393] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f1347e243fd46a5bb7cee5b2e23b665 in current term 24: Already voted for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in this term.
I20260709 14:55:48.894775 11467 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Requested vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196:46277)
I20260709 14:55:48.895638  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:48.897912  2753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cf25c436e92407284806ff0a6ae57b0" candidate_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4f1347e243fd46a5bb7cee5b2e23b665"
I20260709 14:55:48.898746  2753 raft_consensus.cc:3060] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Advancing to term 24
I20260709 14:55:48.897750  2434 catalog_manager.cc:5697] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 reported cstate change: term changed from 28 to 29, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 4f1347e243fd46a5bb7cee5b2e23b665 (127.0.107.196). New cstate: current_term: 29 leader_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:48.906162  2652 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:48.919265  2753 raft_consensus.cc:2468] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc44bf9b26c41148142e09b78ccb3c3 in term 24.
I20260709 14:55:48.920893 11456 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:48.921540  2651 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665, 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74
I20260709 14:55:48.923163 11467 raft_consensus.cc:2804] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 14:55:48.923782 11467 raft_consensus.cc:697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7bc44bf9b26c41148142e09b78ccb3c3, State: Running, Role: LEADER
I20260709 14:55:48.924916 11467 consensus_queue.cc:237] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
W20260709 14:55:48.963240  2724 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:49.001451  2650 consensus_peers.cc:597] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:49.006163  2432 catalog_manager.cc:5697] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 reported cstate change: term changed from 23 to 24, leader changed from 56620e612f0549e6abf320aa80398e74 (127.0.107.194) to 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:49.029515  2724 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 14:55:49.059108  2652 consensus_peers.cc:597] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.136332 11474 raft_consensus.cc:493] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.137068 11474 raft_consensus.cc:515] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:49.148455 11474 leader_election.cc:290] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.151546  2724 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:49.155789  2726 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.156378  2726 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.156829  2726 leader_election.cc:336] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.157251  2726 leader_election.cc:304] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.157995 11474 raft_consensus.cc:2749] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 14:55:49.201711  2726 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.218762  2652 consensus_peers.cc:597] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.258767 11475 raft_consensus.cc:493] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.259478 11475 raft_consensus.cc:515] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:49.261850 11475 leader_election.cc:290] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.268934  2726 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.270171  2652 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:49.276911  2650 leader_election.cc:336] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:49.277362  2650 leader_election.cc:304] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.278141 11475 raft_consensus.cc:2749] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:49.317839 11462 raft_consensus.cc:493] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.318482 11462 raft_consensus.cc:515] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:49.320915 11462 leader_election.cc:290] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.326359  2726 consensus_peers.cc:597] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.329618  2724 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:49.334187  2726 leader_election.cc:336] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.335565  2726 leader_election.cc:304] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.336490 11462 raft_consensus.cc:2749] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:49.348846  2652 consensus_peers.cc:597] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.365640  2685 raft_consensus.cc:1275] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Refusing update from remote peer 4f1347e243fd46a5bb7cee5b2e23b665: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 14:55:49.367439 11462 consensus_queue.cc:1048] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 14:55:49.377460  2726 consensus_peers.cc:597] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:49.378757  2652 consensus_peers.cc:597] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.402698 11456 raft_consensus.cc:493] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.403401 11456 raft_consensus.cc:515] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
W20260709 14:55:49.421044  2726 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.421371   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 14:55:49.421761 11456 leader_election.cc:290] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.431825  2724 leader_election.cc:336] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:49.432680  2724 leader_election.cc:304] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.434403 11456 raft_consensus.cc:2749] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 14:55:49.448596  2753 raft_consensus.cc:1275] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Refusing update from remote peer 7bc44bf9b26c41148142e09b78ccb3c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 14:55:49.450706 11475 consensus_queue.cc:1048] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 14:55:49.453655  2726 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.479445  2724 consensus_peers.cc:597] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:49.483862  2724 consensus_peers.cc:597] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:49.491310 11475 raft_consensus.cc:493] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.492139 11475 raft_consensus.cc:515] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:49.496371 11467 raft_consensus.cc:493] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.497121 11467 raft_consensus.cc:515] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:49.499882 11467 leader_election.cc:290] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:49.501056 11475 leader_election.cc:290] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.508433  2650 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:49.509127  2650 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:49.543100  2724 consensus_peers.cc:597] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 14:55:49.553912  2652 leader_election.cc:336] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.554587  2652 leader_election.cc:304] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.562011 11475 raft_consensus.cc:2749] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 14:55:49.561249  2652 leader_election.cc:336] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.567914  2652 leader_election.cc:304] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.570366 11475 raft_consensus.cc:2749] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 14:55:49.572196  2652 consensus_peers.cc:597] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.580955 11437 raft_consensus.cc:493] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.581709 11437 raft_consensus.cc:515] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:49.586016 11437 leader_election.cc:290] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.593420  2724 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 14:55:49.601145  2726 consensus_peers.cc:597] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.605518  2650 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:49.614140  2652 leader_election.cc:336] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:49.614693  2652 leader_election.cc:304] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bc44bf9b26c41148142e09b78ccb3c3; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.615772 11467 raft_consensus.cc:2749] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 14:55:49.617875   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 14:55:49.635315  2650 consensus_peers.cc:597] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 14:55:49.637919  2652 consensus_peers.cc:597] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.668403  2726 consensus_peers.cc:597] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:49.669149  2723 consensus_peers.cc:597] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:49.680318  2652 consensus_peers.cc:597] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.681768 11474 raft_consensus.cc:493] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.682484 11474 raft_consensus.cc:515] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:49.685052 11474 leader_election.cc:290] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:49.690138  2726 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:49.690778  2724 leader_election.cc:336] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:49.691399  2724 leader_election.cc:304] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.692649 11474 raft_consensus.cc:2749] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 14:55:49.697531   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:55:49.699031   431 tablet_replica.cc:333] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.700349   431 raft_consensus.cc:2243] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.701226   431 raft_consensus.cc:2272] T f231033e78bf42ad8d6dbdf1f54d21be P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.704995   431 tablet_replica.cc:333] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.706156   431 raft_consensus.cc:2243] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.706998   431 raft_consensus.cc:2272] T f01d4b3ec58f4ca3b994c5e66179174a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.710714   431 tablet_replica.cc:333] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.711786   431 raft_consensus.cc:2243] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.712514   431 raft_consensus.cc:2272] T 1c902c27b99b4cc0a5186bc8117246a7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.716553   431 tablet_replica.cc:333] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.717612   431 raft_consensus.cc:2243] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.718499   431 raft_consensus.cc:2272] T 8e31634415994959847625e41eecf9b3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.722553   431 tablet_replica.cc:333] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.723109  2650 consensus_peers.cc:597] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 14:55:49.723733   431 raft_consensus.cc:2243] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.724654   431 raft_consensus.cc:2272] T 6340ce5f0bc04ded8681c8573a0910c1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.728749   431 tablet_replica.cc:333] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.729952   431 raft_consensus.cc:2243] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.731004   431 raft_consensus.cc:2272] T 963a4155717b4274b0121341056b7cb8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.734910   431 tablet_replica.cc:333] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.736068   431 raft_consensus.cc:2243] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.736975   431 raft_consensus.cc:2272] T 194ae28527c14d78817238308fb284c6 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.740908   431 tablet_replica.cc:333] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.741151  2652 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.742233   431 raft_consensus.cc:2243] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.744071   431 raft_consensus.cc:2272] T 31e4166c53c7459b808e9e6841d90c56 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.747675   431 tablet_replica.cc:333] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.748909   431 raft_consensus.cc:2243] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.750039   431 raft_consensus.cc:2272] T c051e44d64d14770b36c5012be981196 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.750296 11474 raft_consensus.cc:493] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:49.751107 11474 raft_consensus.cc:515] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:49.753870 11474 leader_election.cc:290] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
I20260709 14:55:49.753983   431 tablet_replica.cc:333] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.755331   431 raft_consensus.cc:2243] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.756197   431 raft_consensus.cc:2272] T 7abc2eae2c0540e4bc1c7141e81bf63c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.759917   431 tablet_replica.cc:333] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.761147   431 raft_consensus.cc:2243] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.762019   431 raft_consensus.cc:2272] T e5f95289d39644bead88ff8640742871 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.762228  2723 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.765947   431 tablet_replica.cc:333] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.767400   431 raft_consensus.cc:2243] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 LEADER]: Raft consensus shutting down.
W20260709 14:55:49.765761  2726 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:49.768132  2724 leader_election.cc:336] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:49.768616  2724 leader_election.cc:304] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:49.769138   431 raft_consensus.cc:2272] T b47a7b94ee76444db9c73a90f520af7c P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.769667 11474 raft_consensus.cc:2749] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 14:55:49.773175   431 tablet_replica.cc:333] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.774353   431 raft_consensus.cc:2243] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.775972   431 raft_consensus.cc:2272] T b5cca18a56d54a9aae32ff20c48ef0f8 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.779475   431 tablet_replica.cc:333] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.780460   431 raft_consensus.cc:2243] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.781917   431 raft_consensus.cc:2272] T eeb3e91a07fc4383bd55f034f23c3cea P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.785459   431 tablet_replica.cc:333] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.786455   431 raft_consensus.cc:2243] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.787201   431 raft_consensus.cc:2272] T dee6ef23c038421d972947971af9f0cc P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.790817  2726 consensus_peers.cc:597] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.790970   431 tablet_replica.cc:333] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.792225   431 raft_consensus.cc:2243] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.793931   431 raft_consensus.cc:2272] T 7a9cf6350d3b499497e3ace2d1dea988 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.797760   431 tablet_replica.cc:333] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.798792   431 raft_consensus.cc:2243] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.799953   431 raft_consensus.cc:2272] T a2e7eae198c34a6fafb697d50f269192 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.803514   431 tablet_replica.cc:333] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.804478   431 raft_consensus.cc:2243] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.805838   431 raft_consensus.cc:2272] T 92641606b7ec403d843b2ee872402de3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.809103   431 tablet_replica.cc:333] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.810173   431 raft_consensus.cc:2243] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.811178   431 raft_consensus.cc:2272] T f32f10706d04497089f6159669dc04a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.814683   431 tablet_replica.cc:333] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.815721   431 raft_consensus.cc:2243] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.816519   431 raft_consensus.cc:2272] T 7ea5acf8ab374c8fa3009782429fd3ed P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.819849   431 tablet_replica.cc:333] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.820779   431 raft_consensus.cc:2243] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.821573   431 raft_consensus.cc:2272] T 6baf9a66ae7e41df9d5d2079bebb7381 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.823459  2723 consensus_peers.cc:597] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:49.823624  2650 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 14:55:49.825161   431 tablet_replica.cc:333] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.826337   431 raft_consensus.cc:2243] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.827720   431 raft_consensus.cc:2272] T 6cf25c436e92407284806ff0a6ae57b0 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.830849  2652 consensus_peers.cc:597] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.832197   431 tablet_replica.cc:333] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.832304  2723 consensus_peers.cc:597] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.833376   431 raft_consensus.cc:2243] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.835000   431 raft_consensus.cc:2272] T 750a16eb87de49b4a212064463a9d3a4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.838634   431 tablet_replica.cc:333] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.839533  2723 consensus_peers.cc:597] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.839725   431 raft_consensus.cc:2243] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.840734   431 raft_consensus.cc:2272] T 1b2b04c469b2404c908d67b076cf2ba4 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.843485   431 tablet_replica.cc:333] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.844280   431 raft_consensus.cc:2243] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.845582   431 raft_consensus.cc:2272] T 343553f3139e4da4ba16879fae174c5a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.848326   431 tablet_replica.cc:333] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.849077   431 raft_consensus.cc:2243] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.849761   431 raft_consensus.cc:2272] T a594c31352514fb685540e1082f8ee7d P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.852634   431 tablet_replica.cc:333] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.853446   431 raft_consensus.cc:2243] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.854665   431 raft_consensus.cc:2272] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.858525   431 tablet_replica.cc:333] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.859699   431 raft_consensus.cc:2243] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.861471   431 raft_consensus.cc:2272] T 4837b30553bb4eac8840db77c7ffca53 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.862758  2652 consensus_peers.cc:597] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 14:55:49.865635   431 tablet_replica.cc:333] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.866851   431 raft_consensus.cc:2243] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.868470   431 raft_consensus.cc:2272] T 8fb844e4a36641d79afdab73422e06f1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.872756   431 tablet_replica.cc:333] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.873881   431 raft_consensus.cc:2243] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.874763   431 raft_consensus.cc:2272] T da98c08aee5d4d86bce741e08142c1e3 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.878502   431 tablet_replica.cc:333] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.879680   431 raft_consensus.cc:2243] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.881292   431 raft_consensus.cc:2272] T 406cc4479d48441398435d7dc1850d64 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.884167  2723 consensus_peers.cc:597] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.884953   431 tablet_replica.cc:333] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.885109  2723 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.886126   431 raft_consensus.cc:2243] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.887975   431 raft_consensus.cc:2272] T 7584380efde04ee289b42acaea2e4742 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.891849   431 tablet_replica.cc:333] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.892860   431 raft_consensus.cc:2243] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.894443   431 raft_consensus.cc:2272] T e7c0ea3d097a4b27b9367e3b84b6d5fd P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.898556   431 tablet_replica.cc:333] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.899624   431 raft_consensus.cc:2243] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.901175   431 raft_consensus.cc:2272] T 1591def49c3c47edac03ade007ebc91a P 7bc44bf9b26c41148142e09b78ccb3c3 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.904093  2650 consensus_peers.cc:597] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:49.904801   431 tablet_replica.cc:333] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.905826   431 raft_consensus.cc:2243] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.906567   431 raft_consensus.cc:2272] T 498c3de2b5414c968242f7f8fc2ca7a1 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.910574   431 tablet_replica.cc:333] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.911566   431 raft_consensus.cc:2243] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.912981   431 raft_consensus.cc:2272] T c39462fb47b74715a7e5498d49eaab87 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.916509   431 tablet_replica.cc:333] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.917562   431 raft_consensus.cc:2243] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.919086   431 raft_consensus.cc:2272] T e91eefb5789d41f0b4cd60e9f83fef3f P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.922542   431 tablet_replica.cc:333] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.923485   431 raft_consensus.cc:2243] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.924949   431 raft_consensus.cc:2272] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.926993  2723 consensus_peers.cc:597] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.928768   431 tablet_replica.cc:333] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.929852   431 raft_consensus.cc:2243] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.931329   431 raft_consensus.cc:2272] T d87a2c3e021741a28db500c36ebdc494 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.935141   431 tablet_replica.cc:333] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
W20260709 14:55:49.935947  2723 consensus_peers.cc:597] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:49.936076   431 raft_consensus.cc:2243] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.938161   431 raft_consensus.cc:2272] T acb30692225e4daebaa7e02e9cb291b7 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.941991   431 tablet_replica.cc:333] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.943018   431 raft_consensus.cc:2243] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.944644   431 raft_consensus.cc:2272] T c1bae386d4ba412d86a21df33a9fa819 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.948444   431 tablet_replica.cc:333] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.949501   431 raft_consensus.cc:2243] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:49.951031   431 raft_consensus.cc:2272] T 7ab522322e3448f18a90387481cab3ff P 7bc44bf9b26c41148142e09b78ccb3c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.954586   431 tablet_replica.cc:333] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.955453   431 raft_consensus.cc:2243] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.956514   431 raft_consensus.cc:2272] T 100d7ba791034203ab40e147a55e2183 P 7bc44bf9b26c41148142e09b78ccb3c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:49.959906   431 tablet_replica.cc:333] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3: stopping tablet replica
I20260709 14:55:49.960683   431 raft_consensus.cc:2243] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:49.961513   431 raft_consensus.cc:2272] T e9427dea4aea4238bc56533da5d2720e P 7bc44bf9b26c41148142e09b78ccb3c3 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:49.982791  2723 consensus_peers.cc:597] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:50.008857  2723 consensus_peers.cc:597] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:50.027545  2724 consensus_peers.cc:597] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 14:55:50.032457  2723 consensus_peers.cc:597] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:50.038286 11474 raft_consensus.cc:493] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:50.039317 11474 raft_consensus.cc:515] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:50.043555 11474 leader_election.cc:290] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:50.048118  2726 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:50.052474  2724 leader_election.cc:336] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:50.053210  2724 leader_election.cc:304] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:50.054208 11474 raft_consensus.cc:2749] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 14:55:50.055024  2723 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Couldn't send request to peer 7bc44bf9b26c41148142e09b78ccb3c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:50.127738  2724 consensus_peers.cc:597] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 14:55:50.133231 11474 raft_consensus.cc:493] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:50.134016 11474 raft_consensus.cc:515] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:50.136736 11474 leader_election.cc:290] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:50.141155  2724 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:50.143517  2726 leader_election.cc:336] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:50.144120  2726 leader_election.cc:304] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:50.144995 11474 raft_consensus.cc:2749] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:50.164266 11474 raft_consensus.cc:493] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:50.164896 11474 raft_consensus.cc:515] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } }
I20260709 14:55:50.167308 11474 leader_election.cc:290] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091)
W20260709 14:55:50.171775  2726 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:50.173139  2724 leader_election.cc:336] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:50.173609  2724 leader_election.cc:304] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:50.174414 11456 raft_consensus.cc:2749] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 14:55:50.206679  2726 consensus_peers.cc:597] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Couldn't send request to peer 56620e612f0549e6abf320aa80398e74. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:50.445659  2724 consensus_peers.cc:597] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 14:55:50.680701 11456 raft_consensus.cc:493] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 56620e612f0549e6abf320aa80398e74)
I20260709 14:55:50.681289 11456 raft_consensus.cc:515] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
W20260709 14:55:50.682588  2724 consensus_peers.cc:597] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 14:55:50.684713 11456 leader_election.cc:290] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
W20260709 14:55:50.689962  2724 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
W20260709 14:55:50.690516  2726 leader_election.cc:336] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:50.690973  2726 leader_election.cc:304] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:50.691622 11456 raft_consensus.cc:2749] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 14:55:50.730631   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:55:50.836249   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 14:55:50.910087   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:55:50.911217   431 tablet_replica.cc:333] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.912155   431 raft_consensus.cc:2243] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.913064   431 raft_consensus.cc:2272] T dd47a5b34fb04ee887e263844b637438 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.915902   431 tablet_replica.cc:333] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.916882   431 raft_consensus.cc:2243] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.918257   431 raft_consensus.cc:2272] T c051e44d64d14770b36c5012be981196 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.920851   431 tablet_replica.cc:333] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.921540   431 raft_consensus.cc:2243] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.922631   431 raft_consensus.cc:2272] T f231033e78bf42ad8d6dbdf1f54d21be P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.924831   431 tablet_replica.cc:333] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.925509   431 raft_consensus.cc:2243] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.926093   431 raft_consensus.cc:2272] T 947b4a5caeb243ef87d3a535eddda080 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.928562   431 tablet_replica.cc:333] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.929231   431 raft_consensus.cc:2243] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.930271   431 raft_consensus.cc:2272] T 194ae28527c14d78817238308fb284c6 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.932611   431 tablet_replica.cc:333] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.933385   431 raft_consensus.cc:2243] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.933336 11512 raft_consensus.cc:493] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:50.934348   431 raft_consensus.cc:2272] T 406cc4479d48441398435d7dc1850d64 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.934156 11512 raft_consensus.cc:515] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } }
I20260709 14:55:50.936803 11512 leader_election.cc:290] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829), c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831)
I20260709 14:55:50.938071   431 tablet_replica.cc:333] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.939175   431 raft_consensus.cc:2243] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.940438   431 raft_consensus.cc:2272] T 6340ce5f0bc04ded8681c8573a0910c1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:50.941381  2723 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Network error: Client connection negotiation failed: client connection to 127.0.107.195:41829: connect: Connection refused (error 111)
W20260709 14:55:50.941934  2724 leader_election.cc:336] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:50.942302  2724 leader_election.cc:304] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:50.943006 11512 raft_consensus.cc:2749] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 14:55:50.943799   431 tablet_replica.cc:333] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.944751   431 raft_consensus.cc:2243] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 LEADER]: Raft consensus shutting down.
W20260709 14:55:50.945571  2724 consensus_peers.cc:597] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 -> Peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Couldn't send request to peer c1b0e692e4294ee3b8694bbada174ba1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 14:55:50.945989   431 raft_consensus.cc:2272] T e5f95289d39644bead88ff8640742871 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.949455   431 tablet_replica.cc:333] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.950196   431 raft_consensus.cc:2243] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.950906   431 raft_consensus.cc:2272] T 75e6d6e528724dcba0bbaf89427c37a9 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.954012   431 tablet_replica.cc:333] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.954921   431 raft_consensus.cc:2243] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.956475   431 raft_consensus.cc:2272] T 8e31634415994959847625e41eecf9b3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.959187   431 tablet_replica.cc:333] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.959887   431 raft_consensus.cc:2243] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.960532   431 raft_consensus.cc:2272] T b47a7b94ee76444db9c73a90f520af7c P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.963320   431 tablet_replica.cc:333] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.964105   431 raft_consensus.cc:2243] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.964916   431 raft_consensus.cc:2272] T b5cca18a56d54a9aae32ff20c48ef0f8 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.966794 11512 raft_consensus.cc:493] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:50.967478 11512 raft_consensus.cc:515] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:50.968480   431 tablet_replica.cc:333] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.969353   431 raft_consensus.cc:2243] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.969904 11512 leader_election.cc:290] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:50.970057   431 raft_consensus.cc:2272] T 92641606b7ec403d843b2ee872402de3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.973649   431 tablet_replica.cc:333] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
W20260709 14:55:50.974157  2723 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Network error: Client connection negotiation failed: client connection to 127.0.107.195:41829: connect: Connection refused (error 111)
I20260709 14:55:50.974655   431 raft_consensus.cc:2243] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:50.974785  2726 leader_election.cc:336] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:50.975423  2726 leader_election.cc:304] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:50.975701   431 raft_consensus.cc:2272] T 80e0e0c146c54411901808c4bbf22280 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.976194 11512 raft_consensus.cc:2749] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:50.978508   431 tablet_replica.cc:333] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.979406   431 raft_consensus.cc:2243] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.980679   431 raft_consensus.cc:2272] T 1f87ae033ca94d58bdc3a087236657b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.983486   431 tablet_replica.cc:333] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.984215   431 raft_consensus.cc:2243] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.985397   431 raft_consensus.cc:2272] T 9a8bd118baa7481399ce15d422463083 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.988133   431 tablet_replica.cc:333] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.988883   431 raft_consensus.cc:2243] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:50.990134   431 raft_consensus.cc:2272] T a594c31352514fb685540e1082f8ee7d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.993002   431 tablet_replica.cc:333] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.993868   431 raft_consensus.cc:2243] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.994881   431 raft_consensus.cc:2272] T b188cd7e4b5e4d9fb22a3645bdff7da1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:50.997638   431 tablet_replica.cc:333] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:50.998395   431 raft_consensus.cc:2243] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:50.999200   431 raft_consensus.cc:2272] T 343553f3139e4da4ba16879fae174c5a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.001726   431 tablet_replica.cc:333] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.002399   431 raft_consensus.cc:2243] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.003515   431 raft_consensus.cc:2272] T 1b2b04c469b2404c908d67b076cf2ba4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.006338   431 tablet_replica.cc:333] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.007220   431 raft_consensus.cc:2243] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.007082 11512 raft_consensus.cc:493] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:51.007990 11512 raft_consensus.cc:515] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:51.008965   431 raft_consensus.cc:2272] T 8b56ad927e054a56a77c811620dda543 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.010491 11512 leader_election.cc:290] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:51.012426   431 tablet_replica.cc:333] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.013500   431 raft_consensus.cc:2243] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 LEADER]: Raft consensus shutting down.
W20260709 14:55:51.014632  2726 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
I20260709 14:55:51.014979   431 raft_consensus.cc:2272] T a1826b730f9f4ae4b0e05570574563c3 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:51.015332  2723 leader_election.cc:336] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Network error: Client connection negotiation failed: client connection to 127.0.107.195:41829: connect: Connection refused (error 111)
I20260709 14:55:51.015740  2723 leader_election.cc:304] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:51.016486 11512 raft_consensus.cc:2749] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 14:55:51.018170   431 tablet_replica.cc:333] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.018379 11512 raft_consensus.cc:493] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:51.019054   431 raft_consensus.cc:2243] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.019018 11512 raft_consensus.cc:515] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56620e612f0549e6abf320aa80398e74" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40091 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } } peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } }
I20260709 14:55:51.020428   431 raft_consensus.cc:2272] T f32f10706d04497089f6159669dc04a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.021297 11512 leader_election.cc:290] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:51.023602   431 tablet_replica.cc:333] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.024693   431 raft_consensus.cc:2243] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 LEADER]: Raft consensus shutting down.
W20260709 14:55:51.025427  2726 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 56620e612f0549e6abf320aa80398e74 (127.0.107.194:40091): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40091: connect: Connection refused (error 111)
W20260709 14:55:51.026090  2723 leader_election.cc:336] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Network error: Client connection negotiation failed: client connection to 127.0.107.195:41829: connect: Connection refused (error 111)
I20260709 14:55:51.026110   431 raft_consensus.cc:2272] T 7ea5acf8ab374c8fa3009782429fd3ed P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.026618  2723 leader_election.cc:304] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 56620e612f0549e6abf320aa80398e74, 7bc44bf9b26c41148142e09b78ccb3c3
I20260709 14:55:51.027571 11512 raft_consensus.cc:2749] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 14:55:51.028707 11512 raft_consensus.cc:493] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7bc44bf9b26c41148142e09b78ccb3c3)
I20260709 14:55:51.029214 11512 raft_consensus.cc:515] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1347e243fd46a5bb7cee5b2e23b665" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 46277 } } peers { permanent_uuid: "c1b0e692e4294ee3b8694bbada174ba1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37831 } } peers { permanent_uuid: "7bc44bf9b26c41148142e09b78ccb3c3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41829 } }
I20260709 14:55:51.029822   431 tablet_replica.cc:333] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.030871   431 raft_consensus.cc:2243] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.031551   431 raft_consensus.cc:2272] T 4ac95ecab0524f35a894d205dac97a44 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.033959 11512 leader_election.cc:290] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831), 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829)
I20260709 14:55:51.034797   431 tablet_replica.cc:333] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
W20260709 14:55:51.035882  2724 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer c1b0e692e4294ee3b8694bbada174ba1 (127.0.107.193:37831): Network error: Client connection negotiation failed: client connection to 127.0.107.193:37831: connect: Connection refused (error 111)
I20260709 14:55:51.036026   431 raft_consensus.cc:2243] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.036929   431 raft_consensus.cc:2272] T 6cf25c436e92407284806ff0a6ae57b0 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 14:55:51.038522  2723 leader_election.cc:336] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 7bc44bf9b26c41148142e09b78ccb3c3 (127.0.107.195:41829): Network error: Client connection negotiation failed: client connection to 127.0.107.195:41829: connect: Connection refused (error 111)
I20260709 14:55:51.039044  2723 leader_election.cc:304] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f1347e243fd46a5bb7cee5b2e23b665; no voters: 7bc44bf9b26c41148142e09b78ccb3c3, c1b0e692e4294ee3b8694bbada174ba1
I20260709 14:55:51.039662 11512 raft_consensus.cc:2749] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 14:55:51.040012   431 tablet_replica.cc:333] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.040813   431 raft_consensus.cc:2243] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.041596   431 raft_consensus.cc:2272] T 750a16eb87de49b4a212064463a9d3a4 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.044049   431 tablet_replica.cc:333] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.044679   431 raft_consensus.cc:2243] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.045248   431 raft_consensus.cc:2272] T 0d27004b985a45779b0f86fcb7273dbc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.047711   431 tablet_replica.cc:333] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.048430   431 raft_consensus.cc:2243] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.049696   431 raft_consensus.cc:2272] T 6baf9a66ae7e41df9d5d2079bebb7381 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.052538   431 tablet_replica.cc:333] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.053206   431 raft_consensus.cc:2243] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.053869   431 raft_consensus.cc:2272] T 7a9cf6350d3b499497e3ace2d1dea988 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.056370   431 tablet_replica.cc:333] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.056996   431 raft_consensus.cc:2243] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.058102   431 raft_consensus.cc:2272] T 498c3de2b5414c968242f7f8fc2ca7a1 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.060557   431 tablet_replica.cc:333] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.061297   431 raft_consensus.cc:2243] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.062568   431 raft_consensus.cc:2272] T 5e1073216da14f9c91bcc3f31f935620 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.065109   431 tablet_replica.cc:333] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.065764   431 raft_consensus.cc:2243] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.066390   431 raft_consensus.cc:2272] T a57a97528f31400dbc0900e76292b021 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.068883   431 tablet_replica.cc:333] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.069538   431 raft_consensus.cc:2243] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.070017   431 raft_consensus.cc:2272] T 1591def49c3c47edac03ade007ebc91a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.072614   431 tablet_replica.cc:333] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.073292   431 raft_consensus.cc:2243] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.074002   431 raft_consensus.cc:2272] T ff9fe3d5f686477aa2bc32f628dcfb08 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.076635   431 tablet_replica.cc:333] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.077316   431 raft_consensus.cc:2243] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 14:55:51.077907 11512 raft_consensus.cc:670] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 14:55:51.078572   431 raft_consensus.cc:2272] T 7ab522322e3448f18a90387481cab3ff P 4f1347e243fd46a5bb7cee5b2e23b665 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.081243   431 tablet_replica.cc:333] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.081995   431 raft_consensus.cc:2243] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.082475   431 raft_consensus.cc:2272] T c39462fb47b74715a7e5498d49eaab87 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.084722   431 tablet_replica.cc:333] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.085325   431 raft_consensus.cc:2243] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.085821   431 raft_consensus.cc:2272] T e91eefb5789d41f0b4cd60e9f83fef3f P 4f1347e243fd46a5bb7cee5b2e23b665 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.088244   431 tablet_replica.cc:333] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.088874   431 raft_consensus.cc:2243] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.089452   431 raft_consensus.cc:2272] T 3c52e7cc9c9f460c9c052fe21cfb57d7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.091830   431 tablet_replica.cc:333] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.092444   431 raft_consensus.cc:2243] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.093536   431 raft_consensus.cc:2272] T dee6ef23c038421d972947971af9f0cc P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.096127   431 tablet_replica.cc:333] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.096773   431 raft_consensus.cc:2243] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.097494   431 raft_consensus.cc:2272] T 59c97f578e0e4d3f9c754fb31c0b4c6a P 4f1347e243fd46a5bb7cee5b2e23b665 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.099992   431 tablet_replica.cc:333] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.100634   431 raft_consensus.cc:2243] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.101775   431 raft_consensus.cc:2272] T 5d0f08b71bff4091b9f08d398c3f894d P 4f1347e243fd46a5bb7cee5b2e23b665 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.104209   431 tablet_replica.cc:333] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.104786   431 raft_consensus.cc:2243] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.105373   431 raft_consensus.cc:2272] T d87a2c3e021741a28db500c36ebdc494 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.107656   431 tablet_replica.cc:333] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665: stopping tablet replica
I20260709 14:55:51.108234   431 raft_consensus.cc:2243] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:51.108937   431 raft_consensus.cc:2272] T acb30692225e4daebaa7e02e9cb291b7 P 4f1347e243fd46a5bb7cee5b2e23b665 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.579968   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 14:55:51.670559   431 master.cc:562] Master@127.0.107.254:39575 shutting down...
I20260709 14:55:51.719820   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:55:51.720600   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:51.720989   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3f6dfe8cfa964684b547b44a4b308ca2: stopping tablet replica
I20260709 14:55:51.933930   431 master.cc:584] Master@127.0.107.254:39575 shutdown complete.
I20260709 14:55:51.986816   431 test_util.cc:182] -----------------------------------------------
I20260709 14:55:51.987152   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783608598766170-431-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (300745 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 14:55:51.991020   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34669
I20260709 14:55:51.992272   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:55:51.999006 11514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:51.999541 11516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:51.999006 11513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:55:52.000752   431 server_base.cc:1061] running on GCE node
I20260709 14:55:52.004614   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:55:52.004905   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:55:52.005030   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608952005017 us; error 0 us; skew 500 ppm
I20260709 14:55:52.005577   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:55:52.008898   431 webserver.cc:533] Webserver started at http://127.0.107.254:42711/ using document root <none> and password file <none>
I20260709 14:55:52.009553   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:55:52.009732   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:55:52.009992   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:55:52.012015   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "5c52f9dba20147df9be3d15cfc12bf3d"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.017906   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 14:55:52.022205 11521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.023260   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:55:52.023648   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "5c52f9dba20147df9be3d15cfc12bf3d"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.024015   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:55:52.053020   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:55:52.054687   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:55:52.102656   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34669
I20260709 14:55:52.102785 11572 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34669 every 8 connection(s)
I20260709 14:55:52.107213 11573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.120492 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d: Bootstrap starting.
I20260709 14:55:52.126767 11573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:52.132258 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d: No bootstrap required, opened a new log
I20260709 14:55:52.135002 11573 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER }
I20260709 14:55:52.135478 11573 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:52.135675 11573 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c52f9dba20147df9be3d15cfc12bf3d, State: Initialized, Role: FOLLOWER
I20260709 14:55:52.136359 11573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER }
I20260709 14:55:52.136817 11573 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:55:52.137027 11573 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:55:52.137243 11573 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:52.143414 11573 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER }
I20260709 14:55:52.144168 11573 leader_election.cc:304] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c52f9dba20147df9be3d15cfc12bf3d; no voters: 
I20260709 14:55:52.145557 11573 leader_election.cc:290] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:55:52.145982 11576 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:52.147653 11576 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 1 LEADER]: Becoming Leader. State: Replica: 5c52f9dba20147df9be3d15cfc12bf3d, State: Running, Role: LEADER
I20260709 14:55:52.148514 11576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER }
I20260709 14:55:52.149076 11573 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:55:52.154608 11578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c52f9dba20147df9be3d15cfc12bf3d. Latest consensus state: current_term: 1 leader_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER } }
I20260709 14:55:52.155678 11578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [sys.catalog]: This master's current role is: LEADER
I20260709 14:55:52.157378 11577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c52f9dba20147df9be3d15cfc12bf3d" member_type: VOTER } }
I20260709 14:55:52.158313 11577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [sys.catalog]: This master's current role is: LEADER
I20260709 14:55:52.161684 11585 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:55:52.169100 11585 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:55:52.170497   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:55:52.180711 11585 catalog_manager.cc:1383] Generated new cluster ID: 5a8768b230484daf95c8276a1fe55ccd
I20260709 14:55:52.181070 11585 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:55:52.201786 11585 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:55:52.203460 11585 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:55:52.222028 11585 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d: Generated new TSK 0
I20260709 14:55:52.222793 11585 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:55:52.238061   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:55:52.246018 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.248032 11595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.251699 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:55:52.252862   431 server_base.cc:1061] running on GCE node
I20260709 14:55:52.255982   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:55:52.256247   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:55:52.256390   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608952256376 us; error 0 us; skew 500 ppm
I20260709 14:55:52.256930   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:55:52.259879   431 webserver.cc:533] Webserver started at http://127.0.107.193:35225/ using document root <none> and password file <none>
I20260709 14:55:52.260434   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:55:52.260607   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:55:52.260968   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:55:52.262353   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "89721e000fcb47e8a347857c99e1262e"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.267827   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 14:55:52.271888 11602 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.272758   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:55:52.273113   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "89721e000fcb47e8a347857c99e1262e"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.273466   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:55:52.291231   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:55:52.292780   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:55:52.294646   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:55:52.297674   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:55:52.297993   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.298277   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:55:52.298502   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.355826   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38817
I20260709 14:55:52.355924 11664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38817 every 8 connection(s)
I20260709 14:55:52.362150   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:55:52.371390 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.372609 11670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.376700 11672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:55:52.377051   431 server_base.cc:1061] running on GCE node
I20260709 14:55:52.381721 11665 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34669
I20260709 14:55:52.382390 11665 heartbeater.cc:461] Registering TS with master...
I20260709 14:55:52.382815   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:55:52.383195   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:55:52.383427   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608952383404 us; error 0 us; skew 500 ppm
I20260709 14:55:52.383594 11665 heartbeater.cc:507] Master 127.0.107.254:34669 requested a full tablet report, sending...
I20260709 14:55:52.384290   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:55:52.387045 11538 ts_manager.cc:194] Registered new tserver with Master: 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817)
I20260709 14:55:52.387941   431 webserver.cc:533] Webserver started at http://127.0.107.194:33379/ using document root <none> and password file <none>
I20260709 14:55:52.388779   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:55:52.389056   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:55:52.389420   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:55:52.390076 11538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56640
I20260709 14:55:52.391357   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "139311dab688423789a921ffdc3ac2d3"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.397213   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 14:55:52.401299 11677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.402230   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:55:52.402618   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "139311dab688423789a921ffdc3ac2d3"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.403028   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:55:52.417651   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:55:52.419203   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:55:52.420991   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:55:52.423717   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:55:52.424006   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.424300   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:55:52.424504   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.472970   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40695
I20260709 14:55:52.473099 11739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40695 every 8 connection(s)
I20260709 14:55:52.478631   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:55:52.486708 11743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.488948 11744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:52.491468 11746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:55:52.492200 11740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34669
I20260709 14:55:52.492713   431 server_base.cc:1061] running on GCE node
I20260709 14:55:52.492798 11740 heartbeater.cc:461] Registering TS with master...
I20260709 14:55:52.493752   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 14:55:52.493767 11740 heartbeater.cc:507] Master 127.0.107.254:34669 requested a full tablet report, sending...
W20260709 14:55:52.494092   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:55:52.494397   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608952494367 us; error 0 us; skew 500 ppm
I20260709 14:55:52.495143   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:55:52.496901 11538 ts_manager.cc:194] Registered new tserver with Master: 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:52.498628   431 webserver.cc:533] Webserver started at http://127.0.107.195:44721/ using document root <none> and password file <none>
I20260709 14:55:52.499255 11538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56642
I20260709 14:55:52.499552   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:55:52.499867   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:55:52.500209   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:55:52.501631   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "de497551bc0a41b695ba92f9b4acbd50"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.507177   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 14:55:52.511596 11751 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.512676   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 14:55:52.513078   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "de497551bc0a41b695ba92f9b4acbd50"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:52.513465   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:55:52.538750   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:55:52.540264   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:55:52.541776   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:55:52.544772   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:55:52.545083   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.545432   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:55:52.545637   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:52.596911   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36937
I20260709 14:55:52.597025 11813 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36937 every 8 connection(s)
I20260709 14:55:52.613271 11814 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34669
I20260709 14:55:52.613755 11814 heartbeater.cc:461] Registering TS with master...
I20260709 14:55:52.614598 11814 heartbeater.cc:507] Master 127.0.107.254:34669 requested a full tablet report, sending...
I20260709 14:55:52.616997 11538 ts_manager.cc:194] Registered new tserver with Master: de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:52.618023   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017225618s
I20260709 14:55:52.618888 11538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56648
I20260709 14:55:52.649801 11538 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56656:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:55:52.653390 11538 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:55:52.748967 11630 tablet_service.cc:1511] Processing CreateTablet for tablet d15599bec1e24e4cba2c627b79c6ef69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:55:52.749351 11629 tablet_service.cc:1511] Processing CreateTablet for tablet 76202e9e3d414285b5b5e74e7c3cfcf1 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:55:52.750927 11630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15599bec1e24e4cba2c627b79c6ef69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.753790 11705 tablet_service.cc:1511] Processing CreateTablet for tablet d15599bec1e24e4cba2c627b79c6ef69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:55:52.755507 11705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15599bec1e24e4cba2c627b79c6ef69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.757112 11702 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc594ae471945018e876494c15c1bca (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:55:52.758940 11702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc594ae471945018e876494c15c1bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.758548 11703 tablet_service.cc:1511] Processing CreateTablet for tablet eb65804b71684b1088e2fa50565ec42e (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:55:52.760288 11703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb65804b71684b1088e2fa50565ec42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.763396 11701 tablet_service.cc:1511] Processing CreateTablet for tablet 208124883986430b9f4fe7a186a88f69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:55:52.764228 11704 tablet_service.cc:1511] Processing CreateTablet for tablet 76202e9e3d414285b5b5e74e7c3cfcf1 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:55:52.765188 11701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208124883986430b9f4fe7a186a88f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.769454 11700 tablet_service.cc:1511] Processing CreateTablet for tablet cfe48a0dc399431c9c8aa8f7fbd37649 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:55:52.771214 11629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76202e9e3d414285b5b5e74e7c3cfcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.770020 11704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76202e9e3d414285b5b5e74e7c3cfcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.772957 11779 tablet_service.cc:1511] Processing CreateTablet for tablet d15599bec1e24e4cba2c627b79c6ef69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 14:55:52.774662 11779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15599bec1e24e4cba2c627b79c6ef69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.775301 11700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe48a0dc399431c9c8aa8f7fbd37649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.780314 11699 tablet_service.cc:1511] Processing CreateTablet for tablet aabf8384894d446794ad27fc47b3e006 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:55:52.780763 11628 tablet_service.cc:1511] Processing CreateTablet for tablet eb65804b71684b1088e2fa50565ec42e (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:55:52.782327 11699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aabf8384894d446794ad27fc47b3e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.782338 11628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb65804b71684b1088e2fa50565ec42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.783550 11627 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc594ae471945018e876494c15c1bca (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:55:52.785310 11627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc594ae471945018e876494c15c1bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.788080 11625 tablet_service.cc:1511] Processing CreateTablet for tablet cfe48a0dc399431c9c8aa8f7fbd37649 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:55:52.792181 11622 tablet_service.cc:1511] Processing CreateTablet for tablet 3efe74b6b60446349bd2e3d11a24f749 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:55:52.792537 11698 tablet_service.cc:1511] Processing CreateTablet for tablet 2d82a650a0cf40d79b65b353285d3ca9 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:55:52.794175 11698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d82a650a0cf40d79b65b353285d3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.794529 11697 tablet_service.cc:1511] Processing CreateTablet for tablet 3efe74b6b60446349bd2e3d11a24f749 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:55:52.795044 11623 tablet_service.cc:1511] Processing CreateTablet for tablet 2d82a650a0cf40d79b65b353285d3ca9 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:55:52.796367 11697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efe74b6b60446349bd2e3d11a24f749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.796485 11626 tablet_service.cc:1511] Processing CreateTablet for tablet 208124883986430b9f4fe7a186a88f69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:55:52.798146 11624 tablet_service.cc:1511] Processing CreateTablet for tablet aabf8384894d446794ad27fc47b3e006 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:55:52.800879 11696 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdad235fefd47d299dfc68992124b27 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:55:52.802582 11696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdad235fefd47d299dfc68992124b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.804397 11778 tablet_service.cc:1511] Processing CreateTablet for tablet 76202e9e3d414285b5b5e74e7c3cfcf1 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 14:55:52.806358 11778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76202e9e3d414285b5b5e74e7c3cfcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.807289 11626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208124883986430b9f4fe7a186a88f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.811378 11623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d82a650a0cf40d79b65b353285d3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.815541 11622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efe74b6b60446349bd2e3d11a24f749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.819370 11624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aabf8384894d446794ad27fc47b3e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.822726 11625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe48a0dc399431c9c8aa8f7fbd37649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.824828 11829 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:52.824395 11777 tablet_service.cc:1511] Processing CreateTablet for tablet eb65804b71684b1088e2fa50565ec42e (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 14:55:52.826167 11777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb65804b71684b1088e2fa50565ec42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.827165 11776 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc594ae471945018e876494c15c1bca (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 14:55:52.828842 11776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc594ae471945018e876494c15c1bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.836962 11775 tablet_service.cc:1511] Processing CreateTablet for tablet 208124883986430b9f4fe7a186a88f69 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 14:55:52.840510 11621 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdad235fefd47d299dfc68992124b27 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:55:52.842352 11621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdad235fefd47d299dfc68992124b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.843133 11775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208124883986430b9f4fe7a186a88f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.873332 11829 tablet_bootstrap.cc:654] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:52.884114 11774 tablet_service.cc:1511] Processing CreateTablet for tablet cfe48a0dc399431c9c8aa8f7fbd37649 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 14:55:52.886263 11774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfe48a0dc399431c9c8aa8f7fbd37649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.913897 11773 tablet_service.cc:1511] Processing CreateTablet for tablet aabf8384894d446794ad27fc47b3e006 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 14:55:52.917773 11772 tablet_service.cc:1511] Processing CreateTablet for tablet 2d82a650a0cf40d79b65b353285d3ca9 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 14:55:52.923786 11770 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdad235fefd47d299dfc68992124b27 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 14:55:52.920832 11771 tablet_service.cc:1511] Processing CreateTablet for tablet 3efe74b6b60446349bd2e3d11a24f749 (DEFAULT_TABLE table=test-workload [id=dea8db676a2048f3b9e1f0d2c3d34af0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 14:55:52.935608 11773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aabf8384894d446794ad27fc47b3e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.951967 11771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efe74b6b60446349bd2e3d11a24f749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.954689 11772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d82a650a0cf40d79b65b353285d3ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:52.963333 11831 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:52.967343 11770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdad235fefd47d299dfc68992124b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:55:53.005330 11832 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.014391 11829 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.015229 11829 ts_tablet_manager.cc:1403] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.191s	user 0.027s	sys 0.029s
I20260709 14:55:53.017791 11831 tablet_bootstrap.cc:654] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.018988 11832 tablet_bootstrap.cc:654] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.031213 11829 raft_consensus.cc:359] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.032230 11829 raft_consensus.cc:385] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.032706 11829 raft_consensus.cc:740] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.033926 11829 consensus_queue.cc:260] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.053516 11835 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.054143 11835 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.070150 11832 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.070974 11832 ts_tablet_manager.cc:1403] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.066s	user 0.023s	sys 0.017s
I20260709 14:55:53.076474 11832 raft_consensus.cc:359] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.077140 11831 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.077466 11832 raft_consensus.cc:385] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.077863 11831 ts_tablet_manager.cc:1403] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.116s	user 0.000s	sys 0.020s
I20260709 14:55:53.077899 11832 raft_consensus.cc:740] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.079113 11832 consensus_queue.cc:260] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.083014 11831 raft_consensus.cc:359] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.084342 11831 raft_consensus.cc:385] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.085358 11835 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.084800 11831 raft_consensus.cc:740] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.091218 11831 consensus_queue.cc:260] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.107461 11740 heartbeater.cc:499] Master 127.0.107.254:34669 was elected leader, sending a full tablet report...
I20260709 14:55:53.110311 11814 heartbeater.cc:499] Master 127.0.107.254:34669 was elected leader, sending a full tablet report...
I20260709 14:55:53.120461 11829 ts_tablet_manager.cc:1434] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.105s	user 0.015s	sys 0.022s
I20260709 14:55:53.122370 11829 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.140904 11832 ts_tablet_manager.cc:1434] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.070s	user 0.015s	sys 0.016s
I20260709 14:55:53.147360 11831 ts_tablet_manager.cc:1434] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.069s	user 0.031s	sys 0.034s
I20260709 14:55:53.147656 11832 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.147871 11665 heartbeater.cc:499] Master 127.0.107.254:34669 was elected leader, sending a full tablet report...
I20260709 14:55:53.148818 11831 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.152690 11829 tablet_bootstrap.cc:654] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.157217 11831 tablet_bootstrap.cc:654] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
W20260709 14:55:53.161718 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:55:53.162773 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
W20260709 14:55:53.165330 11679 leader_election.cc:343] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.166369 11829 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.167181 11829 ts_tablet_manager.cc:1403] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.002s
I20260709 14:55:53.170902 11829 raft_consensus.cc:359] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.171792 11829 raft_consensus.cc:385] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.172101 11829 raft_consensus.cc:740] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.172950 11829 consensus_queue.cc:260] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.175220 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.176003 11829 ts_tablet_manager.cc:1434] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
W20260709 14:55:53.177649 11681 leader_election.cc:343] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.177955 11829 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.178309 11681 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.179808 11835 raft_consensus.cc:2749] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.182989 11832 tablet_bootstrap.cc:654] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.191835 11829 tablet_bootstrap.cc:654] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.199249 11831 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.199961 11831 ts_tablet_manager.cc:1403] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.016s
I20260709 14:55:53.201555 11832 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.202308 11832 ts_tablet_manager.cc:1403] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.003s
I20260709 14:55:53.203778 11831 raft_consensus.cc:359] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.204887 11831 raft_consensus.cc:385] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.205220 11831 raft_consensus.cc:740] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.206163 11831 consensus_queue.cc:260] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.206214 11832 raft_consensus.cc:359] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.208961 11831 ts_tablet_manager.cc:1434] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:55:53.209177 11832 raft_consensus.cc:385] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.209566 11832 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.210333 11831 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.210695 11832 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.212411 11829 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.213102 11829 ts_tablet_manager.cc:1403] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.008s
I20260709 14:55:53.216519 11832 ts_tablet_manager.cc:1434] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 14:55:53.217290 11829 raft_consensus.cc:359] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.217959 11832 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.218403 11829 raft_consensus.cc:385] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.218904 11829 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.219601 11831 tablet_bootstrap.cc:654] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.220234 11829 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.226125 11829 ts_tablet_manager.cc:1434] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 14:55:53.227752 11829 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
W20260709 14:55:53.228366 11741 tablet.cc:2406] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 14:55:53.233112 11832 tablet_bootstrap.cc:654] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.235980 11829 tablet_bootstrap.cc:654] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.247155 11829 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.247751 11832 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.247928 11829 ts_tablet_manager.cc:1403] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 14:55:53.248749 11832 ts_tablet_manager.cc:1403] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.011s
I20260709 14:55:53.253374 11829 raft_consensus.cc:359] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.253952 11832 raft_consensus.cc:359] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.254776 11832 raft_consensus.cc:385] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.254406 11829 raft_consensus.cc:385] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.255211 11832 raft_consensus.cc:740] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.255484 11829 raft_consensus.cc:740] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.256204 11832 consensus_queue.cc:260] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.256522 11829 consensus_queue.cc:260] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.255611 11831 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.257737 11831 ts_tablet_manager.cc:1403] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.022s
I20260709 14:55:53.259212 11829 ts_tablet_manager.cc:1434] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 14:55:53.260628 11829 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.261533 11831 raft_consensus.cc:359] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.262423 11831 raft_consensus.cc:385] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.262755 11831 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.263856 11831 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.264336 11832 ts_tablet_manager.cc:1434] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 14:55:53.265965 11832 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.266731 11831 ts_tablet_manager.cc:1434] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 14:55:53.268093 11831 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.268885 11829 tablet_bootstrap.cc:654] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.274258 11832 tablet_bootstrap.cc:654] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.277437 11831 tablet_bootstrap.cc:654] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.285681 11829 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.286369 11829 ts_tablet_manager.cc:1403] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 14:55:53.288807 11832 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.289484 11831 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.289505 11832 ts_tablet_manager.cc:1403] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 14:55:53.289875 11829 raft_consensus.cc:359] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.290449 11831 ts_tablet_manager.cc:1403] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 14:55:53.290779 11829 raft_consensus.cc:385] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.291234 11829 raft_consensus.cc:740] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.292200 11829 consensus_queue.cc:260] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.293536 11832 raft_consensus.cc:359] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.294396 11832 raft_consensus.cc:385] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.294911 11832 raft_consensus.cc:740] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.294262 11831 raft_consensus.cc:359] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.295496 11831 raft_consensus.cc:385] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.296034 11831 raft_consensus.cc:740] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.295845 11832 consensus_queue.cc:260] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.296569 11829 ts_tablet_manager.cc:1434] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 14:55:53.298020 11829 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.298611 11832 ts_tablet_manager.cc:1434] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 14:55:53.299345 11835 raft_consensus.cc:493] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.300002 11832 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.299928 11835 raft_consensus.cc:515] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.303041 11831 consensus_queue.cc:260] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.303737 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.305769 11829 tablet_bootstrap.cc:654] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
W20260709 14:55:53.305850 11679 leader_election.cc:343] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.306387 11831 ts_tablet_manager.cc:1434] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 14:55:53.307788 11831 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.308799 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.309463 11840 raft_consensus.cc:493] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.309772 11789 raft_consensus.cc:2468] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:53.310101 11840 raft_consensus.cc:515] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.315092 11840 leader_election.cc:290] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.318665 11831 tablet_bootstrap.cc:654] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.322254 11832 tablet_bootstrap.cc:654] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.325825 11681 leader_election.cc:304] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.327852 11836 raft_consensus.cc:2804] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.328532 11836 raft_consensus.cc:493] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.328992 11836 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.335453 11835 leader_election.cc:290] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.363969 11840 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.364923 11840 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.368225 11840 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.369719 11835 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.370627 11835 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.373780 11835 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.375998 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
W20260709 14:55:53.378947 11681 leader_election.cc:343] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.405046 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
W20260709 14:55:53.408833 11679 leader_election.cc:343] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.409693 11679 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.411020 11835 raft_consensus.cc:2749] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.424634 11836 raft_consensus.cc:515] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.430029 11832 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.431200 11832 ts_tablet_manager.cc:1403] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.131s	user 0.017s	sys 0.052s
I20260709 14:55:53.436555 11832 raft_consensus.cc:359] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.437772 11832 raft_consensus.cc:385] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.438202 11832 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.439571 11832 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.443441 11832 ts_tablet_manager.cc:1434] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 14:55:53.445325 11832 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.456502 11832 tablet_bootstrap.cc:654] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.461364 11840 raft_consensus.cc:493] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.461403 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:55:53.462359 11840 raft_consensus.cc:515] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
W20260709 14:55:53.463896 11679 leader_election.cc:343] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.464915 11840 leader_election.cc:290] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.466085 11837 raft_consensus.cc:493] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.466722 11837 raft_consensus.cc:515] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.469544 11836 leader_election.cc:290] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.472424 11829 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.473178 11829 ts_tablet_manager.cc:1403] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.175s	user 0.066s	sys 0.096s
I20260709 14:55:53.474291 11786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.475471 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.476259 11787 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.477176 11829 raft_consensus.cc:359] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.478055 11837 leader_election.cc:290] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:53.478108 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.478111 11829 raft_consensus.cc:385] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.479873 11829 raft_consensus.cc:740] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.480805 11829 consensus_queue.cc:260] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.482748 11829 ts_tablet_manager.cc:1434] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 14:55:53.484028 11829 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.474575 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.487386 11788 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:53.491672 11715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.494258 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.495672 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
W20260709 14:55:53.496656 11605 leader_election.cc:343] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.496774 11713 raft_consensus.cc:2468] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
W20260709 14:55:53.498703 11606 leader_election.cc:343] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:55:53.498986 11605 leader_election.cc:343] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:55:53.499576 11606 leader_election.cc:343] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.500080 11606 leader_election.cc:304] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e; no voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.500195 11605 leader_election.cc:304] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.501824 11837 raft_consensus.cc:493] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.494266 11829 tablet_bootstrap.cc:654] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.502493 11837 raft_consensus.cc:515] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.505669 11840 raft_consensus.cc:2749] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.506568 11840 raft_consensus.cc:2804] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.507120 11840 raft_consensus.cc:493] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.507503 11840 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.509579 11787 raft_consensus.cc:2468] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:53.510386 11832 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.511524 11832 ts_tablet_manager.cc:1403] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.067s	user 0.032s	sys 0.017s
I20260709 14:55:53.506681 11831 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.515450 11831 ts_tablet_manager.cc:1403] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.208s	user 0.052s	sys 0.062s
I20260709 14:55:53.515600 11681 leader_election.cc:304] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.517320 11836 raft_consensus.cc:2804] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.517930 11836 raft_consensus.cc:697] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:55:53.518955 11836 consensus_queue.cc:237] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.521618 11832 raft_consensus.cc:359] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.523028 11832 raft_consensus.cc:385] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.525110 11837 leader_election.cc:290] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:53.525746 11606 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3
I20260709 14:55:53.525424 11831 raft_consensus.cc:359] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.528759 11835 raft_consensus.cc:493] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.528965 11861 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.529732 11861 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.529492 11835 raft_consensus.cc:515] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.530404 11861 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.533711 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.534356 11829 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.528230 11831 raft_consensus.cc:385] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.523435 11832 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.535104 11831 raft_consensus.cc:740] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
W20260709 14:55:53.536235 11679 leader_election.cc:343] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.535104 11829 ts_tablet_manager.cc:1403] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.023s
I20260709 14:55:53.539335 11835 leader_election.cc:290] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.535876 11832 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.535976 11831 consensus_queue.cc:260] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.539664 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.540807 11787 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:53.540647 11829 raft_consensus.cc:359] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.541550 11829 raft_consensus.cc:385] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.541893 11829 raft_consensus.cc:740] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.541986 11832 ts_tablet_manager.cc:1434] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 14:55:53.542644 11681 leader_election.cc:304] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.542862 11829 consensus_queue.cc:260] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.543558 11832 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.544272 11835 raft_consensus.cc:2804] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.544754 11835 raft_consensus.cc:493] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.545199 11835 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.575701 11832 tablet_bootstrap.cc:654] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.575937 11861 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.581121 11840 raft_consensus.cc:515] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.585789 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:53.586539 11713 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.587277 11835 raft_consensus.cc:515] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.590795 11835 leader_election.cc:290] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.591439 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.592135 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.592545 11788 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:55:53.594077 11606 leader_election.cc:343] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.595824 11861 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.597638 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.598455 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:53.598647 11789 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.597054 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e"
W20260709 14:55:53.601132 11679 leader_election.cc:343] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:55:53.602166 11605 leader_election.cc:343] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.597370 11840 leader_election.cc:290] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.604974 11715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.605783 11715 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 0.
I20260709 14:55:53.604346 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.608397 11714 raft_consensus.cc:2393] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de497551bc0a41b695ba92f9b4acbd50 in current term 1: Already voted for candidate 139311dab688423789a921ffdc3ac2d3 in this term.
I20260709 14:55:53.609453 11829 ts_tablet_manager.cc:1434] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.052s
I20260709 14:55:53.611109 11829 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.611339 11831 ts_tablet_manager.cc:1434] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.095s	user 0.015s	sys 0.008s
I20260709 14:55:53.616158 11831 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.618937 11754 leader_election.cc:304] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:53.620076 11837 raft_consensus.cc:2804] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.620503 11837 raft_consensus.cc:493] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.621901 11837 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.624271 11538 catalog_manager.cc:5697] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:53.635228 11713 raft_consensus.cc:2468] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:53.637542 11605 leader_election.cc:304] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.641712 11868 raft_consensus.cc:493] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.642409 11868 raft_consensus.cc:515] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.645218 11868 leader_election.cc:290] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:53.645854 11868 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.646373 11868 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.647078 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.647959 11829 tablet_bootstrap.cc:654] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.649030 11868 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817)
I20260709 14:55:53.634085 11865 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.650802 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.653388 11865 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.656385 11865 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.656594 11789 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:53.657487 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.657918 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.658391 11714 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:53.639478 11840 raft_consensus.cc:2804] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.659428 11840 raft_consensus.cc:697] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Becoming Leader. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Running, Role: LEADER
I20260709 14:55:53.660099 11832 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.659926 11605 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:53.660853 11832 ts_tablet_manager.cc:1403] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.025s
I20260709 14:55:53.661031 11865 raft_consensus.cc:2804] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.661463 11865 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:55:53.661633 11754 leader_election.cc:343] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.661857 11865 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.660528 11840 consensus_queue.cc:237] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.663450 11640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.664610 11832 raft_consensus.cc:359] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.665103 11639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.665503 11832 raft_consensus.cc:385] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.665863 11832 raft_consensus.cc:740] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.665939 11639 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 0.
I20260709 14:55:53.666858 11837 raft_consensus.cc:515] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.667537 11638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
W20260709 14:55:53.667747 11754 leader_election.cc:343] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.668691 11832 consensus_queue.cc:260] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.669312 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.672683 11788 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:53.673854 11832 ts_tablet_manager.cc:1434] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 14:55:53.674279 11681 leader_election.cc:304] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.675174 11832 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
W20260709 14:55:53.675503 11753 leader_election.cc:343] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.676066 11753 leader_election.cc:304] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e
W20260709 14:55:53.678146 11753 leader_election.cc:343] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.678418 11836 raft_consensus.cc:2804] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.679064 11867 raft_consensus.cc:2749] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.679344 11836 raft_consensus.cc:697] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:55:53.679539 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:55:53.680410 11637 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.680531 11836 consensus_queue.cc:237] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.669595 11837 leader_election.cc:290] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:53.671146 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:53.685256 11832 tablet_bootstrap.cc:654] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.689579 11831 tablet_bootstrap.cc:654] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.674660 11606 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3
W20260709 14:55:53.696561 11606 leader_election.cc:343] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.698223 11861 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.678676 11753 leader_election.cc:304] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.689554 11713 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:55:53.703245 11753 leader_election.cc:343] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.715124 11753 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.711421 11837 raft_consensus.cc:2749] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.714067 11861 raft_consensus.cc:697] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Becoming Leader. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Running, Role: LEADER
I20260709 14:55:53.718518 11868 raft_consensus.cc:2749] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.729264 11864 raft_consensus.cc:493] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.730091 11864 raft_consensus.cc:515] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.733692 11639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.734560 11639 raft_consensus.cc:2468] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:53.736161 11679 leader_election.cc:304] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:53.741122 11835 raft_consensus.cc:2804] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.741571 11835 raft_consensus.cc:493] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.741652 11829 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.741988 11835 raft_consensus.cc:3060] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.742301 11829 ts_tablet_manager.cc:1403] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.132s	user 0.025s	sys 0.013s
I20260709 14:55:53.744199 11865 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.742364 11832 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.745853 11832 ts_tablet_manager.cc:1403] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.021s
I20260709 14:55:53.723150 11861 consensus_queue.cc:237] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.753348 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:53.754305 11714 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.754418 11829 raft_consensus.cc:359] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.757860 11829 raft_consensus.cc:385] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.758330 11829 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.749738 11832 raft_consensus.cc:359] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.759603 11832 raft_consensus.cc:385] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.760025 11832 raft_consensus.cc:740] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.763226 11832 consensus_queue.cc:260] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.761152 11829 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.753352 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.768659 11788 raft_consensus.cc:2468] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:53.756251 11864 leader_election.cc:290] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.735766 11637 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:53.744253 11713 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:53.776058 11832 ts_tablet_manager.cc:1434] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.009s
I20260709 14:55:53.803007 11832 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.774766 11835 raft_consensus.cc:515] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.781222 11753 leader_election.cc:304] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:53.810214 11829 ts_tablet_manager.cc:1434] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.067s	user 0.015s	sys 0.011s
I20260709 14:55:53.811753 11829 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.799376 11714 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:53.813311 11837 raft_consensus.cc:2804] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.814057 11837 raft_consensus.cc:697] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:55:53.815167 11837 consensus_queue.cc:237] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.819957 11832 tablet_bootstrap.cc:654] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.820714 11829 tablet_bootstrap.cc:654] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.819103 11865 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.820405 11605 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:53.822350 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.824404 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:55:53.825181 11637 raft_consensus.cc:3060] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.825392 11835 leader_election.cc:290] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.824141 11536 catalog_manager.cc:5697] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89721e000fcb47e8a347857c99e1262e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "89721e000fcb47e8a347857c99e1262e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
W20260709 14:55:53.831166 11606 leader_election.cc:343] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:55:53.833666 11861 raft_consensus.cc:2804] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.833679 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208124883986430b9f4fe7a186a88f69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:53.834750 11861 raft_consensus.cc:697] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Becoming Leader. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Running, Role: LEADER
I20260709 14:55:53.834942 11788 raft_consensus.cc:3060] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.835851 11861 consensus_queue.cc:237] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.836426 11831 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.838790 11831 ts_tablet_manager.cc:1403] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.222s	user 0.026s	sys 0.025s
I20260709 14:55:53.846284 11831 raft_consensus.cc:359] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.847558 11831 raft_consensus.cc:385] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.853356 11831 raft_consensus.cc:740] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.854399 11831 consensus_queue.cc:260] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.856137 11637 raft_consensus.cc:2468] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:53.857106 11831 ts_tablet_manager.cc:1434] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 14:55:53.858101 11832 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:53.858753 11832 ts_tablet_manager.cc:1403] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.001s
I20260709 14:55:53.859431 11679 leader_election.cc:304] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:53.861162 11835 raft_consensus.cc:2804] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:53.861920 11835 raft_consensus.cc:697] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:55:53.863345 11835 consensus_queue.cc:237] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.864591 11831 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.867045 11832 raft_consensus.cc:359] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.868149 11832 raft_consensus.cc:385] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.872434 11832 raft_consensus.cc:740] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.872283 11788 raft_consensus.cc:2468] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:53.877460 11832 consensus_queue.cc:260] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.878537 11831 tablet_bootstrap.cc:654] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.872572 11829 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.879801 11829 ts_tablet_manager.cc:1403] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.068s	user 0.004s	sys 0.029s
I20260709 14:55:53.884858 11829 raft_consensus.cc:359] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.885787 11832 ts_tablet_manager.cc:1434] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.003s
I20260709 14:55:53.886147 11829 raft_consensus.cc:385] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.890319 11829 raft_consensus.cc:740] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.891629 11832 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: Bootstrap starting.
I20260709 14:55:53.891752 11829 consensus_queue.cc:260] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.896178 11829 ts_tablet_manager.cc:1434] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.002s
I20260709 14:55:53.897882 11829 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: Bootstrap starting.
I20260709 14:55:53.903419 11837 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.904392 11837 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.908061 11829 tablet_bootstrap.cc:654] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.911119 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.912379 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.913673 11714 raft_consensus.cc:2393] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de497551bc0a41b695ba92f9b4acbd50 in current term 1: Already voted for candidate 89721e000fcb47e8a347857c99e1262e in this term.
I20260709 14:55:53.916196 11754 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de497551bc0a41b695ba92f9b4acbd50; no voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:53.917930 11868 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.928375 11829 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: No bootstrap required, opened a new log
I20260709 14:55:53.929387 11829 ts_tablet_manager.cc:1403] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.006s
I20260709 14:55:53.930423 11837 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:53.933854 11829 raft_consensus.cc:359] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.935375 11829 raft_consensus.cc:385] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.935773 11829 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.936723 11829 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.940562 11829 ts_tablet_manager.cc:1434] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 14:55:53.945139 11835 consensus_queue.cc:1048] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:53.962525 11868 raft_consensus.cc:2749] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:53.963284 11861 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.964164 11861 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.965595 11835 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:53.966662 11832 tablet_bootstrap.cc:654] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.966262 11835 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.968317 11831 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:53.969028 11831 ts_tablet_manager.cc:1403] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.105s	user 0.041s	sys 0.054s
I20260709 14:55:53.969153 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.969980 11789 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:53.974277 11831 raft_consensus.cc:359] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.975534 11831 raft_consensus.cc:385] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:53.975618 11789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:53.975982 11831 raft_consensus.cc:740] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:53.976568 11789 raft_consensus.cc:2393] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 139311dab688423789a921ffdc3ac2d3 in current term 1: Already voted for candidate 89721e000fcb47e8a347857c99e1262e in this term.
I20260709 14:55:53.977022 11831 consensus_queue.cc:260] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:53.974443 11606 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:53.979177 11865 raft_consensus.cc:2804] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:53.979632 11865 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:53.980101 11865 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.980664 11835 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.981387 11831 ts_tablet_manager.cc:1434] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.003s
I20260709 14:55:53.984532 11831 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:53.986088 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:53.988919 11679 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:53.991447 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:53.992589 11877 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:53.995450 11831 tablet_bootstrap.cc:654] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:53.999043 11861 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:53.992254 11714 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:54.003432 11877 raft_consensus.cc:2749] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 14:55:54.012892 11679 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:54.014991 11536 catalog_manager.cc:5697] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89721e000fcb47e8a347857c99e1262e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "89721e000fcb47e8a347857c99e1262e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.016227 11865 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.017225 11536 catalog_manager.cc:5697] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89721e000fcb47e8a347857c99e1262e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "89721e000fcb47e8a347857c99e1262e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.019982 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:54.020674 11714 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.030164 11714 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:54.032828 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:54.034159 11787 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.033499 11865 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.040010 11831 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:54.040505 11605 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:54.040776 11831 ts_tablet_manager.cc:1403] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.057s	user 0.027s	sys 0.018s
I20260709 14:55:54.041679 11865 raft_consensus.cc:2804] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:54.042627 11865 raft_consensus.cc:697] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Becoming Leader. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Running, Role: LEADER
I20260709 14:55:54.041839 11537 catalog_manager.cc:5697] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:54.043771 11865 consensus_queue.cc:237] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.044651 11831 raft_consensus.cc:359] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.045469 11831 raft_consensus.cc:385] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:54.045763 11831 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:54.046002 11787 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:54.046716 11831 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.049335 11831 ts_tablet_manager.cc:1434] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:55:54.049968 11832 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: No bootstrap required, opened a new log
I20260709 14:55:54.050504 11831 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:54.050657 11832 ts_tablet_manager.cc:1403] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: Time spent bootstrapping tablet: real 0.159s	user 0.039s	sys 0.042s
I20260709 14:55:54.054188 11832 raft_consensus.cc:359] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.055063 11832 raft_consensus.cc:385] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:54.055384 11832 raft_consensus.cc:740] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Initialized, Role: FOLLOWER
I20260709 14:55:54.056118 11861 consensus_queue.cc:1048] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.058055 11831 tablet_bootstrap.cc:654] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:54.063048 11538 catalog_manager.cc:5697] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.056133 11832 consensus_queue.cc:260] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.065259 11538 catalog_manager.cc:5697] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.068958 11832 ts_tablet_manager.cc:1434] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 14:55:54.086638 11831 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:54.087728 11831 ts_tablet_manager.cc:1403] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.003s
I20260709 14:55:54.091578 11831 raft_consensus.cc:359] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.092448 11831 raft_consensus.cc:385] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:54.092808 11831 raft_consensus.cc:740] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:54.093895 11831 consensus_queue.cc:260] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.096616 11831 ts_tablet_manager.cc:1434] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:55:54.103839 11831 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:54.106994 11538 catalog_manager.cc:5697] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89721e000fcb47e8a347857c99e1262e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "89721e000fcb47e8a347857c99e1262e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.112775 11831 tablet_bootstrap.cc:654] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Neither blocks nor log segments found. Creating new log.
I20260709 14:55:54.116283 11787 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.129271 11840 consensus_queue.cc:1048] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.134418 11831 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: No bootstrap required, opened a new log
I20260709 14:55:54.135195 11831 ts_tablet_manager.cc:1403] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.007s
I20260709 14:55:54.138298 11831 raft_consensus.cc:359] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.139216 11831 raft_consensus.cc:385] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:55:54.139532 11831 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:54.140425 11840 raft_consensus.cc:493] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:54.141201 11840 raft_consensus.cc:515] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.140480 11831 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.143901 11831 ts_tablet_manager.cc:1434] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:55:54.144330 11840 leader_election.cc:290] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.144501 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:54.144860 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:54.146607 11605 leader_election.cc:304] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e; no voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:54.147496 11840 raft_consensus.cc:2749] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:55:54.156491 11840 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:54.157071 11840 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.160166 11840 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.160526 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:54.160905 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:54.161517 11714 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:54.161589 11787 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 0.
I20260709 14:55:54.162961 11605 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:54.163895 11840 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:54.164250 11840 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:54.164548 11840 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:55:54.164618 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:55:54.171017 11840 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.172991 11840 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.174259 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:54.174237 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "89721e000fcb47e8a347857c99e1262e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:54.175313 11787 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.175520 11714 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.184338 11787 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:54.184338 11714 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89721e000fcb47e8a347857c99e1262e in term 1.
I20260709 14:55:54.186025 11606 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:54.186878 11840 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:54.187433 11840 raft_consensus.cc:697] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Becoming Leader. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Running, Role: LEADER
I20260709 14:55:54.188369 11840 consensus_queue.cc:237] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.191725 11836 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.198606 11538 catalog_manager.cc:5697] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89721e000fcb47e8a347857c99e1262e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "89721e000fcb47e8a347857c99e1262e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.199611 11836 raft_consensus.cc:493] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:54.200359 11836 raft_consensus.cc:515] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.203794 11836 leader_election.cc:290] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.205298 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:54.205778 11639 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.206295 11788 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:54.207666 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e" is_pre_election: true
I20260709 14:55:54.208603 11637 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 0.
I20260709 14:55:54.210320 11681 leader_election.cc:304] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:54.211367 11877 raft_consensus.cc:2804] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:55:54.211769 11877 raft_consensus.cc:493] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:54.212121 11877 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.219877 11836 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.224118 11877 raft_consensus.cc:515] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.225705 11840 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.228024 11877 leader_election.cc:290] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:54.228955 11637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:55:54.229660 11637 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.229529 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:54.230605 11787 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.238181 11637 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:54.243252 11679 leader_election.cc:304] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:55:54.248499 11787 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:54.248942 11877 raft_consensus.cc:2804] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:55:54.249629 11877 raft_consensus.cc:697] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:55:54.250689 11877 consensus_queue.cc:237] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:54.256039 11861 consensus_queue.cc:1048] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.257102 11865 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.276244 11536 catalog_manager.cc:5697] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:54.285792 11873 consensus_queue.cc:1048] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.312850 11837 consensus_queue.cc:1048] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:55:54.327442 11909 consensus_queue.cc:1048] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.382468 11877 consensus_queue.cc:1048] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.395956 11864 consensus_queue.cc:1048] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.523418 11639 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:55:54.591575   431 ts_manager.cc:295] Set tserver state for 89721e000fcb47e8a347857c99e1262e to MAINTENANCE_MODE
I20260709 14:55:54.592568   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 14:55:54.616446 11873 consensus_queue.cc:1048] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.622502   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:55:54.624241   431 tablet_replica.cc:333] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.625296   431 raft_consensus.cc:2243] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:54.625988   431 raft_consensus.cc:2272] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.630492   431 tablet_replica.cc:333] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.631472   431 raft_consensus.cc:2243] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:55:54.632443   431 pending_rounds.cc:70] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Trying to abort 1 pending ops.
I20260709 14:55:54.632742   431 pending_rounds.cc:77] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305662276365721600 op_type: NO_OP noop_request { }
I20260709 14:55:54.633714   431 raft_consensus.cc:2889] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:55:54.633598 11864 consensus_queue.cc:1048] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:55:54.634356   431 raft_consensus.cc:2272] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.635494 11840 consensus_queue.cc:1048] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:55:54.638321   431 tablet_replica.cc:333] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.639485   431 raft_consensus.cc:2243] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:54.640036   431 raft_consensus.cc:2272] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.643108   431 tablet_replica.cc:333] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.644658   431 raft_consensus.cc:2243] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Raft consensus shutting down.
W20260709 14:55:54.645502 11840 consensus_queue.cc:1175] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de497551bc0a41b695ba92f9b4acbd50" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 14:55:54.646629   431 raft_consensus.cc:2272] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.650549   431 tablet_replica.cc:333] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.651629   431 raft_consensus.cc:2243] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:55:54.653100   431 raft_consensus.cc:2272] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.657285   431 tablet_replica.cc:333] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
W20260709 14:55:54.657436 11679 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 258 similar messages]
I20260709 14:55:54.658414   431 raft_consensus.cc:2243] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:55:54.659979   431 raft_consensus.cc:2272] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.663296   431 tablet_replica.cc:333] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.664404   431 raft_consensus.cc:2243] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:54.665094   431 raft_consensus.cc:2272] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.668927   431 tablet_replica.cc:333] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
W20260709 14:55:54.669464 11679 consensus_peers.cc:597] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:54.669911   431 raft_consensus.cc:2243] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:54.671222   431 raft_consensus.cc:2272] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.674230   431 tablet_replica.cc:333] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.675228   431 raft_consensus.cc:2243] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:55:54.675809   431 raft_consensus.cc:2272] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.678714   431 tablet_replica.cc:333] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: stopping tablet replica
I20260709 14:55:54.679461   431 raft_consensus.cc:2243] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:55:54.680577   431 raft_consensus.cc:2272] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:55:54.707391   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:55:54.737689 11679 consensus_peers.cc:597] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:54.932801 11679 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:54.945791 11753 consensus_peers.cc:597] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:54.994076 11679 consensus_peers.cc:597] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:55:55.165856 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:55:55.763105 11923 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.763836 11923 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.765827 11923 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:55.766098 11835 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.766983 11835 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.767282 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:55.768103 11713 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:55.769806 11754 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:55.770004 11835 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:55.771375 11923 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:55:55.771910 11923 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.771763 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
W20260709 14:55:55.772298 11753 leader_election.cc:336] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.772651 11923 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:55:55.774215 11679 leader_election.cc:336] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.779618 11923 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.780795 11787 raft_consensus.cc:2393] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 139311dab688423789a921ffdc3ac2d3 in current term 2: Already voted for candidate de497551bc0a41b695ba92f9b4acbd50 in this term.
I20260709 14:55:55.781649 11923 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:55.782667 11835 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.783013 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:55.783275 11681 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:55.783776 11713 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:55:55.783375 11835 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.786671 11835 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
W20260709 14:55:55.787829 11753 leader_election.cc:336] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.788360 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:55.790892 11923 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.791527 11923 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
W20260709 14:55:55.792560 11679 leader_election.cc:336] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.793051 11713 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:55:55.793372 11679 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:55.793819 11864 raft_consensus.cc:2749] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:55:55.794422 11923 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:55.795264 11835 raft_consensus.cc:2749] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:55:55.796308 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:55.795795 11754 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:55.797309 11713 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:55.797528 11923 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:55:55.798795 11754 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:55.798229 11923 raft_consensus.cc:697] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:55:55.799712 11922 raft_consensus.cc:2804] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 14:55:55.799729 11753 leader_election.cc:336] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.800186 11922 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:55.800594 11922 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:55:55.800700 11923 consensus_queue.cc:237] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.809015 11922 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.811672 11922 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:55.810907 11537 catalog_manager.cc:5697] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 89721e000fcb47e8a347857c99e1262e (127.0.107.193) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:55.814357 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:55.815202 11713 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:55:55.816376 11753 leader_election.cc:336] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.823935 11713 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:55:55.825419 11754 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:55.826440 11922 raft_consensus.cc:2804] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:55:55.827009 11922 raft_consensus.cc:697] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:55:55.827826 11922 consensus_queue.cc:237] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.836884 11537 catalog_manager.cc:5697] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 89721e000fcb47e8a347857c99e1262e (127.0.107.193) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:55.919951 11835 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:55:55.920670 11835 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.922912 11835 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:55.924437 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
I20260709 14:55:55.925282 11787 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 1.
I20260709 14:55:55.926684 11681 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:55.927726 11835 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 14:55:55.928185 11679 leader_election.cc:336] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.928238 11835 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:55:55.928999 11835 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:55:55.935390 11835 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.937450 11835 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:55.939124 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:55:55.940002 11787 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:55:55.942109 11679 leader_election.cc:336] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:55.947041 11787 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 2.
I20260709 14:55:55.948504 11681 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:55.949437 11835 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:55:55.949892 11835 raft_consensus.cc:697] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:55:55.950796 11835 consensus_queue.cc:237] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:55.960100 11537 catalog_manager.cc:5697] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 1 to 2, leader changed from 89721e000fcb47e8a347857c99e1262e (127.0.107.193) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:55:56.073592 11864 raft_consensus.cc:493] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:56.074137 11864 raft_consensus.cc:515] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.076278 11864 leader_election.cc:290] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:56.077764 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
W20260709 14:55:56.080912 11679 leader_election.cc:336] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:56.081359 11679 leader_election.cc:304] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:56.082090 11864 raft_consensus.cc:2749] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:55:56.083081 11922 raft_consensus.cc:493] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:56.083576 11922 raft_consensus.cc:515] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.085363 11922 leader_election.cc:290] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:56.086633 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:56.087503 11713 raft_consensus.cc:2468] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:56.088822 11754 leader_election.cc:304] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:56.089716 11922 raft_consensus.cc:2804] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 14:55:56.090116 11922 raft_consensus.cc:493] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
W20260709 14:55:56.090247 11753 leader_election.cc:336] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:56.090585 11922 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:55:56.097767 11922 raft_consensus.cc:515] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.100041 11922 leader_election.cc:290] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:56.101311 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efe74b6b60446349bd2e3d11a24f749" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:56.102082 11713 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:55:56.104502 11753 leader_election.cc:336] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:56.109149 11713 raft_consensus.cc:2468] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:55:56.110524 11754 leader_election.cc:304] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:56.111508 11922 raft_consensus.cc:2804] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:55:56.111975 11922 raft_consensus.cc:697] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:55:56.112874 11922 consensus_queue.cc:237] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.121291 11537 catalog_manager.cc:5697] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 89721e000fcb47e8a347857c99e1262e (127.0.107.193) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:55:56.132593 11864 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:56.133188 11864 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.135061 11864 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:55:56.136633 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de497551bc0a41b695ba92f9b4acbd50" is_pre_election: true
W20260709 14:55:56.139475 11679 leader_election.cc:336] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:56.139964 11679 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3; no voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50
I20260709 14:55:56.140796 11835 raft_consensus.cc:2749] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:55:56.141892 11922 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:56.142562 11922 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.144585 11922 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:56.145936 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3" is_pre_election: true
I20260709 14:55:56.146790 11713 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 1.
I20260709 14:55:56.147953 11754 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:55:56.148699 11922 raft_consensus.cc:2804] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 14:55:56.149035 11753 leader_election.cc:336] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
I20260709 14:55:56.149158 11922 raft_consensus.cc:493] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 89721e000fcb47e8a347857c99e1262e)
I20260709 14:55:56.149834 11922 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:55:56.156507 11922 raft_consensus.cc:515] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.159291 11922 leader_election.cc:290] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:55:56.160643 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76202e9e3d414285b5b5e74e7c3cfcf1" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:55:56.161476 11713 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 14:55:56.164207 11753 leader_election.cc:336] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111)
W20260709 14:55:56.167255 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:55:56.169266 11713 raft_consensus.cc:2468] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:55:56.170756 11754 leader_election.cc:304] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 89721e000fcb47e8a347857c99e1262e
I20260709 14:55:56.171680 11922 raft_consensus.cc:2804] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:55:56.172251 11922 raft_consensus.cc:697] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:55:56.173261 11922 consensus_queue.cc:237] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:56.183316 11537 catalog_manager.cc:5697] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 89721e000fcb47e8a347857c99e1262e (127.0.107.193) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
W20260709 14:55:56.237264 11753 consensus_peers.cc:597] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:56.248468 11713 raft_consensus.cc:1275] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:55:56.250316 11922 consensus_queue.cc:1048] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:55:56.266613 11753 consensus_peers.cc:597] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:56.279224 11713 raft_consensus.cc:1275] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:55:56.280874 11922 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:55:56.382682 11679 consensus_peers.cc:597] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:56.493840 11788 raft_consensus.cc:1275] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 14:55:56.495749 11835 consensus_queue.cc:1048] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 14:55:56.528949 11753 consensus_peers.cc:597] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:56.571125 11713 raft_consensus.cc:1275] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:55:56.572794 11937 consensus_queue.cc:1048] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:55:56.590040 11753 consensus_peers.cc:597] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:55:56.633623 11713 raft_consensus.cc:1275] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:55:56.635242 11922 consensus_queue.cc:1048] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:55:57.049340 11679 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:57.168658 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 14:55:57.267834 11679 consensus_peers.cc:597] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:57.511053 11753 consensus_peers.cc:597] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:57.519035 11679 consensus_peers.cc:597] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:57.626327 11679 consensus_peers.cc:597] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:58.170285 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 14:55:58.736640 11753 consensus_peers.cc:597] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:58.753813 11753 consensus_peers.cc:597] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:59.046021 11679 consensus_peers.cc:597] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:59.140594 11753 consensus_peers.cc:597] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:59.143759 11753 consensus_peers.cc:597] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 14:55:59.171592 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 14:55:59.628258 11679 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 14:55:59.668830 11753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111) [suppressed 95 similar messages]
I20260709 14:55:59.733850   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:55:59.743021 11948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:59.744078 11949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:55:59.745117 11951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:55:59.748581   431 server_base.cc:1061] running on GCE node
I20260709 14:55:59.750082   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:55:59.750412   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:55:59.750675   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608959750652 us; error 0 us; skew 500 ppm
I20260709 14:55:59.751556   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:55:59.756057   431 webserver.cc:533] Webserver started at http://127.0.107.193:35225/ using document root <none> and password file <none>
I20260709 14:55:59.756814   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:55:59.757063   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 14:55:59.762039   431 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 14:55:59.767271 11956 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:55:59.768254   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 14:55:59.768651   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "89721e000fcb47e8a347857c99e1262e"
format_stamp: "Formatted at 2026-07-09 14:55:52 on dist-test-slave-wg8n"
I20260709 14:55:59.769135   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:55:59.781136   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:55:59.782769   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:55:59.784685   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:55:59.789952 11963 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 14:55:59.806275 11679 consensus_peers.cc:597] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 14:55:59.866194   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 14:55:59.866663   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.078s	user 0.003s	sys 0.001s
I20260709 14:55:59.867069   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 14:55:59.874439 11963 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:59.892261 11963 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:55:59.893338 11963 tablet_bootstrap.cc:492] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:55:59.894234 11963 ts_tablet_manager.cc:1403] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 14:55:59.897827 11963 raft_consensus.cc:359] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.899068 11963 raft_consensus.cc:740] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:59.900053 11963 consensus_queue.cc:260] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.903677 11963 ts_tablet_manager.cc:1434] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.001s
I20260709 14:55:59.904771 11963 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:59.924808 11963 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:55:59.926090 11963 tablet_bootstrap.cc:492] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:55:59.926975 11963 ts_tablet_manager.cc:1403] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.022s	user 0.021s	sys 0.001s
I20260709 14:55:59.930142 11963 raft_consensus.cc:359] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.930959 11963 raft_consensus.cc:740] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:59.931715 11963 consensus_queue.cc:260] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.933940 11963 ts_tablet_manager.cc:1434] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 14:55:59.934973 11963 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:59.947039   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 14:55:59.947532   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.080s	user 0.077s	sys 0.003s
I20260709 14:55:59.954949 11963 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:55:59.956403 11963 tablet_bootstrap.cc:492] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:55:59.957417 11963 ts_tablet_manager.cc:1403] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 14:55:59.961443 11963 raft_consensus.cc:359] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.962718 11963 raft_consensus.cc:740] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:59.964207 11963 consensus_queue.cc:260] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.967659 11963 ts_tablet_manager.cc:1434] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.004s
I20260709 14:55:59.969080 11963 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:55:59.991499 11963 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:55:59.992805 11963 tablet_bootstrap.cc:492] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:55:59.993829 11963 ts_tablet_manager.cc:1403] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.001s
I20260709 14:55:59.996428 11963 raft_consensus.cc:359] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:55:59.997365 11963 raft_consensus.cc:740] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:55:59.998292 11963 consensus_queue.cc:260] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.002146 11963 ts_tablet_manager.cc:1434] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:56:00.003587 11963 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:56:00.028618 11963 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:56:00.029743   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38817
I20260709 14:56:00.029847 12025 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38817 every 8 connection(s)
I20260709 14:56:00.030004 11963 tablet_bootstrap.cc:492] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.031348 11963 ts_tablet_manager.cc:1403] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 14:56:00.035173 11963 raft_consensus.cc:359] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.036247 11963 raft_consensus.cc:740] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.036862   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:00.040504 11963 consensus_queue.cc:260] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.043733 11963 ts_tablet_manager.cc:1434] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 14:56:00.044878 11963 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:56:00.064443 12026 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34669
I20260709 14:56:00.066241 12026 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:00.068506 12026 heartbeater.cc:507] Master 127.0.107.254:34669 requested a full tablet report, sending...
I20260709 14:56:00.082528 11537 ts_manager.cc:194] Re-registered known tserver with Master: 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817)
I20260709 14:56:00.087648 12000 raft_consensus.cc:3060] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.088837 11963 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 14:56:00.090310 11963 tablet_bootstrap.cc:492] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.091296 11963 ts_tablet_manager.cc:1403] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.047s	user 0.027s	sys 0.016s
I20260709 14:56:00.094924 11963 raft_consensus.cc:359] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.096078 11963 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.097005 11963 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.100821 11963 ts_tablet_manager.cc:1434] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 14:56:00.101998 11963 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:56:00.102391 11714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cc594ae471945018e876494c15c1bca"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
mode: GRACEFUL
new_leader_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:32924
I20260709 14:56:00.103415 11714 raft_consensus.cc:606] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Received request to transfer leadership to TS de497551bc0a41b695ba92f9b4acbd50
I20260709 14:56:00.111605 11537 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50620
I20260709 14:56:00.116892 12026 heartbeater.cc:499] Master 127.0.107.254:34669 was elected leader, sending a full tablet report...
I20260709 14:56:00.130215 11963 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:56:00.131659 11963 tablet_bootstrap.cc:492] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.131932 11788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
mode: GRACEFUL
new_leader_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:56724
I20260709 14:56:00.132588 11788 raft_consensus.cc:606] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Received request to transfer leadership to TS 139311dab688423789a921ffdc3ac2d3
I20260709 14:56:00.132572 11963 ts_tablet_manager.cc:1403] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 14:56:00.134645   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:00.135473   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:00.136404   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:00.136668 11963 raft_consensus.cc:359] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.137465 11963 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.138340 11963 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.140735 11963 ts_tablet_manager.cc:1434] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 14:56:00.141767 11963 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:56:00.158115 11963 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:56:00.159127 11963 tablet_bootstrap.cc:492] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.159801 11963 ts_tablet_manager.cc:1403] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 14:56:00.162215 11963 raft_consensus.cc:359] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.162793 11963 raft_consensus.cc:740] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.163470 11963 consensus_queue.cc:260] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.165314 11963 ts_tablet_manager.cc:1434] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 14:56:00.166318 11963 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
W20260709 14:56:00.172912 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:00.183175 11963 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:56:00.184128 11963 tablet_bootstrap.cc:492] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.184782 11963 ts_tablet_manager.cc:1403] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 14:56:00.186894 11963 raft_consensus.cc:359] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.187440 11963 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.188086 11963 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.189821 11963 ts_tablet_manager.cc:1434] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 14:56:00.190606 11963 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Bootstrap starting.
I20260709 14:56:00.206750 12000 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.207531 11963 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 14:56:00.208570 11963 tablet_bootstrap.cc:492] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Bootstrap complete.
I20260709 14:56:00.209244 11963 ts_tablet_manager.cc:1403] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 14:56:00.211723 11963 raft_consensus.cc:359] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.212363 11963 raft_consensus.cc:740] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721e000fcb47e8a347857c99e1262e, State: Initialized, Role: FOLLOWER
I20260709 14:56:00.213060 11963 consensus_queue.cc:260] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.213810 12000 pending_rounds.cc:85] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Aborting all ops after (but not including) 0
I20260709 14:56:00.214282 12000 pending_rounds.cc:107] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 14:56:00.214589 12000 raft_consensus.cc:2889] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 14:56:00.215164 11963 ts_tablet_manager.cc:1434] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 14:56:00.271318 12000 raft_consensus.cc:3060] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.286170 11835 raft_consensus.cc:993] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: : Instructing follower de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:00.286713 11835 raft_consensus.cc:1081] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Signalling peer de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:00.289075 11788 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cc594ae471945018e876494c15c1bca"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:40966
I20260709 14:56:00.289976 11788 raft_consensus.cc:493] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:00.290493 11788 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.300114 11788 raft_consensus.cc:515] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.302289 11788 leader_election.cc:290] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:56:00.303669 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:00.303960 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc594ae471945018e876494c15c1bca" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:56:00.304479 12000 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.304859 11714 raft_consensus.cc:3055] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:00.305286 11714 raft_consensus.cc:740] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:00.306443 11714 consensus_queue.cc:260] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.308113 11714 raft_consensus.cc:3060] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.311411 12000 raft_consensus.cc:2468] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:00.312994 11753 leader_election.cc:304] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:56:00.314090 11944 raft_consensus.cc:2804] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:00.315788 11944 raft_consensus.cc:697] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:00.316951 11944 consensus_queue.cc:237] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.317827 11714 raft_consensus.cc:2468] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:00.326413 11537 catalog_manager.cc:5697] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 139311dab688423789a921ffdc3ac2d3 (127.0.107.194) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:00.370540 12000 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.383373 12000 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:00.385485 11944 raft_consensus.cc:993] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: : Instructing follower 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:00.386351 12046 raft_consensus.cc:1081] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Signalling peer 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:00.388571 11714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:53266
I20260709 14:56:00.389334 11714 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:00.389824 11714 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:00.400276 11714 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.404276 11714 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:56:00.404439 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:00.404541 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:56:00.405267 12000 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:00.405400 11788 raft_consensus.cc:3055] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:00.405956 11788 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:00.407094 11788 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.408947 11788 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:00.413962 12000 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:00.415455 11679 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:56:00.416889 12044 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:00.417528 12044 raft_consensus.cc:697] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:00.418607 12044 consensus_queue.cc:237] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:00.419458 11788 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:00.428519 11537 catalog_manager.cc:5697] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 2 to 3, leader changed from de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:00.820098 11714 raft_consensus.cc:1275] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:00.821749 12055 consensus_queue.cc:1048] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:00.830920 11999 raft_consensus.cc:1275] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:00.832950 12057 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:56:00.839860 12000 raft_consensus.cc:1275] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:00.841907 11944 consensus_queue.cc:1048] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:00.848750 11788 raft_consensus.cc:1275] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:00.850767 12056 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:56:01.148643   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:01.157568 11788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
mode: GRACEFUL
new_leader_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:56724
I20260709 14:56:01.158313 11788 raft_consensus.cc:606] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Received request to transfer leadership to TS 139311dab688423789a921ffdc3ac2d3
I20260709 14:56:01.162213 11714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
mode: GRACEFUL
new_leader_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:32924
I20260709 14:56:01.163058 11714 raft_consensus.cc:606] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Received request to transfer leadership to TS de497551bc0a41b695ba92f9b4acbd50
I20260709 14:56:01.164809   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:01.165490   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:01.166285   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:56:01.174224 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:01.216799 12046 raft_consensus.cc:993] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: : Instructing follower 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:01.217655 11944 raft_consensus.cc:1081] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Signalling peer 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:01.221473 11714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:53266
I20260709 14:56:01.222200 11714 raft_consensus.cc:493] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:01.222601 11714 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:01.231663 11714 raft_consensus.cc:515] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.233741 11714 leader_election.cc:290] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:56:01.234752 11999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:01.235019 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d82a650a0cf40d79b65b353285d3ca9" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:56:01.235575 11999 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:01.235677 11788 raft_consensus.cc:3055] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:01.236135 11788 raft_consensus.cc:740] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:01.236987 11788 consensus_queue.cc:260] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.238202 11788 raft_consensus.cc:3060] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:01.242408 11999 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:01.243999 11679 leader_election.cc:304] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:56:01.245002 12056 raft_consensus.cc:2804] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:01.245355 11788 raft_consensus.cc:2468] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:01.245616 12056 raft_consensus.cc:697] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:01.247457 12056 consensus_queue.cc:237] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.256611 11538 catalog_manager.cc:5697] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 2 to 3, leader changed from de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:01.659747 12067 raft_consensus.cc:993] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: : Instructing follower de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:01.660347 12067 raft_consensus.cc:1081] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Signalling peer de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:01.662421 11788 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:40966
I20260709 14:56:01.663152 11788 raft_consensus.cc:493] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:01.663489 11788 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:01.667469 11787 raft_consensus.cc:1275] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:01.669366 12067 consensus_queue.cc:1048] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:56:01.671131 11788 raft_consensus.cc:515] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.673933 11788 leader_election.cc:290] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:56:01.675276 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:01.675469 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:56:01.676292 12000 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:01.676534 11714 raft_consensus.cc:3055] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:01.677090 11714 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:01.678094 11714 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.679492 11714 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:01.688661 11999 raft_consensus.cc:1275] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:01.690323 12000 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:01.691867 11714 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:01.693854 11754 leader_election.cc:304] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:56:01.694567 12056 consensus_queue.cc:1048] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:56:01.695170 11944 raft_consensus.cc:2804] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:01.695765 11944 raft_consensus.cc:697] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:01.697033 11944 consensus_queue.cc:237] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:01.708562 11538 catalog_manager.cc:5697] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 139311dab688423789a921ffdc3ac2d3 (127.0.107.194) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:02.174337   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 14:56:02.175498 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:02.185283 11787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
mode: GRACEFUL
new_leader_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:56724
I20260709 14:56:02.186040 11787 raft_consensus.cc:606] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 LEADER]: Received request to transfer leadership to TS 139311dab688423789a921ffdc3ac2d3
I20260709 14:56:02.189414 11714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
mode: GRACEFUL
new_leader_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:32924
I20260709 14:56:02.190006 11714 raft_consensus.cc:606] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Received request to transfer leadership to TS de497551bc0a41b695ba92f9b4acbd50
I20260709 14:56:02.191383   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:02.191918   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:02.192585   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:02.204533 11714 raft_consensus.cc:1275] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:02.206185 11944 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:02.223424 12000 raft_consensus.cc:1275] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:02.225117 12076 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:02.522408 12082 raft_consensus.cc:993] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: : Instructing follower de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:02.523098 12082 raft_consensus.cc:1081] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Signalling peer de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:02.524863 11787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:40966
I20260709 14:56:02.525499 11787 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:02.525911 11787 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:02.532752 11787 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.534760 11787 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 3 election: Requested vote from peers 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695), 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817)
I20260709 14:56:02.535813 11714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:56:02.536028 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:02.536589 11714 raft_consensus.cc:3055] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:02.536711 12000 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:02.537001 11714 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:02.537911 11714 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.539398 11714 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:02.539975 11944 raft_consensus.cc:993] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: : Instructing follower 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:02.540611 12076 raft_consensus.cc:1081] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 LEADER]: Signalling peer 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:02.542448 11713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:53266
I20260709 14:56:02.543102 11713 raft_consensus.cc:493] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:02.543506 11713 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:02.545719 12000 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 3.
I20260709 14:56:02.547326 11753 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:56:02.548277 12076 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:02.548385 11714 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 3.
I20260709 14:56:02.549314 12076 raft_consensus.cc:697] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:02.550534 12076 consensus_queue.cc:237] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.553507 11713 raft_consensus.cc:515] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.558735 11713 leader_election.cc:290] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:56:02.559000 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:02.559775 12000 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:02.560470 11537 catalog_manager.cc:5697] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 2 to 3, leader changed from 139311dab688423789a921ffdc3ac2d3 (127.0.107.194) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:02.564646 11787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfe48a0dc399431c9c8aa8f7fbd37649" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:56:02.565549 11787 raft_consensus.cc:3055] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:02.565936 11787 raft_consensus.cc:740] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:02.566912 11787 consensus_queue.cc:260] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.568847 11787 raft_consensus.cc:3060] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:02.569955 12000 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 2.
I20260709 14:56:02.571719 11679 leader_election.cc:304] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:56:02.572832 12082 raft_consensus.cc:2804] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:02.573544 12082 raft_consensus.cc:697] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:02.574692 12082 consensus_queue.cc:237] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:02.584021 11787 raft_consensus.cc:2468] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 2.
I20260709 14:56:02.587939 11537 catalog_manager.cc:5697] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 1 to 2, leader changed from de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:02.957856 12000 raft_consensus.cc:1275] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:02.959475 12082 consensus_queue.cc:1048] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:02.971745 11787 raft_consensus.cc:1275] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:02.974141 12083 consensus_queue.cc:1048] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:02.976058 11999 raft_consensus.cc:1275] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:56:02.979017 12089 consensus_queue.cc:1048] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:02.993904 11713 raft_consensus.cc:1275] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 14:56:02.995882 12076 consensus_queue.cc:1048] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 14:56:03.176792 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:03.202517   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:03.211990 11787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
mode: GRACEFUL
new_leader_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:56724
I20260709 14:56:03.212795 11787 raft_consensus.cc:606] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Received request to transfer leadership to TS 139311dab688423789a921ffdc3ac2d3
I20260709 14:56:03.216225 11713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
mode: GRACEFUL
new_leader_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:32924
I20260709 14:56:03.217079 11713 raft_consensus.cc:606] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Received request to transfer leadership to TS de497551bc0a41b695ba92f9b4acbd50
I20260709 14:56:03.219095   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:03.219828   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:03.220568   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:03.353432 12082 raft_consensus.cc:993] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: : Instructing follower de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:03.354094 12083 raft_consensus.cc:1081] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Signalling peer de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:03.355901 11787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:40966
I20260709 14:56:03.356544 11787 raft_consensus.cc:493] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:03.356894 11787 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:03.364169 11787 raft_consensus.cc:515] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.366089 11787 leader_election.cc:290] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 4 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:56:03.366923 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:03.367238 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb65804b71684b1088e2fa50565ec42e" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:56:03.367690 12000 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:03.367916 11713 raft_consensus.cc:3055] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:56:03.368309 11713 raft_consensus.cc:740] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:03.369104 11713 consensus_queue.cc:260] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.370208 11713 raft_consensus.cc:3060] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:03.373807 12000 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 4.
I20260709 14:56:03.375438 11753 leader_election.cc:304] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:56:03.376293 12076 raft_consensus.cc:2804] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:56:03.376818 12076 raft_consensus.cc:697] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 4 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:03.376890 11713 raft_consensus.cc:2468] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 4.
I20260709 14:56:03.377836 12076 consensus_queue.cc:237] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.387285 11538 catalog_manager.cc:5697] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 3 to 4, leader changed from 139311dab688423789a921ffdc3ac2d3 (127.0.107.194) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:03.622687 12104 raft_consensus.cc:993] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: : Instructing follower 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:03.623322 12076 raft_consensus.cc:1081] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Signalling peer 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:03.625149 11713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:53266
I20260709 14:56:03.625726 11713 raft_consensus.cc:493] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:03.626173 11713 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:03.632346 11713 raft_consensus.cc:515] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.634372 11713 leader_election.cc:290] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:56:03.635995 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:56:03.636421 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aabf8384894d446794ad27fc47b3e006" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:03.637142 12000 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:03.636901 11788 raft_consensus.cc:3055] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:03.638230 11788 raft_consensus.cc:740] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:03.639284 11788 consensus_queue.cc:260] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.640913 11788 raft_consensus.cc:3060] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:03.647135 12000 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:03.648994 11679 leader_election.cc:304] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:56:03.649916 12082 raft_consensus.cc:2804] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:03.650027 11788 raft_consensus.cc:2468] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 3.
I20260709 14:56:03.650569 12082 raft_consensus.cc:697] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:03.651667 12082 consensus_queue.cc:237] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:03.660838 11538 catalog_manager.cc:5697] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 2 to 3, leader changed from de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:03.913904 12000 raft_consensus.cc:1275] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:56:03.915578 12076 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:56:03.939265 11713 raft_consensus.cc:1275] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 14:56:03.941022 12104 consensus_queue.cc:1048] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:56:04.070781 11788 raft_consensus.cc:1275] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:04.072239 12083 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:56:04.082934 12000 raft_consensus.cc:1275] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:04.084704 12113 consensus_queue.cc:1048] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:56:04.178201 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:04.231383   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:04.239675 11713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
mode: GRACEFUL
new_leader_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:32924
I20260709 14:56:04.240358 11713 raft_consensus.cc:606] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Received request to transfer leadership to TS de497551bc0a41b695ba92f9b4acbd50
I20260709 14:56:04.243515 11788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
mode: GRACEFUL
new_leader_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:56724
I20260709 14:56:04.244092 11788 raft_consensus.cc:606] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 LEADER]: Received request to transfer leadership to TS 139311dab688423789a921ffdc3ac2d3
I20260709 14:56:04.245559   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:04.246176   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:04.246919   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:04.470105 11944 raft_consensus.cc:993] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: : Instructing follower 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:04.470709 12104 raft_consensus.cc:1081] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 LEADER]: Signalling peer 139311dab688423789a921ffdc3ac2d3 to start an election
I20260709 14:56:04.472553 11713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69"
dest_uuid: "139311dab688423789a921ffdc3ac2d3"
 from {username='slave'} at 127.0.0.1:53266
I20260709 14:56:04.473176 11713 raft_consensus.cc:493] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:04.473519 11713 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:04.479416 11713 raft_consensus.cc:515] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.481163 11713 leader_election.cc:290] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 4 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195:36937)
I20260709 14:56:04.482280 11788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
I20260709 14:56:04.482282 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15599bec1e24e4cba2c627b79c6ef69" candidate_uuid: "139311dab688423789a921ffdc3ac2d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:04.483029 11788 raft_consensus.cc:3055] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 14:56:04.483144 12000 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:04.483312 11788 raft_consensus.cc:740] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:04.484191 11788 consensus_queue.cc:260] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.485260 11788 raft_consensus.cc:3060] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 14:56:04.489491 12000 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 4.
I20260709 14:56:04.491015 11788 raft_consensus.cc:2468] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139311dab688423789a921ffdc3ac2d3 in term 4.
I20260709 14:56:04.491110 11679 leader_election.cc:304] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139311dab688423789a921ffdc3ac2d3, 89721e000fcb47e8a347857c99e1262e; no voters: 
I20260709 14:56:04.492219 12113 raft_consensus.cc:2804] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 14:56:04.492743 12113 raft_consensus.cc:697] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 4 LEADER]: Becoming Leader. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:04.493508 12113 consensus_queue.cc:237] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.501731 11538 catalog_manager.cc:5697] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 reported cstate change: term changed from 3 to 4, leader changed from de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195) to 139311dab688423789a921ffdc3ac2d3 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "139311dab688423789a921ffdc3ac2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:04.583003 12083 raft_consensus.cc:993] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: : Instructing follower de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:04.583590 12113 raft_consensus.cc:1081] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Signalling peer de497551bc0a41b695ba92f9b4acbd50 to start an election
I20260709 14:56:04.585392 11788 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27"
dest_uuid: "de497551bc0a41b695ba92f9b4acbd50"
 from {username='slave'} at 127.0.0.1:40966
I20260709 14:56:04.585985 11788 raft_consensus.cc:493] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:04.586289 11788 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:04.592072 11788 raft_consensus.cc:515] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.593740 11788 leader_election.cc:290] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Requested vote from peers 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817), 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695)
I20260709 14:56:04.594870 12000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89721e000fcb47e8a347857c99e1262e"
I20260709 14:56:04.594882 11713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdad235fefd47d299dfc68992124b27" candidate_uuid: "de497551bc0a41b695ba92f9b4acbd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "139311dab688423789a921ffdc3ac2d3"
I20260709 14:56:04.595561 12000 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:04.595667 11713 raft_consensus.cc:3055] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:04.596061 11713 raft_consensus.cc:740] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 139311dab688423789a921ffdc3ac2d3, State: Running, Role: LEADER
I20260709 14:56:04.596918 11713 consensus_queue.cc:260] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.598223 11713 raft_consensus.cc:3060] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:04.604106 12000 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:04.604962 11713 raft_consensus.cc:2468] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de497551bc0a41b695ba92f9b4acbd50 in term 2.
I20260709 14:56:04.605676 11753 leader_election.cc:304] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89721e000fcb47e8a347857c99e1262e, de497551bc0a41b695ba92f9b4acbd50; no voters: 
I20260709 14:56:04.606644 12104 raft_consensus.cc:2804] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:04.607167 12104 raft_consensus.cc:697] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Becoming Leader. State: Replica: de497551bc0a41b695ba92f9b4acbd50, State: Running, Role: LEADER
I20260709 14:56:04.608101 12104 consensus_queue.cc:237] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } }
I20260709 14:56:04.616175 11538 catalog_manager.cc:5697] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 reported cstate change: term changed from 1 to 2, leader changed from 139311dab688423789a921ffdc3ac2d3 (127.0.107.194) to de497551bc0a41b695ba92f9b4acbd50 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "de497551bc0a41b695ba92f9b4acbd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:04.883030 11788 raft_consensus.cc:1275] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:56:04.884502 12083 consensus_queue.cc:1048] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de497551bc0a41b695ba92f9b4acbd50" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:56:04.897220 12000 raft_consensus.cc:1275] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Refusing update from remote peer 139311dab688423789a921ffdc3ac2d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 14:56:04.899371 12083 consensus_queue.cc:1048] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 14:56:05.179616 11586 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 14:56:05.213441 12000 raft_consensus.cc:1275] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:05.215010 12104 consensus_queue.cc:1048] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721e000fcb47e8a347857c99e1262e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:05.228971 11713 raft_consensus.cc:1275] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Refusing update from remote peer de497551bc0a41b695ba92f9b4acbd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:05.230571 12128 consensus_queue.cc:1048] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139311dab688423789a921ffdc3ac2d3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:05.254884   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:05.289584   431 tablet_server.cc:179] TabletServer@127.0.107.193:38817 shutting down...
I20260709 14:56:05.316969   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:05.317862   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.318984   431 raft_consensus.cc:2243] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.319736   431 raft_consensus.cc:2272] T cfe48a0dc399431c9c8aa8f7fbd37649 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.322917   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.323958   431 raft_consensus.cc:2243] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.324587   431 raft_consensus.cc:2272] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.327450   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.328157   431 raft_consensus.cc:2243] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.328835   431 raft_consensus.cc:2272] T aabf8384894d446794ad27fc47b3e006 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.331372   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.332060   431 raft_consensus.cc:2243] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.332631   431 raft_consensus.cc:2272] T d15599bec1e24e4cba2c627b79c6ef69 P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.335738   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.336476   431 raft_consensus.cc:2243] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.337001   431 raft_consensus.cc:2272] T 3efe74b6b60446349bd2e3d11a24f749 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.339373   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.340024   431 raft_consensus.cc:2243] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.340533   431 raft_consensus.cc:2272] T 9cc594ae471945018e876494c15c1bca P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.342764   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.343477   431 raft_consensus.cc:2243] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.343957   431 raft_consensus.cc:2272] T 208124883986430b9f4fe7a186a88f69 P 89721e000fcb47e8a347857c99e1262e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.346269   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.346930   431 raft_consensus.cc:2243] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.347451   431 raft_consensus.cc:2272] T eb65804b71684b1088e2fa50565ec42e P 89721e000fcb47e8a347857c99e1262e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.349691   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.350265   431 raft_consensus.cc:2243] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.350739   431 raft_consensus.cc:2272] T 2d82a650a0cf40d79b65b353285d3ca9 P 89721e000fcb47e8a347857c99e1262e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.353116   431 tablet_replica.cc:333] stopping tablet replica
I20260709 14:56:05.353708   431 raft_consensus.cc:2243] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.354206   431 raft_consensus.cc:2272] T 1fdad235fefd47d299dfc68992124b27 P 89721e000fcb47e8a347857c99e1262e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.389431   431 tablet_server.cc:196] TabletServer@127.0.107.193:38817 shutdown complete.
W20260709 14:56:05.399387 11679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260709 14:56:05.403779 11679 consensus_peers.cc:597] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:05.404368 11679 consensus_peers.cc:597] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:05.417932   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 14:56:05.429807 11753 consensus_peers.cc:597] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:05.454770 11754 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 -> Peer 139311dab688423789a921ffdc3ac2d3 (127.0.107.194:40695): Couldn't send request to peer 139311dab688423789a921ffdc3ac2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:05.457176   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:05.458235   431 tablet_replica.cc:333] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.459393   431 raft_consensus.cc:2243] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.460577   431 raft_consensus.cc:2272] T d15599bec1e24e4cba2c627b79c6ef69 P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.464630   431 tablet_replica.cc:333] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.465457   431 raft_consensus.cc:2243] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.466040   431 raft_consensus.cc:2272] T 1fdad235fefd47d299dfc68992124b27 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:05.467430 11753 consensus_peers.cc:597] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:05.469378   431 tablet_replica.cc:333] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.470180   431 raft_consensus.cc:2243] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.471351   431 raft_consensus.cc:2272] T 208124883986430b9f4fe7a186a88f69 P 139311dab688423789a921ffdc3ac2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.474213   431 tablet_replica.cc:333] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.475071   431 raft_consensus.cc:2243] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.476080   431 raft_consensus.cc:2272] T cfe48a0dc399431c9c8aa8f7fbd37649 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.478758   431 tablet_replica.cc:333] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.479606   431 raft_consensus.cc:2243] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.480185   431 raft_consensus.cc:2272] T 3efe74b6b60446349bd2e3d11a24f749 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.482887   431 tablet_replica.cc:333] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.483556   431 raft_consensus.cc:2243] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.484059   431 raft_consensus.cc:2272] T 9cc594ae471945018e876494c15c1bca P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.486690   431 tablet_replica.cc:333] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.487486   431 raft_consensus.cc:2243] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.488447   431 raft_consensus.cc:2272] T aabf8384894d446794ad27fc47b3e006 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.490974   431 tablet_replica.cc:333] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.491631   431 raft_consensus.cc:2243] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.492177   431 raft_consensus.cc:2272] T 76202e9e3d414285b5b5e74e7c3cfcf1 P 139311dab688423789a921ffdc3ac2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.495126   431 tablet_replica.cc:333] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.495838   431 raft_consensus.cc:2243] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.496407   431 raft_consensus.cc:2272] T eb65804b71684b1088e2fa50565ec42e P 139311dab688423789a921ffdc3ac2d3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.499191   431 tablet_replica.cc:333] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3: stopping tablet replica
I20260709 14:56:05.499902   431 raft_consensus.cc:2243] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.500847   431 raft_consensus.cc:2272] T 2d82a650a0cf40d79b65b353285d3ca9 P 139311dab688423789a921ffdc3ac2d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.538266   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 14:56:05.573716   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 14:56:05.612264 11753 consensus_peers.cc:597] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:05.616108 11753 consensus_peers.cc:597] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 -> Peer 89721e000fcb47e8a347857c99e1262e (127.0.107.193:38817): Couldn't send request to peer 89721e000fcb47e8a347857c99e1262e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:05.616750   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:05.617805   431 tablet_replica.cc:333] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.618712   431 raft_consensus.cc:2243] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.619457   431 raft_consensus.cc:2272] T 208124883986430b9f4fe7a186a88f69 P de497551bc0a41b695ba92f9b4acbd50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.622570   431 tablet_replica.cc:333] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.623571   431 raft_consensus.cc:2243] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.624742   431 raft_consensus.cc:2272] T 76202e9e3d414285b5b5e74e7c3cfcf1 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.628921   431 tablet_replica.cc:333] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.629643   431 raft_consensus.cc:2243] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.630194   431 raft_consensus.cc:2272] T 2d82a650a0cf40d79b65b353285d3ca9 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.632830   431 tablet_replica.cc:333] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.633513   431 raft_consensus.cc:2243] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 4 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.634479   431 raft_consensus.cc:2272] T eb65804b71684b1088e2fa50565ec42e P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.637178   431 tablet_replica.cc:333] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.637835   431 raft_consensus.cc:2243] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.638906   431 raft_consensus.cc:2272] T 1fdad235fefd47d299dfc68992124b27 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.642199   431 tablet_replica.cc:333] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.643000   431 raft_consensus.cc:2243] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.644066   431 raft_consensus.cc:2272] T 3efe74b6b60446349bd2e3d11a24f749 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.646762   431 tablet_replica.cc:333] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.647523   431 raft_consensus.cc:2243] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.648496   431 raft_consensus.cc:2272] T 9cc594ae471945018e876494c15c1bca P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.651065   431 tablet_replica.cc:333] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.651708   431 raft_consensus.cc:2243] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.652261   431 raft_consensus.cc:2272] T aabf8384894d446794ad27fc47b3e006 P de497551bc0a41b695ba92f9b4acbd50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.654819   431 tablet_replica.cc:333] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.655499   431 raft_consensus.cc:2243] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.656013   431 raft_consensus.cc:2272] T d15599bec1e24e4cba2c627b79c6ef69 P de497551bc0a41b695ba92f9b4acbd50 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.659348   431 tablet_replica.cc:333] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50: stopping tablet replica
I20260709 14:56:05.660331   431 raft_consensus.cc:2243] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:05.661139   431 raft_consensus.cc:2272] T cfe48a0dc399431c9c8aa8f7fbd37649 P de497551bc0a41b695ba92f9b4acbd50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.699556   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:56:05.731472   431 master.cc:562] Master@127.0.107.254:34669 shutting down...
I20260709 14:56:05.764580   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:05.765295   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:05.765691   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5c52f9dba20147df9be3d15cfc12bf3d: stopping tablet replica
I20260709 14:56:05.789285   431 master.cc:584] Master@127.0.107.254:34669 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13851 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 14:56:05.843166   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:44483
I20260709 14:56:05.844503   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:05.855042 12137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:05.855062 12136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:05.855109   431 server_base.cc:1061] running on GCE node
W20260709 14:56:05.855083 12139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:05.857101   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:05.857316   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:05.857437   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608965857426 us; error 0 us; skew 500 ppm
I20260709 14:56:05.857976   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:05.860816   431 webserver.cc:533] Webserver started at http://127.0.107.254:34559/ using document root <none> and password file <none>
I20260709 14:56:05.861377   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:05.861550   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:05.861830   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:05.863209   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "83ac9cd2ca804f418dc885954e7bc548"
format_stamp: "Formatted at 2026-07-09 14:56:05 on dist-test-slave-wg8n"
I20260709 14:56:05.868415   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:56:05.872367 12144 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:05.873266   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 14:56:05.873624   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "83ac9cd2ca804f418dc885954e7bc548"
format_stamp: "Formatted at 2026-07-09 14:56:05 on dist-test-slave-wg8n"
I20260709 14:56:05.873978   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:05.884966   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:05.886457   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:05.933166   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:44483
I20260709 14:56:05.933279 12195 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:44483 every 8 connection(s)
I20260709 14:56:05.937825 12196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:05.953845 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548: Bootstrap starting.
I20260709 14:56:05.959950 12196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:05.965657 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548: No bootstrap required, opened a new log
I20260709 14:56:05.968567 12196 raft_consensus.cc:359] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER }
I20260709 14:56:05.969074 12196 raft_consensus.cc:385] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:05.969358 12196 raft_consensus.cc:740] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ac9cd2ca804f418dc885954e7bc548, State: Initialized, Role: FOLLOWER
I20260709 14:56:05.970072 12196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER }
I20260709 14:56:05.970625 12196 raft_consensus.cc:399] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:56:05.970948 12196 raft_consensus.cc:493] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:56:05.971279 12196 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:05.977653 12196 raft_consensus.cc:515] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER }
I20260709 14:56:05.978384 12196 leader_election.cc:304] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83ac9cd2ca804f418dc885954e7bc548; no voters: 
I20260709 14:56:05.979849 12196 leader_election.cc:290] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:56:05.980293 12199 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:05.981985 12199 raft_consensus.cc:697] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 1 LEADER]: Becoming Leader. State: Replica: 83ac9cd2ca804f418dc885954e7bc548, State: Running, Role: LEADER
I20260709 14:56:05.982923 12199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER }
I20260709 14:56:05.983337 12196 sys_catalog.cc:565] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:56:05.988617 12200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83ac9cd2ca804f418dc885954e7bc548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER } }
I20260709 14:56:05.989199 12201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ac9cd2ca804f418dc885954e7bc548. Latest consensus state: current_term: 1 leader_uuid: "83ac9cd2ca804f418dc885954e7bc548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ac9cd2ca804f418dc885954e7bc548" member_type: VOTER } }
I20260709 14:56:05.989768 12200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:05.990233 12201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:05.994032 12206 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:56:06.002844 12206 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:56:06.004137   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:56:06.014217 12206 catalog_manager.cc:1383] Generated new cluster ID: e066eee10d2c4a3cad397c96da8b8020
I20260709 14:56:06.014626 12206 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:56:06.042701 12206 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:56:06.044363 12206 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:56:06.057852 12206 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548: Generated new TSK 0
I20260709 14:56:06.058660 12206 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:56:06.071693   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:06.081548 12217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.082353 12218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.085026 12220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:06.085817   431 server_base.cc:1061] running on GCE node
I20260709 14:56:06.087122   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:06.087419   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:06.087635   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608966087608 us; error 0 us; skew 500 ppm
I20260709 14:56:06.088307   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:06.091385   431 webserver.cc:533] Webserver started at http://127.0.107.193:42927/ using document root <none> and password file <none>
I20260709 14:56:06.091987   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:06.092204   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:06.092531   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:06.093900   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "c9efa1a027cf40fd845ffa003905353a"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.099695   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 14:56:06.104064 12225 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.105129   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:56:06.105571   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "c9efa1a027cf40fd845ffa003905353a"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.106010   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:06.123188   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:06.124626   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:06.126451   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:06.129271   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:06.129536   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.129824   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:06.130020   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.182297   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:39217
I20260709 14:56:06.182449 12287 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:39217 every 8 connection(s)
I20260709 14:56:06.188575   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:06.199147 12293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.198437 12292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.204344 12295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:06.204710 12288 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44483
I20260709 14:56:06.205365 12288 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:06.205642   431 server_base.cc:1061] running on GCE node
I20260709 14:56:06.206472 12288 heartbeater.cc:507] Master 127.0.107.254:44483 requested a full tablet report, sending...
I20260709 14:56:06.207163   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:06.207444   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:06.207700   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608966207673 us; error 0 us; skew 500 ppm
I20260709 14:56:06.208477   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:06.209646 12161 ts_manager.cc:194] Registered new tserver with Master: c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217)
I20260709 14:56:06.212037   431 webserver.cc:533] Webserver started at http://127.0.107.194:42519/ using document root <none> and password file <none>
I20260709 14:56:06.212772 12161 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53412
I20260709 14:56:06.212909   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:06.213340   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:06.213702   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:06.215443   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.225376   431 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.009s	sys 0.000s
I20260709 14:56:06.231324 12300 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.232453   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:56:06.232923   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.233276   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:06.258112   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:06.259704   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:06.261555   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:06.264325   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:06.264683   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.264961   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:06.265146   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.315371   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44547
I20260709 14:56:06.315496 12362 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44547 every 8 connection(s)
I20260709 14:56:06.321256   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:06.331660 12367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.332628 12366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:06.336736 12369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:06.337308   431 server_base.cc:1061] running on GCE node
I20260709 14:56:06.338181 12363 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44483
I20260709 14:56:06.338675   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 14:56:06.338682 12363 heartbeater.cc:461] Registering TS with master...
W20260709 14:56:06.339097   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:06.339339   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608966339316 us; error 0 us; skew 500 ppm
I20260709 14:56:06.339936 12363 heartbeater.cc:507] Master 127.0.107.254:44483 requested a full tablet report, sending...
I20260709 14:56:06.340027   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:06.342638 12161 ts_manager.cc:194] Registered new tserver with Master: 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:06.343441   431 webserver.cc:533] Webserver started at http://127.0.107.195:35119/ using document root <none> and password file <none>
I20260709 14:56:06.344157   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:06.344393   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:06.344501 12161 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53424
I20260709 14:56:06.344682   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:06.346387   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "41ecfa2642b5425fbd49db03cd04fd61"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.351730   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 14:56:06.355669 12374 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.356731   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 14:56:06.357120   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "41ecfa2642b5425fbd49db03cd04fd61"
format_stamp: "Formatted at 2026-07-09 14:56:06 on dist-test-slave-wg8n"
I20260709 14:56:06.357494   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:06.383378   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:06.384995   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:06.386631   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:06.389473   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:06.389707   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.390038   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:06.390228   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:06.443527   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:45617
I20260709 14:56:06.443667 12436 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:45617 every 8 connection(s)
I20260709 14:56:06.461083 12437 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44483
I20260709 14:56:06.461666 12437 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:06.462555 12437 heartbeater.cc:507] Master 127.0.107.254:44483 requested a full tablet report, sending...
I20260709 14:56:06.465077 12161 ts_manager.cc:194] Registered new tserver with Master: 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:06.465306   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018280597s
I20260709 14:56:06.467353 12161 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53426
I20260709 14:56:06.500036 12161 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53442:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:06.502969 12161 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:06.573576 12402 tablet_service.cc:1511] Processing CreateTablet for tablet 8d123549adfb42db8fafdb2dc0cba725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:06.573972 12401 tablet_service.cc:1511] Processing CreateTablet for tablet 9c3414ce19a14106b4ce0f57b6291b2c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:06.575443 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d123549adfb42db8fafdb2dc0cba725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.576296 12401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c3414ce19a14106b4ce0f57b6291b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.583647 12328 tablet_service.cc:1511] Processing CreateTablet for tablet 8d123549adfb42db8fafdb2dc0cba725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:06.583986 12327 tablet_service.cc:1511] Processing CreateTablet for tablet 9c3414ce19a14106b4ce0f57b6291b2c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:06.585605 12328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d123549adfb42db8fafdb2dc0cba725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.587031 12252 tablet_service.cc:1511] Processing CreateTablet for tablet 9c3414ce19a14106b4ce0f57b6291b2c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:06.588775 12252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c3414ce19a14106b4ce0f57b6291b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.592240 12253 tablet_service.cc:1511] Processing CreateTablet for tablet 8d123549adfb42db8fafdb2dc0cba725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2cd1f69909ff401fbc91432512d1868a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:06.593931 12253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d123549adfb42db8fafdb2dc0cba725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.595501 12327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c3414ce19a14106b4ce0f57b6291b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:06.610710 12452 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: Bootstrap starting.
I20260709 14:56:06.615365 12453 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: Bootstrap starting.
I20260709 14:56:06.623329 12452 tablet_bootstrap.cc:654] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.625638 12453 tablet_bootstrap.cc:654] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.638069 12455 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: Bootstrap starting.
I20260709 14:56:06.641799 12452 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: No bootstrap required, opened a new log
I20260709 14:56:06.639643 12453 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: No bootstrap required, opened a new log
I20260709 14:56:06.642479 12452 ts_tablet_manager.cc:1403] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.000s
I20260709 14:56:06.642711 12453 ts_tablet_manager.cc:1403] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.003s
I20260709 14:56:06.646090 12452 raft_consensus.cc:359] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.647027 12452 raft_consensus.cc:385] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.647360 12452 raft_consensus.cc:740] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.648569 12453 raft_consensus.cc:359] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.648757 12452 consensus_queue.cc:260] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.649629 12453 raft_consensus.cc:385] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.650015 12453 raft_consensus.cc:740] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.650961 12455 tablet_bootstrap.cc:654] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.651223 12453 consensus_queue.cc:260] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.652216 12437 heartbeater.cc:499] Master 127.0.107.254:44483 was elected leader, sending a full tablet report...
I20260709 14:56:06.652705 12452 ts_tablet_manager.cc:1434] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:56:06.654342 12452 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Bootstrap starting.
I20260709 14:56:06.666393 12452 tablet_bootstrap.cc:654] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.689929 12363 heartbeater.cc:499] Master 127.0.107.254:44483 was elected leader, sending a full tablet report...
I20260709 14:56:06.690893 12453 ts_tablet_manager.cc:1434] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent starting tablet: real 0.048s	user 0.033s	sys 0.012s
I20260709 14:56:06.692827 12453 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Bootstrap starting.
I20260709 14:56:06.705435 12453 tablet_bootstrap.cc:654] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.722703 12455 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: No bootstrap required, opened a new log
I20260709 14:56:06.722733 12452 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: No bootstrap required, opened a new log
I20260709 14:56:06.723212 12453 tablet_bootstrap.cc:492] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: No bootstrap required, opened a new log
I20260709 14:56:06.723507 12455 ts_tablet_manager.cc:1403] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: Time spent bootstrapping tablet: real 0.086s	user 0.029s	sys 0.050s
I20260709 14:56:06.723603 12452 ts_tablet_manager.cc:1403] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Time spent bootstrapping tablet: real 0.070s	user 0.039s	sys 0.028s
I20260709 14:56:06.723798 12453 ts_tablet_manager.cc:1403] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.011s
I20260709 14:56:06.726576 12455 raft_consensus.cc:359] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.727377 12455 raft_consensus.cc:385] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.727263 12452 raft_consensus.cc:359] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.727941 12455 raft_consensus.cc:740] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.727556 12453 raft_consensus.cc:359] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.728269 12452 raft_consensus.cc:385] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.728667 12452 raft_consensus.cc:740] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.728447 12453 raft_consensus.cc:385] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.729239 12453 raft_consensus.cc:740] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.728864 12455 consensus_queue.cc:260] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.729648 12452 consensus_queue.cc:260] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.730343 12453 consensus_queue.cc:260] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.732419 12452 ts_tablet_manager.cc:1434] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 14:56:06.734431 12288 heartbeater.cc:499] Master 127.0.107.254:44483 was elected leader, sending a full tablet report...
I20260709 14:56:06.735095 12455 ts_tablet_manager.cc:1434] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 14:56:06.735682 12453 ts_tablet_manager.cc:1434] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 14:56:06.737643 12455 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Bootstrap starting.
I20260709 14:56:06.746337 12455 tablet_bootstrap.cc:654] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:06.756824 12455 tablet_bootstrap.cc:492] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: No bootstrap required, opened a new log
I20260709 14:56:06.757416 12455 ts_tablet_manager.cc:1403] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 14:56:06.760264 12455 raft_consensus.cc:359] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.760926 12455 raft_consensus.cc:385] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:06.761197 12455 raft_consensus.cc:740] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Initialized, Role: FOLLOWER
I20260709 14:56:06.761909 12455 consensus_queue.cc:260] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.763767 12455 ts_tablet_manager.cc:1434] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 14:56:06.891400 12458 raft_consensus.cc:493] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:06.892179 12458 raft_consensus.cc:515] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.894610 12458 leader_election.cc:290] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617), c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217)
I20260709 14:56:06.907521 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61" is_pre_election: true
I20260709 14:56:06.908614 12412 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 0.
I20260709 14:56:06.908548 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a" is_pre_election: true
I20260709 14:56:06.909711 12263 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 0.
I20260709 14:56:06.910557 12301 leader_election.cc:304] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:06.911669 12458 raft_consensus.cc:2804] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:06.912026 12458 raft_consensus.cc:493] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:06.912305 12458 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:06.918896 12458 raft_consensus.cc:515] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.920826 12458 leader_election.cc:290] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 1 election: Requested vote from peers 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617), c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217)
I20260709 14:56:06.921939 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:06.922065 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
I20260709 14:56:06.922664 12412 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:06.922751 12263 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:06.930379 12263 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 1.
I20260709 14:56:06.930382 12412 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 1.
I20260709 14:56:06.932034 12301 leader_election.cc:304] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:06.932965 12458 raft_consensus.cc:2804] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:06.934414 12458 raft_consensus.cc:697] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 LEADER]: Becoming Leader. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Running, Role: LEADER
I20260709 14:56:06.935431 12458 consensus_queue.cc:237] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:06.946554 12159 catalog_manager.cc:5697] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:06.978545 12462 raft_consensus.cc:493] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:06.979419 12462 raft_consensus.cc:515] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:06.981837 12462 leader_election.cc:290] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547), 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:06.994472 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" is_pre_election: true
I20260709 14:56:06.995473 12338 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 0.
I20260709 14:56:06.995872 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61" is_pre_election: true
I20260709 14:56:06.996752 12412 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 0.
I20260709 14:56:06.997344 12226 leader_election.cc:304] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d825bba4c494d8cb7dc387e5fcfe3bb, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:06.998577 12462 raft_consensus.cc:2804] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:06.999032 12462 raft_consensus.cc:493] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 14:56:06.999114 12209 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:06.999411 12462 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.006753 12462 raft_consensus.cc:515] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.008893 12462 leader_election.cc:290] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 1 election: Requested vote from peers 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547), 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:07.009845 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.010074 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:07.010564 12338 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.010695 12412 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.018087 12412 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 1.
I20260709 14:56:07.018404 12338 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 1.
I20260709 14:56:07.019630 12226 leader_election.cc:304] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:07.020522 12462 raft_consensus.cc:2804] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:07.021692 12462 raft_consensus.cc:697] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 LEADER]: Becoming Leader. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:07.022666 12462 consensus_queue.cc:237] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.031656 12159 catalog_manager.cc:5697] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a reported cstate change: term changed from 0 to 1, leader changed from <none> to c9efa1a027cf40fd845ffa003905353a (127.0.107.193). New cstate: current_term: 1 leader_uuid: "c9efa1a027cf40fd845ffa003905353a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:07.097873 12159 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53444:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:07.100824 12159 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:07.130787 12252 tablet_service.cc:1511] Processing CreateTablet for tablet e1ffea299c184035ab9d3a3cd3f8ef9f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:07.132103 12327 tablet_service.cc:1511] Processing CreateTablet for tablet e1ffea299c184035ab9d3a3cd3f8ef9f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:07.132453 12252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ffea299c184035ab9d3a3cd3f8ef9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.133601 12327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ffea299c184035ab9d3a3cd3f8ef9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.136102 12402 tablet_service.cc:1511] Processing CreateTablet for tablet e1ffea299c184035ab9d3a3cd3f8ef9f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 14:56:07.137811 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ffea299c184035ab9d3a3cd3f8ef9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.138895 12401 tablet_service.cc:1511] Processing CreateTablet for tablet 11da2b9e0a7745768f3ea9beea0e3eff (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:07.140535 12401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11da2b9e0a7745768f3ea9beea0e3eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.136106 12253 tablet_service.cc:1511] Processing CreateTablet for tablet 11da2b9e0a7745768f3ea9beea0e3eff (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:07.144023 12328 tablet_service.cc:1511] Processing CreateTablet for tablet 11da2b9e0a7745768f3ea9beea0e3eff (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2b2887b0b55745508ce07f0b9afefd6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 14:56:07.144595 12253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11da2b9e0a7745768f3ea9beea0e3eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.145553 12328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11da2b9e0a7745768f3ea9beea0e3eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:07.162230 12452 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: Bootstrap starting.
I20260709 14:56:07.167091 12453 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Bootstrap starting.
I20260709 14:56:07.169261 12455 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Bootstrap starting.
I20260709 14:56:07.170544 12452 tablet_bootstrap.cc:654] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.175239 12453 tablet_bootstrap.cc:654] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.180095 12455 tablet_bootstrap.cc:654] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.192070 12453 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: No bootstrap required, opened a new log
I20260709 14:56:07.192962 12453 ts_tablet_manager.cc:1403] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.007s
I20260709 14:56:07.198010 12453 raft_consensus.cc:359] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.199236 12453 raft_consensus.cc:385] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.199676 12453 raft_consensus.cc:740] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.203136 12453 consensus_queue.cc:260] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.206238 12453 ts_tablet_manager.cc:1434] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 14:56:07.210703 12452 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: No bootstrap required, opened a new log
I20260709 14:56:07.211194 12453 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: Bootstrap starting.
I20260709 14:56:07.211505 12452 ts_tablet_manager.cc:1403] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: Time spent bootstrapping tablet: real 0.050s	user 0.006s	sys 0.021s
I20260709 14:56:07.214789 12455 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: No bootstrap required, opened a new log
I20260709 14:56:07.215668 12452 raft_consensus.cc:359] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.217586 12452 raft_consensus.cc:385] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.218204 12452 raft_consensus.cc:740] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.219323 12452 consensus_queue.cc:260] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.220036 12453 tablet_bootstrap.cc:654] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.227233 12452 ts_tablet_manager.cc:1434] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.005s
I20260709 14:56:07.228374 12453 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: No bootstrap required, opened a new log
I20260709 14:56:07.228618 12452 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: Bootstrap starting.
I20260709 14:56:07.229265 12453 ts_tablet_manager.cc:1403] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 14:56:07.232065 12455 ts_tablet_manager.cc:1403] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.011s
I20260709 14:56:07.233407 12453 raft_consensus.cc:359] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.234476 12453 raft_consensus.cc:385] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.234890 12453 raft_consensus.cc:740] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.235821 12455 raft_consensus.cc:359] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.236696 12455 raft_consensus.cc:385] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.236155 12453 consensus_queue.cc:260] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.236935 12452 tablet_bootstrap.cc:654] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.237040 12455 raft_consensus.cc:740] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.238318 12455 consensus_queue.cc:260] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.239401 12453 ts_tablet_manager.cc:1434] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 14:56:07.242664 12455 ts_tablet_manager.cc:1434] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 14:56:07.243811 12452 tablet_bootstrap.cc:492] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: No bootstrap required, opened a new log
I20260709 14:56:07.244256 12455 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: Bootstrap starting.
I20260709 14:56:07.244441 12452 ts_tablet_manager.cc:1403] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 14:56:07.252468 12455 tablet_bootstrap.cc:654] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:07.257328 12452 raft_consensus.cc:359] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.258252 12452 raft_consensus.cc:385] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.258562 12452 raft_consensus.cc:740] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.259477 12455 tablet_bootstrap.cc:492] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: No bootstrap required, opened a new log
I20260709 14:56:07.259435 12452 consensus_queue.cc:260] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.260159 12455 ts_tablet_manager.cc:1403] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 14:56:07.262162 12452 ts_tablet_manager.cc:1434] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 14:56:07.263348 12455 raft_consensus.cc:359] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.264108 12455 raft_consensus.cc:385] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:07.264429 12455 raft_consensus.cc:740] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Initialized, Role: FOLLOWER
I20260709 14:56:07.265496 12455 consensus_queue.cc:260] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.268190 12455 ts_tablet_manager.cc:1434] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 14:56:07.315969 12457 raft_consensus.cc:493] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:07.316879 12457 raft_consensus.cc:515] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.320931 12457 leader_election.cc:290] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:07.337124 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" is_pre_election: true
I20260709 14:56:07.338047 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a" is_pre_election: true
I20260709 14:56:07.338109 12338 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 0.
I20260709 14:56:07.339087 12263 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 0.
I20260709 14:56:07.340925 12375 leader_election.cc:304] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:07.343132 12457 raft_consensus.cc:2804] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:07.343585 12457 raft_consensus.cc:493] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:07.343623 12458 consensus_queue.cc:1048] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.344064 12457 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.355057 12457 raft_consensus.cc:515] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.362353 12457 leader_election.cc:290] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 election: Requested vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:07.366724 12458 consensus_queue.cc:1048] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.369050 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
I20260709 14:56:07.369884 12263 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.369987 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.370782 12338 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.382949 12338 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 1.
I20260709 14:56:07.384474 12375 leader_election.cc:304] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:07.385663 12457 raft_consensus.cc:2804] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:07.386289 12457 raft_consensus.cc:697] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Becoming Leader. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Running, Role: LEADER
I20260709 14:56:07.385881 12263 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 1.
I20260709 14:56:07.387344 12457 consensus_queue.cc:237] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.403436 12160 catalog_manager.cc:5697] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "41ecfa2642b5425fbd49db03cd04fd61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:07.414453 12462 consensus_queue.cc:1048] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.429101 12462 consensus_queue.cc:1048] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.437036 12457 raft_consensus.cc:493] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:07.437685 12457 raft_consensus.cc:515] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.440940 12457 leader_election.cc:290] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:07.441262 12262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a" is_pre_election: true
I20260709 14:56:07.442144 12262 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 0.
I20260709 14:56:07.443847 12378 leader_election.cc:304] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:07.443943 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" is_pre_election: true
I20260709 14:56:07.444764 12338 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 0.
I20260709 14:56:07.444905 12457 raft_consensus.cc:2804] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:07.445950 12457 raft_consensus.cc:493] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:07.446364 12457 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.455147 12457 raft_consensus.cc:515] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.456986 12457 leader_election.cc:290] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 election: Requested vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:07.458078 12262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
I20260709 14:56:07.458170 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.458958 12262 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.459017 12338 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:07.465840 12262 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 1.
I20260709 14:56:07.465888 12338 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 1.
I20260709 14:56:07.467465 12378 leader_election.cc:304] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:07.468300 12457 raft_consensus.cc:2804] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:07.468829 12457 raft_consensus.cc:697] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Becoming Leader. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Running, Role: LEADER
I20260709 14:56:07.469624 12457 consensus_queue.cc:237] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.478396 12161 catalog_manager.cc:5697] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "41ecfa2642b5425fbd49db03cd04fd61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:07.592710 12338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725"
dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
mode: GRACEFUL
new_leader_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:42784
I20260709 14:56:07.593813 12338 raft_consensus.cc:606] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 LEADER]: Received request to transfer leadership to TS c9efa1a027cf40fd845ffa003905353a
I20260709 14:56:07.611354 12262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c"
dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
mode: GRACEFUL
new_leader_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
 from {username='slave'} at 127.0.0.1:36628
I20260709 14:56:07.612134 12262 raft_consensus.cc:606] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 LEADER]: Received request to transfer leadership to TS 41ecfa2642b5425fbd49db03cd04fd61
I20260709 14:56:07.631429 12412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f"
dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
mode: GRACEFUL
new_leader_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:49894
I20260709 14:56:07.632226 12412 raft_consensus.cc:606] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Received request to transfer leadership to TS c9efa1a027cf40fd845ffa003905353a
I20260709 14:56:07.639875 12412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff"
dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
mode: GRACEFUL
new_leader_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:49894
I20260709 14:56:07.640589 12412 raft_consensus.cc:606] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Received request to transfer leadership to TS c9efa1a027cf40fd845ffa003905353a
I20260709 14:56:07.819111 12457 consensus_queue.cc:1048] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.844885 12457 consensus_queue.cc:1048] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.856150 12457 raft_consensus.cc:993] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: : Instructing follower c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.856922 12501 raft_consensus.cc:1081] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Signalling peer c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.859143 12262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff"
dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:36622
I20260709 14:56:07.859930 12262 raft_consensus.cc:493] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:07.860313 12262 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.870776 12262 raft_consensus.cc:515] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.873660 12262 leader_election.cc:290] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Requested vote from peers 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547), 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:07.876019 12338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.875910 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:07.877208 12338 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.877346 12412 raft_consensus.cc:3055] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:07.878134 12412 raft_consensus.cc:740] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Running, Role: LEADER
I20260709 14:56:07.879854 12412 consensus_queue.cc:260] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.882254 12412 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.890889 12338 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:07.892781 12226 leader_election.cc:304] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d825bba4c494d8cb7dc387e5fcfe3bb, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:07.894179 12462 raft_consensus.cc:2804] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:07.894896 12462 raft_consensus.cc:697] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Becoming Leader. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:07.896304 12501 consensus_queue.cc:1048] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.896209 12462 consensus_queue.cc:237] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.898154 12412 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:07.908092 12457 raft_consensus.cc:993] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: : Instructing follower c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.908685 12457 raft_consensus.cc:1081] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Signalling peer c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.909747 12160 catalog_manager.cc:5697] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a reported cstate change: term changed from 1 to 2, leader changed from 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195) to c9efa1a027cf40fd845ffa003905353a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c9efa1a027cf40fd845ffa003905353a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:07.912076 12263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f"
dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:36622
I20260709 14:56:07.912925 12263 raft_consensus.cc:493] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:07.913429 12263 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.915467 12501 consensus_queue.cc:1048] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.929157 12263 raft_consensus.cc:515] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.930787 12262 raft_consensus.cc:1240] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Rejecting Update request from peer 41ecfa2642b5425fbd49db03cd04fd61 for earlier term 1. Current term is 2. Ops: []
I20260709 14:56:07.936622 12501 consensus_queue.cc:1059] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:07.937700 12457 raft_consensus.cc:3055] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:07.938145 12457 raft_consensus.cc:740] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Running, Role: LEADER
I20260709 14:56:07.939342 12457 consensus_queue.cc:260] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.940766 12263 leader_election.cc:290] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Requested vote from peers 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547), 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:07.941380 12337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.942175 12337 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.943428 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ffea299c184035ab9d3a3cd3f8ef9f" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:07.945578 12457 raft_consensus.cc:3060] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.959579 12337 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:07.961984 12226 leader_election.cc:304] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d825bba4c494d8cb7dc387e5fcfe3bb, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:07.963258 12462 raft_consensus.cc:2804] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:07.964191 12462 raft_consensus.cc:697] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Becoming Leader. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:07.967630 12505 raft_consensus.cc:993] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: : Instructing follower c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.965936 12462 consensus_queue.cc:237] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:07.968465 12506 raft_consensus.cc:1081] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 LEADER]: Signalling peer c9efa1a027cf40fd845ffa003905353a to start an election
I20260709 14:56:07.969098 12412 raft_consensus.cc:2468] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:07.971890 12263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725"
dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
 from {username='slave'} at 127.0.0.1:36616
I20260709 14:56:07.972645 12263 raft_consensus.cc:493] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:07.973119 12263 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.978516 12160 catalog_manager.cc:5697] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a reported cstate change: term changed from 1 to 2, leader changed from 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195) to c9efa1a027cf40fd845ffa003905353a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c9efa1a027cf40fd845ffa003905353a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:07.988443 12263 raft_consensus.cc:515] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:07.991488 12263 leader_election.cc:290] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Requested vote from peers 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:07.994119 12337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:07.993866 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d123549adfb42db8fafdb2dc0cba725" candidate_uuid: "c9efa1a027cf40fd845ffa003905353a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:07.994937 12337 raft_consensus.cc:3055] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:07.995379 12337 raft_consensus.cc:740] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Running, Role: LEADER
I20260709 14:56:07.995468 12412 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:07.996719 12337 consensus_queue.cc:260] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:07.998404 12337 raft_consensus.cc:3060] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:08.006662 12412 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:08.008414 12226 leader_election.cc:304] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, c9efa1a027cf40fd845ffa003905353a; no voters: 
I20260709 14:56:08.009099 12337 raft_consensus.cc:2468] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efa1a027cf40fd845ffa003905353a in term 2.
I20260709 14:56:08.009557 12462 raft_consensus.cc:2804] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:08.010252 12462 raft_consensus.cc:697] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Becoming Leader. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:08.011592 12462 consensus_queue.cc:237] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } }
I20260709 14:56:08.019981 12160 catalog_manager.cc:5697] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a reported cstate change: term changed from 1 to 2, leader changed from 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194) to c9efa1a027cf40fd845ffa003905353a (127.0.107.193). New cstate: current_term: 2 leader_uuid: "c9efa1a027cf40fd845ffa003905353a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:08.031351 12470 raft_consensus.cc:993] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: : Instructing follower 41ecfa2642b5425fbd49db03cd04fd61 to start an election
I20260709 14:56:08.032212 12508 raft_consensus.cc:1081] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 LEADER]: Signalling peer 41ecfa2642b5425fbd49db03cd04fd61 to start an election
I20260709 14:56:08.034746 12412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c"
dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
 from {username='slave'} at 127.0.0.1:49884
I20260709 14:56:08.035588 12412 raft_consensus.cc:493] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:08.036033 12412 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:08.043354 12412 raft_consensus.cc:515] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.045975 12412 leader_election.cc:290] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 2 election: Requested vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547)
I20260709 14:56:08.047230 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
I20260709 14:56:08.047230 12337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c3414ce19a14106b4ce0f57b6291b2c" candidate_uuid: "41ecfa2642b5425fbd49db03cd04fd61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
I20260709 14:56:08.048127 12263 raft_consensus.cc:3055] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:08.048218 12337 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:08.048498 12263 raft_consensus.cc:740] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:08.049552 12263 consensus_queue.cc:260] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.050777 12263 raft_consensus.cc:3060] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:08.055454 12337 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 2.
I20260709 14:56:08.057253 12375 leader_election.cc:304] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:08.057722 12263 raft_consensus.cc:2468] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ecfa2642b5425fbd49db03cd04fd61 in term 2.
I20260709 14:56:08.058230 12457 raft_consensus.cc:2804] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:08.059001 12457 raft_consensus.cc:697] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 LEADER]: Becoming Leader. State: Replica: 41ecfa2642b5425fbd49db03cd04fd61, State: Running, Role: LEADER
I20260709 14:56:08.060238 12457 consensus_queue.cc:237] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.068477 12160 catalog_manager.cc:5697] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 reported cstate change: term changed from 1 to 2, leader changed from c9efa1a027cf40fd845ffa003905353a (127.0.107.193) to 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "41ecfa2642b5425fbd49db03cd04fd61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:08.171851   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 14:56:08.173728   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 14:56:08.174227   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 14:56:08.188724 12263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff"
dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
mode: GRACEFUL
new_leader_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
 from {username='slave'} at 127.0.0.1:36630
I20260709 14:56:08.189419 12263 raft_consensus.cc:606] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Received request to transfer leadership to TS 9d825bba4c494d8cb7dc387e5fcfe3bb
I20260709 14:56:08.190922   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 14:56:08.283699 12412 raft_consensus.cc:1275] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Refusing update from remote peer c9efa1a027cf40fd845ffa003905353a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:08.285636 12508 consensus_queue.cc:1048] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:08.298463 12337 raft_consensus.cc:1275] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Refusing update from remote peer c9efa1a027cf40fd845ffa003905353a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:08.302392 12508 consensus_queue.cc:1048] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:08.312569 12511 raft_consensus.cc:993] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: : Instructing follower 9d825bba4c494d8cb7dc387e5fcfe3bb to start an election
I20260709 14:56:08.313252 12508 raft_consensus.cc:1081] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Signalling peer 9d825bba4c494d8cb7dc387e5fcfe3bb to start an election
I20260709 14:56:08.315294 12337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff"
dest_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb"
 from {username='slave'} at 127.0.0.1:42770
I20260709 14:56:08.316035 12337 raft_consensus.cc:493] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:08.316471 12337 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:08.323868 12337 raft_consensus.cc:515] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.326431 12337 leader_election.cc:290] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 3 election: Requested vote from peers c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217), 41ecfa2642b5425fbd49db03cd04fd61 (127.0.107.195:45617)
I20260709 14:56:08.327677 12263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c9efa1a027cf40fd845ffa003905353a"
I20260709 14:56:08.328107 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11da2b9e0a7745768f3ea9beea0e3eff" candidate_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41ecfa2642b5425fbd49db03cd04fd61"
I20260709 14:56:08.328575 12263 raft_consensus.cc:3055] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:08.328775 12412 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:08.329020 12263 raft_consensus.cc:740] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c9efa1a027cf40fd845ffa003905353a, State: Running, Role: LEADER
I20260709 14:56:08.330020 12263 consensus_queue.cc:260] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.331566 12263 raft_consensus.cc:3060] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:08.338124 12412 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 3.
I20260709 14:56:08.339890 12301 leader_election.cc:304] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ecfa2642b5425fbd49db03cd04fd61, 9d825bba4c494d8cb7dc387e5fcfe3bb; no voters: 
I20260709 14:56:08.340811 12263 raft_consensus.cc:2468] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d825bba4c494d8cb7dc387e5fcfe3bb in term 3.
I20260709 14:56:08.340989 12505 raft_consensus.cc:2804] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:08.341733 12505 raft_consensus.cc:697] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 LEADER]: Becoming Leader. State: Replica: 9d825bba4c494d8cb7dc387e5fcfe3bb, State: Running, Role: LEADER
I20260709 14:56:08.342886 12505 consensus_queue.cc:237] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } }
I20260709 14:56:08.350760 12161 catalog_manager.cc:5697] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb reported cstate change: term changed from 2 to 3, leader changed from c9efa1a027cf40fd845ffa003905353a (127.0.107.193) to 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194). New cstate: current_term: 3 leader_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9efa1a027cf40fd845ffa003905353a" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:08.437129   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 14:56:08.476994   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:08.477914   431 tablet_replica.cc:333] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: stopping tablet replica
I20260709 14:56:08.478681   431 raft_consensus.cc:2243] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:08.479516   431 pending_rounds.cc:70] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Trying to abort 2 pending ops.
I20260709 14:56:08.479744   431 pending_rounds.cc:77] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305662332808732672 op_type: NO_OP noop_request { }
I20260709 14:56:08.480083   431 raft_consensus.cc:2889] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.480373   431 pending_rounds.cc:77] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305662330763677696 op_type: NO_OP noop_request { }
I20260709 14:56:08.480633   431 raft_consensus.cc:2889] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.480942   431 raft_consensus.cc:2272] T e1ffea299c184035ab9d3a3cd3f8ef9f P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.485086   431 tablet_replica.cc:333] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a: stopping tablet replica
I20260709 14:56:08.485651 12412 raft_consensus.cc:1275] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Refusing update from remote peer c9efa1a027cf40fd845ffa003905353a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:08.486147   431 raft_consensus.cc:2243] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.486878   431 raft_consensus.cc:2272] T 11da2b9e0a7745768f3ea9beea0e3eff P c9efa1a027cf40fd845ffa003905353a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.487691 12508 consensus_queue.cc:1048] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ecfa2642b5425fbd49db03cd04fd61" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:08.491202   431 tablet_replica.cc:333] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: stopping tablet replica
I20260709 14:56:08.492462   431 raft_consensus.cc:2243] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:08.493467   431 pending_rounds.cc:70] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Trying to abort 1 pending ops.
I20260709 14:56:08.493744   431 pending_rounds.cc:77] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305662332985847808 op_type: NO_OP noop_request { }
I20260709 14:56:08.494028   431 raft_consensus.cc:2889] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.494314   431 raft_consensus.cc:2272] T 8d123549adfb42db8fafdb2dc0cba725 P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.499329   431 tablet_replica.cc:333] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a: stopping tablet replica
I20260709 14:56:08.500057   431 raft_consensus.cc:2243] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.500561   431 raft_consensus.cc:2272] T 9c3414ce19a14106b4ce0f57b6291b2c P c9efa1a027cf40fd845ffa003905353a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.531054   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 14:56:08.546494 12337 raft_consensus.cc:1275] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Refusing update from remote peer 41ecfa2642b5425fbd49db03cd04fd61: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:08.548822 12457 consensus_queue.cc:1048] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d825bba4c494d8cb7dc387e5fcfe3bb" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:08.559871   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 14:56:08.579594 12378 consensus_peers.cc:597] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 -> Peer c9efa1a027cf40fd845ffa003905353a (127.0.107.193:39217): Couldn't send request to peer c9efa1a027cf40fd845ffa003905353a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39217: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:08.581303 12375 consensus_peers.cc:597] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 -> Peer 9d825bba4c494d8cb7dc387e5fcfe3bb (127.0.107.194:44547): Couldn't send request to peer 9d825bba4c494d8cb7dc387e5fcfe3bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:08.604439   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:08.605448   431 tablet_replica.cc:333] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb: stopping tablet replica
I20260709 14:56:08.606262   431 raft_consensus.cc:2243] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.606894   431 raft_consensus.cc:2272] T e1ffea299c184035ab9d3a3cd3f8ef9f P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.610010   431 tablet_replica.cc:333] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: stopping tablet replica
I20260709 14:56:08.610674   431 raft_consensus.cc:2243] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:08.611478   431 pending_rounds.cc:70] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Trying to abort 1 pending ops.
I20260709 14:56:08.611689   431 pending_rounds.cc:77] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305662334341468160 op_type: NO_OP noop_request { }
I20260709 14:56:08.611999   431 raft_consensus.cc:2889] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.612298   431 raft_consensus.cc:2272] T 11da2b9e0a7745768f3ea9beea0e3eff P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.615557   431 tablet_replica.cc:333] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: stopping tablet replica
I20260709 14:56:08.616190   431 raft_consensus.cc:2243] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.616678   431 pending_rounds.cc:70] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Trying to abort 1 pending ops.
I20260709 14:56:08.616859   431 pending_rounds.cc:77] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305662333183909888 op_type: NO_OP noop_request { }
I20260709 14:56:08.617133   431 raft_consensus.cc:2889] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.617452   431 raft_consensus.cc:2272] T 9c3414ce19a14106b4ce0f57b6291b2c P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.620795   431 tablet_replica.cc:333] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb: stopping tablet replica
I20260709 14:56:08.621387   431 raft_consensus.cc:2243] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.621819   431 raft_consensus.cc:2272] T 8d123549adfb42db8fafdb2dc0cba725 P 9d825bba4c494d8cb7dc387e5fcfe3bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.637717   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 14:56:08.664469   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 14:56:08.694175   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:08.695456   431 tablet_replica.cc:333] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61: stopping tablet replica
I20260709 14:56:08.696347   431 raft_consensus.cc:2243] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.697026   431 raft_consensus.cc:2272] T e1ffea299c184035ab9d3a3cd3f8ef9f P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.700768   431 tablet_replica.cc:333] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61: stopping tablet replica
I20260709 14:56:08.701495   431 raft_consensus.cc:2243] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.702081   431 raft_consensus.cc:2272] T 11da2b9e0a7745768f3ea9beea0e3eff P 41ecfa2642b5425fbd49db03cd04fd61 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.705422   431 tablet_replica.cc:333] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: stopping tablet replica
I20260709 14:56:08.706156   431 raft_consensus.cc:2243] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:08.706642   431 pending_rounds.cc:70] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Trying to abort 1 pending ops.
I20260709 14:56:08.706879   431 pending_rounds.cc:77] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305662332985847808 op_type: NO_OP noop_request { }
I20260709 14:56:08.707197   431 raft_consensus.cc:2889] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 14:56:08.707500   431 raft_consensus.cc:2272] T 8d123549adfb42db8fafdb2dc0cba725 P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.711140   431 tablet_replica.cc:333] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61: stopping tablet replica
I20260709 14:56:08.711834   431 raft_consensus.cc:2243] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:08.712906   431 raft_consensus.cc:2272] T 9c3414ce19a14106b4ce0f57b6291b2c P 41ecfa2642b5425fbd49db03cd04fd61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.742015   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:56:08.762208   431 master.cc:562] Master@127.0.107.254:44483 shutting down...
I20260709 14:56:08.788013   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:08.788735   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:08.789129   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 83ac9cd2ca804f418dc885954e7bc548: stopping tablet replica
I20260709 14:56:08.815364   431 master.cc:584] Master@127.0.107.254:44483 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3020 ms)
[----------] 6 tests from LeaderRebalancerTest (369911 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 14:56:08.863747   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:46621
I20260709 14:56:08.865075   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:08.872567 12517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:08.872991 12516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:08.873006 12519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:08.874797   431 server_base.cc:1061] running on GCE node
I20260709 14:56:08.875875   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:08.876104   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:08.876227   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608968876216 us; error 0 us; skew 500 ppm
I20260709 14:56:08.876791   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:08.880111   431 webserver.cc:533] Webserver started at http://127.0.107.254:45029/ using document root <none> and password file <none>
I20260709 14:56:08.880725   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:08.880910   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:08.881155   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:08.882725   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "7547cd1ca894443f892fa6f941dda1d9"
format_stamp: "Formatted at 2026-07-09 14:56:08 on dist-test-slave-wg8n"
I20260709 14:56:08.888574   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:56:08.893668 12524 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:08.894780   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:56:08.895435   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "7547cd1ca894443f892fa6f941dda1d9"
format_stamp: "Formatted at 2026-07-09 14:56:08 on dist-test-slave-wg8n"
I20260709 14:56:08.895785   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:08.928761   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:08.930419   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:08.979357   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:46621
I20260709 14:56:08.979491 12575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:46621 every 8 connection(s)
I20260709 14:56:08.983866 12576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:08.997926 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: Bootstrap starting.
I20260709 14:56:09.004081 12576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:09.009619 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: No bootstrap required, opened a new log
I20260709 14:56:09.012544 12576 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER }
I20260709 14:56:09.013094 12576 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:09.013355 12576 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7547cd1ca894443f892fa6f941dda1d9, State: Initialized, Role: FOLLOWER
I20260709 14:56:09.014101 12576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER }
I20260709 14:56:09.014614 12576 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:56:09.014901 12576 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:56:09.015189 12576 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:09.021553 12576 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER }
I20260709 14:56:09.022258 12576 leader_election.cc:304] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7547cd1ca894443f892fa6f941dda1d9; no voters: 
I20260709 14:56:09.023684 12576 leader_election.cc:290] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:56:09.024151 12579 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:09.025799 12579 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7547cd1ca894443f892fa6f941dda1d9, State: Running, Role: LEADER
I20260709 14:56:09.026722 12579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER }
I20260709 14:56:09.027387 12576 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:56:09.030346 12581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7547cd1ca894443f892fa6f941dda1d9. Latest consensus state: current_term: 1 leader_uuid: "7547cd1ca894443f892fa6f941dda1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER } }
I20260709 14:56:09.030197 12580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7547cd1ca894443f892fa6f941dda1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7547cd1ca894443f892fa6f941dda1d9" member_type: VOTER } }
I20260709 14:56:09.031219 12581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:09.031345 12580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:09.037426 12584 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:56:09.043387 12584 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:56:09.046159   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:56:09.054016 12584 catalog_manager.cc:1383] Generated new cluster ID: 807c8c68cc4040ccb8e1174fa60bca95
I20260709 14:56:09.054419 12584 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:56:09.083045 12584 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:56:09.085014 12584 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:56:09.113283 12584 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: Generated new TSK 0
I20260709 14:56:09.114041 12584 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:56:09.178188   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:09.187578 12598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:09.187896 12597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:09.189404 12600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:09.190003   431 server_base.cc:1061] running on GCE node
I20260709 14:56:09.191236   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:09.191462   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:09.191684   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608969191661 us; error 0 us; skew 500 ppm
I20260709 14:56:09.192301   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:09.195264   431 webserver.cc:533] Webserver started at http://127.0.107.193:38921/ using document root <none> and password file <none>
I20260709 14:56:09.195845   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:09.196067   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:09.196384   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:09.197716   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "2e8307f11bd74ea49e419aac823055e4"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.203261   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.006s
I20260709 14:56:09.207592 12605 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.208694   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 14:56:09.209201   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "2e8307f11bd74ea49e419aac823055e4"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.209561   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:09.237587   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:09.239666   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:09.241621   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:09.244890   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:09.245227   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.245519   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:09.245751   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.303215   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:34601
I20260709 14:56:09.303323 12667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:34601 every 8 connection(s)
I20260709 14:56:09.309471   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:09.319839 12672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:09.322233 12673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:09.324334 12675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:09.327090   431 server_base.cc:1061] running on GCE node
I20260709 14:56:09.327405 12668 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46621
I20260709 14:56:09.327965 12668 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:09.329034 12668 heartbeater.cc:507] Master 127.0.107.254:46621 requested a full tablet report, sending...
I20260709 14:56:09.329622   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:09.329968   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:09.330191   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608969330163 us; error 0 us; skew 500 ppm
I20260709 14:56:09.331086   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:09.332618 12541 ts_manager.cc:194] Registered new tserver with Master: 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601)
I20260709 14:56:09.335132   431 webserver.cc:533] Webserver started at http://127.0.107.194:34141/ using document root <none> and password file <none>
I20260709 14:56:09.335662 12541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48394
I20260709 14:56:09.335860   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:09.336423   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:09.336810   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:09.338423   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "15534220b49c48e4a153b4563479d054"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.344767   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 14:56:09.349517 12680 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.350713   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:56:09.351217   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "15534220b49c48e4a153b4563479d054"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.351576   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:09.365252   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:09.366881   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:09.368650   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:09.372004   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:09.372315   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.372648   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:09.372865   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.428439   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:33531
I20260709 14:56:09.428570 12742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:33531 every 8 connection(s)
I20260709 14:56:09.434813   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:09.446286 12746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:09.447574 12747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:09.450663 12743 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46621
I20260709 14:56:09.451345 12743 heartbeater.cc:461] Registering TS with master...
W20260709 14:56:09.452101 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:09.452476   431 server_base.cc:1061] running on GCE node
I20260709 14:56:09.452744 12743 heartbeater.cc:507] Master 127.0.107.254:46621 requested a full tablet report, sending...
I20260709 14:56:09.453989   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:09.454277   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:09.454499   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608969454479 us; error 0 us; skew 500 ppm
I20260709 14:56:09.455399   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:09.455875 12541 ts_manager.cc:194] Registered new tserver with Master: 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:09.458045 12541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33400
I20260709 14:56:09.459415   431 webserver.cc:533] Webserver started at http://127.0.107.195:46335/ using document root <none> and password file <none>
I20260709 14:56:09.460579   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:09.460892   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:09.461257   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:09.462530   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.468046   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 14:56:09.472568 12754 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.473856   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 14:56:09.474228   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
format_stamp: "Formatted at 2026-07-09 14:56:09 on dist-test-slave-wg8n"
I20260709 14:56:09.474537   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:09.497339   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:09.498795   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:09.500530   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:09.503265   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:09.503484   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.503816   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:09.503988   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:09.555830   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:45519
I20260709 14:56:09.555936 12816 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:45519 every 8 connection(s)
I20260709 14:56:09.572621 12817 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46621
I20260709 14:56:09.573237 12817 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:09.574167 12817 heartbeater.cc:507] Master 127.0.107.254:46621 requested a full tablet report, sending...
I20260709 14:56:09.576901 12541 ts_manager.cc:194] Registered new tserver with Master: 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:09.577554   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0177099s
I20260709 14:56:09.578893 12541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33402
I20260709 14:56:09.612183 12541 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33408:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:09.616370 12541 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:09.701251 12708 tablet_service.cc:1511] Processing CreateTablet for tablet ea32fc7c4e7f41cca6beaa1694925a61 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:09.703145 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea32fc7c4e7f41cca6beaa1694925a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.717927 12704 tablet_service.cc:1511] Processing CreateTablet for tablet e95f790959a342c5bfea1c293cf1a739 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:09.718469 12707 tablet_service.cc:1511] Processing CreateTablet for tablet c78ce6c1a4c44290bdd90177bde180e5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:09.719961 12704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95f790959a342c5bfea1c293cf1a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.721015 12705 tablet_service.cc:1511] Processing CreateTablet for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:09.722596 12705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.724073 12703 tablet_service.cc:1511] Processing CreateTablet for tablet fd619b7389e14e05859cb4dc49118e0b (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:09.725642 12703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd619b7389e14e05859cb4dc49118e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.726502 12706 tablet_service.cc:1511] Processing CreateTablet for tablet a86407143c684a358b111eaada3f75d5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:09.728096 12706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86407143c684a358b111eaada3f75d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.728302 12702 tablet_service.cc:1511] Processing CreateTablet for tablet de7fdd7742cb4669b10b276d7e672f12 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:09.728598 12701 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91efd7d2344d2e930909ffe9b79303 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:09.729789 12707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78ce6c1a4c44290bdd90177bde180e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.732649 12633 tablet_service.cc:1511] Processing CreateTablet for tablet ea32fc7c4e7f41cca6beaa1694925a61 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:09.734385 12633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea32fc7c4e7f41cca6beaa1694925a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.736814 12631 tablet_service.cc:1511] Processing CreateTablet for tablet a86407143c684a358b111eaada3f75d5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:09.738170 12700 tablet_service.cc:1511] Processing CreateTablet for tablet 242b524686de4239a46c51ee07876660 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:09.740499 12632 tablet_service.cc:1511] Processing CreateTablet for tablet c78ce6c1a4c44290bdd90177bde180e5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:09.742178 12702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7fdd7742cb4669b10b276d7e672f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.743121 12700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242b524686de4239a46c51ee07876660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.747387 12701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91efd7d2344d2e930909ffe9b79303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.738605 12631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86407143c684a358b111eaada3f75d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.754604 12632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78ce6c1a4c44290bdd90177bde180e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.758284 12835 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:09.760249 12630 tablet_service.cc:1511] Processing CreateTablet for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:09.762120 12630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.764315 12782 tablet_service.cc:1511] Processing CreateTablet for tablet ea32fc7c4e7f41cca6beaa1694925a61 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:09.764771 12629 tablet_service.cc:1511] Processing CreateTablet for tablet e95f790959a342c5bfea1c293cf1a739 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:09.766119 12782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea32fc7c4e7f41cca6beaa1694925a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.768177 12781 tablet_service.cc:1511] Processing CreateTablet for tablet c78ce6c1a4c44290bdd90177bde180e5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:09.769912 12781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78ce6c1a4c44290bdd90177bde180e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.769560 12627 tablet_service.cc:1511] Processing CreateTablet for tablet de7fdd7742cb4669b10b276d7e672f12 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:09.772310 12780 tablet_service.cc:1511] Processing CreateTablet for tablet a86407143c684a358b111eaada3f75d5 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:09.774447 12779 tablet_service.cc:1511] Processing CreateTablet for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:09.797963 12626 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91efd7d2344d2e930909ffe9b79303 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:09.777458 12778 tablet_service.cc:1511] Processing CreateTablet for tablet e95f790959a342c5bfea1c293cf1a739 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:09.798682 12776 tablet_service.cc:1511] Processing CreateTablet for tablet de7fdd7742cb4669b10b276d7e672f12 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:09.780601 12777 tablet_service.cc:1511] Processing CreateTablet for tablet fd619b7389e14e05859cb4dc49118e0b (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:09.768208 12628 tablet_service.cc:1511] Processing CreateTablet for tablet fd619b7389e14e05859cb4dc49118e0b (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:09.802541 12625 tablet_service.cc:1511] Processing CreateTablet for tablet 242b524686de4239a46c51ee07876660 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:09.795606 12629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95f790959a342c5bfea1c293cf1a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.805094 12627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7fdd7742cb4669b10b276d7e672f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.805972 12774 tablet_service.cc:1511] Processing CreateTablet for tablet 242b524686de4239a46c51ee07876660 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:09.807361 12625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242b524686de4239a46c51ee07876660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.802398 12775 tablet_service.cc:1511] Processing CreateTablet for tablet 6d91efd7d2344d2e930909ffe9b79303 (DEFAULT_TABLE table=test-workload [id=312a8937168c42f9ba938a5d65d43c59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:09.808780 12626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91efd7d2344d2e930909ffe9b79303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.795284 12835 tablet_bootstrap.cc:654] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:09.811254 12628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd619b7389e14e05859cb4dc49118e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.822942 12780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86407143c684a358b111eaada3f75d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.823865 12776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7fdd7742cb4669b10b276d7e672f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.828334 12774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242b524686de4239a46c51ee07876660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.829123 12775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d91efd7d2344d2e930909ffe9b79303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.830060 12777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd619b7389e14e05859cb4dc49118e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.834626 12778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95f790959a342c5bfea1c293cf1a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.835981 12779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb41a14b8ed34f7d9fffa0a7cb555dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:09.941370 12835 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:09.942142 12835 ts_tablet_manager.cc:1403] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.184s	user 0.029s	sys 0.025s
I20260709 14:56:09.946053 12835 raft_consensus.cc:359] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:09.960634 12835 raft_consensus.cc:385] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:09.961074 12835 raft_consensus.cc:740] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:09.962221 12835 consensus_queue.cc:260] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:09.975241 12837 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:09.978066 12838 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:09.980175 12835 ts_tablet_manager.cc:1434] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.038s	user 0.015s	sys 0.000s
I20260709 14:56:09.980367 12743 heartbeater.cc:499] Master 127.0.107.254:46621 was elected leader, sending a full tablet report...
I20260709 14:56:09.981590 12835 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:09.987511 12838 tablet_bootstrap.cc:654] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:09.989759 12835 tablet_bootstrap.cc:654] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:09.997962 12837 tablet_bootstrap.cc:654] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.003890 12835 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.004675 12835 ts_tablet_manager.cc:1403] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 14:56:10.008584 12835 raft_consensus.cc:359] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.009470 12835 raft_consensus.cc:385] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.009796 12835 raft_consensus.cc:740] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.012653 12835 consensus_queue.cc:260] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.014163 12838 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.015029 12838 ts_tablet_manager.cc:1403] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.038s	user 0.027s	sys 0.008s
I20260709 14:56:10.015713 12837 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.015594 12835 ts_tablet_manager.cc:1434] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 14:56:10.016594 12837 ts_tablet_manager.cc:1403] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.008s
I20260709 14:56:10.017468 12835 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.019244 12838 raft_consensus.cc:359] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.020231 12838 raft_consensus.cc:385] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.020581 12837 raft_consensus.cc:359] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.021824 12837 raft_consensus.cc:385] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.022202 12837 raft_consensus.cc:740] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.023486 12837 consensus_queue.cc:260] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.020584 12838 raft_consensus.cc:740] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.026796 12835 tablet_bootstrap.cc:654] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.026806 12838 consensus_queue.cc:260] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.033131 12668 heartbeater.cc:499] Master 127.0.107.254:46621 was elected leader, sending a full tablet report...
I20260709 14:56:10.034022 12837 ts_tablet_manager.cc:1434] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 14:56:10.035326 12837 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.037366 12838 ts_tablet_manager.cc:1434] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.005s
I20260709 14:56:10.043442 12839 raft_consensus.cc:493] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.044013 12835 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.044521 12817 heartbeater.cc:499] Master 127.0.107.254:46621 was elected leader, sending a full tablet report...
I20260709 14:56:10.044776 12837 tablet_bootstrap.cc:654] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.044945 12835 ts_tablet_manager.cc:1403] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.008s
I20260709 14:56:10.044466 12839 raft_consensus.cc:515] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
W20260709 14:56:10.039887 12589 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:10.049539 12835 raft_consensus.cc:359] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.051149 12835 raft_consensus.cc:385] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.051679 12835 raft_consensus.cc:740] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.052862 12835 consensus_queue.cc:260] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.059926 12839 leader_election.cc:290] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.060925 12838 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.064973 12835 ts_tablet_manager.cc:1434] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 14:56:10.067224 12835 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.078178 12835 tablet_bootstrap.cc:654] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.076771 12838 tablet_bootstrap.cc:654] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.092854 12837 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.092859 12835 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.093843 12835 ts_tablet_manager.cc:1403] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.005s
I20260709 14:56:10.093843 12837 ts_tablet_manager.cc:1403] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.059s	user 0.028s	sys 0.023s
I20260709 14:56:10.097802 12835 raft_consensus.cc:359] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.099195 12837 raft_consensus.cc:359] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.099874 12835 raft_consensus.cc:385] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.100087 12837 raft_consensus.cc:385] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.100472 12835 raft_consensus.cc:740] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.100579 12837 raft_consensus.cc:740] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.102166 12835 consensus_queue.cc:260] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.102253 12837 consensus_queue.cc:260] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.105217 12835 ts_tablet_manager.cc:1434] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 14:56:10.105279 12837 ts_tablet_manager.cc:1434] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 14:56:10.106755 12835 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.106755 12837 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.111835 12838 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.113026 12838 ts_tablet_manager.cc:1403] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.004s
I20260709 14:56:10.115667 12837 tablet_bootstrap.cc:654] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.115667 12835 tablet_bootstrap.cc:654] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.116762 12838 raft_consensus.cc:359] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.117630 12838 raft_consensus.cc:385] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.117959 12838 raft_consensus.cc:740] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.118903 12838 consensus_queue.cc:260] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.122018 12838 ts_tablet_manager.cc:1434] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 14:56:10.123510 12838 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.129321 12835 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.130457 12835 ts_tablet_manager.cc:1403] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.008s
I20260709 14:56:10.135078 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.135452 12835 raft_consensus.cc:359] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.137295 12835 raft_consensus.cc:385] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 14:56:10.137612 12684 leader_election.cc:343] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.137953 12835 raft_consensus.cc:740] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.141119 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.141276 12835 consensus_queue.cc:260] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
W20260709 14:56:10.143975 12682 leader_election.cc:343] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.144572 12682 leader_election.cc:304] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15534220b49c48e4a153b4563479d054; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.146011 12839 raft_consensus.cc:2749] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.146953 12837 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.147605 12837 ts_tablet_manager.cc:1403] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.008s
I20260709 14:56:10.148103 12835 ts_tablet_manager.cc:1434] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 14:56:10.149550 12835 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.149832 12838 tablet_bootstrap.cc:654] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.151510 12837 raft_consensus.cc:359] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.152437 12837 raft_consensus.cc:385] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.152757 12837 raft_consensus.cc:740] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.153885 12837 consensus_queue.cc:260] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.157644 12837 ts_tablet_manager.cc:1434] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:56:10.159588 12835 tablet_bootstrap.cc:654] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.160480 12837 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.172600 12837 tablet_bootstrap.cc:654] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.187860 12837 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.188854 12837 ts_tablet_manager.cc:1403] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.008s
I20260709 14:56:10.193126 12837 raft_consensus.cc:359] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.194046 12837 raft_consensus.cc:385] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.194401 12837 raft_consensus.cc:740] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.195554 12837 consensus_queue.cc:260] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.199415 12837 ts_tablet_manager.cc:1434] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.002s
I20260709 14:56:10.201246 12837 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.212217 12837 tablet_bootstrap.cc:654] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.219050 12835 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.219058 12837 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.220003 12835 ts_tablet_manager.cc:1403] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.071s	user 0.033s	sys 0.030s
I20260709 14:56:10.220018 12837 ts_tablet_manager.cc:1403] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 14:56:10.220794 12838 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.221479 12838 ts_tablet_manager.cc:1403] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.098s	user 0.023s	sys 0.043s
I20260709 14:56:10.223281 12837 raft_consensus.cc:359] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.224028 12837 raft_consensus.cc:385] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.224375 12837 raft_consensus.cc:740] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.223855 12835 raft_consensus.cc:359] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.224695 12838 raft_consensus.cc:359] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.225209 12837 consensus_queue.cc:260] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.225526 12835 raft_consensus.cc:385] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.225570 12838 raft_consensus.cc:385] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.226080 12835 raft_consensus.cc:740] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.226131 12838 raft_consensus.cc:740] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.227240 12835 consensus_queue.cc:260] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.227921 12837 ts_tablet_manager.cc:1434] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 14:56:10.227214 12838 consensus_queue.cc:260] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.230257 12835 ts_tablet_manager.cc:1434] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.008s
I20260709 14:56:10.231542 12835 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.231557 12837 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.240610 12837 tablet_bootstrap.cc:654] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.240609 12835 tablet_bootstrap.cc:654] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.243618 12838 ts_tablet_manager.cc:1434] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.002s
I20260709 14:56:10.245422 12838 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.254379 12835 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.255362 12835 ts_tablet_manager.cc:1403] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.011s
I20260709 14:56:10.255375 12838 tablet_bootstrap.cc:654] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.259903 12835 raft_consensus.cc:359] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.261219 12835 raft_consensus.cc:385] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.261686 12835 raft_consensus.cc:740] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.263216 12835 consensus_queue.cc:260] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.267800 12835 ts_tablet_manager.cc:1434] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.006s
I20260709 14:56:10.269678 12835 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.274484 12838 tablet_bootstrap.cc:492] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.275326 12838 ts_tablet_manager.cc:1403] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.030s	user 0.027s	sys 0.003s
I20260709 14:56:10.279353 12838 raft_consensus.cc:359] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.280040 12835 tablet_bootstrap.cc:654] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.280326 12838 raft_consensus.cc:385] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.280691 12838 raft_consensus.cc:740] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.281903 12838 consensus_queue.cc:260] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.288820 12838 ts_tablet_manager.cc:1434] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 14:56:10.289238 12837 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.289935 12837 ts_tablet_manager.cc:1403] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.034s
I20260709 14:56:10.289961 12838 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.293352 12837 raft_consensus.cc:359] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.294178 12837 raft_consensus.cc:385] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.294443 12837 raft_consensus.cc:740] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.295411 12837 consensus_queue.cc:260] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.297757 12843 raft_consensus.cc:493] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.297752 12835 tablet_bootstrap.cc:492] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.298740 12835 ts_tablet_manager.cc:1403] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.005s
I20260709 14:56:10.298460 12843 raft_consensus.cc:515] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.299111 12838 tablet_bootstrap.cc:654] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.302809 12835 raft_consensus.cc:359] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.303865 12835 raft_consensus.cc:385] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.304248 12835 raft_consensus.cc:740] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.305317 12835 consensus_queue.cc:260] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.308486 12835 ts_tablet_manager.cc:1434] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:56:10.309854 12835 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:10.316140 12843 leader_election.cc:290] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.317672 12835 tablet_bootstrap.cc:654] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.328027 12837 ts_tablet_manager.cc:1434] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.004s
I20260709 14:56:10.336345 12839 raft_consensus.cc:493] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.337338 12839 raft_consensus.cc:515] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.341224 12849 raft_consensus.cc:493] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.342501 12849 raft_consensus.cc:515] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.344766 12837 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.347036 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.349259 12849 leader_election.cc:290] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.350332 12849 raft_consensus.cc:493] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.351009 12849 raft_consensus.cc:515] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
W20260709 14:56:10.351589 12684 leader_election.cc:343] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.352991 12838 tablet_bootstrap.cc:492] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.353699 12838 ts_tablet_manager.cc:1403] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.064s	user 0.009s	sys 0.030s
I20260709 14:56:10.355064 12642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.355988 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.356357 12642 raft_consensus.cc:2468] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:10.357775 12838 raft_consensus.cc:359] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
W20260709 14:56:10.358208 12682 leader_election.cc:343] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.359548 12684 leader_election.cc:304] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.358119 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.360462 12839 leader_election.cc:290] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.360951 12643 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:10.361481 12839 raft_consensus.cc:2804] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.362076 12839 raft_consensus.cc:493] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.358688 12838 raft_consensus.cc:385] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.362658 12839 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.347019 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.349097 12835 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:10.364403 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.364820 12835 ts_tablet_manager.cc:1403] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.009s
I20260709 14:56:10.365159 12791 raft_consensus.cc:2468] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:10.366894 12682 leader_election.cc:304] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.362774 12684 leader_election.cc:304] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.369239 12849 leader_election.cc:290] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.369732 12843 raft_consensus.cc:493] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.370375 12849 raft_consensus.cc:2804] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.370297 12843 raft_consensus.cc:515] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.370868 12849 raft_consensus.cc:493] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.371274 12849 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.373553 12843 leader_election.cc:290] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.362188 12837 tablet_bootstrap.cc:654] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.362787 12838 raft_consensus.cc:740] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.376123 12835 raft_consensus.cc:359] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.380743 12835 raft_consensus.cc:385] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.381064 12835 raft_consensus.cc:740] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.381917 12835 consensus_queue.cc:260] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.382690 12868 raft_consensus.cc:2804] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.383423 12868 raft_consensus.cc:493] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.383172 12838 consensus_queue.cc:260] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.383858 12868 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.384339 12792 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:10.389581 12838 ts_tablet_manager.cc:1434] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.001s
I20260709 14:56:10.391490 12838 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.392369 12839 raft_consensus.cc:515] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.393512 12867 raft_consensus.cc:493] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.396452 12835 ts_tablet_manager.cc:1434] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 14:56:10.396302 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.397363 12643 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.399729 12844 raft_consensus.cc:493] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.400449 12837 tablet_bootstrap.cc:492] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.401191 12837 ts_tablet_manager.cc:1403] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.060s	user 0.006s	sys 0.009s
I20260709 14:56:10.400592 12844 raft_consensus.cc:515] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.396646 12839 leader_election.cc:290] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.405134 12837 raft_consensus.cc:359] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.405738 12844 leader_election.cc:290] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601)
I20260709 14:56:10.394225 12867 raft_consensus.cc:515] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.415835 12844 raft_consensus.cc:493] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.417003 12844 raft_consensus.cc:515] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.410389 12837 raft_consensus.cc:385] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.424459 12837 raft_consensus.cc:740] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.444707 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.437640 12843 raft_consensus.cc:493] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:56:10.447631 12682 leader_election.cc:343] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.447564 12843 raft_consensus.cc:515] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.447574 12642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.449002 12642 raft_consensus.cc:2468] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:10.450364 12843 leader_election.cc:290] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.431619 12868 raft_consensus.cc:515] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.451359 12684 leader_election.cc:304] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.451047 12837 consensus_queue.cc:260] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.456061 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.456489 12868 leader_election.cc:290] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.456549 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.457181 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.458359 12642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
W20260709 14:56:10.458778 12607 leader_election.cc:343] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.459088 12642 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.457957 12792 raft_consensus.cc:2468] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.460978 12791 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.461544 12607 leader_election.cc:304] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.462761 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.463686 12791 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.463889 12867 leader_election.cc:290] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.452497 12844 leader_election.cc:290] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:10.457345 12868 raft_consensus.cc:2804] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.441028 12838 tablet_bootstrap.cc:654] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.465867 12868 raft_consensus.cc:493] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.468532 12868 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.469523 12837 ts_tablet_manager.cc:1434] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.068s	user 0.002s	sys 0.010s
I20260709 14:56:10.471028 12837 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.467005 12843 raft_consensus.cc:2804] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.472962 12843 raft_consensus.cc:493] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.473453 12843 raft_consensus.cc:3060] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.475564 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.477011 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.478134 12718 raft_consensus.cc:2468] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.479645 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.480736 12717 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.480944 12849 raft_consensus.cc:515] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.481261 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.482061 12715 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:10.482767 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.483825 12716 raft_consensus.cc:2468] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.484493 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
W20260709 14:56:10.485885 12682 leader_election.cc:343] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.486204 12609 leader_election.cc:304] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.487505 12715 raft_consensus.cc:2468] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:10.489699 12609 leader_election.cc:304] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.490109 12874 raft_consensus.cc:2804] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.490672 12874 raft_consensus.cc:493] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.497903 12874 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.505198 12791 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.507040 12682 leader_election.cc:304] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:10.508054 12849 leader_election.cc:290] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.508811 12867 raft_consensus.cc:2804] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.509479 12867 raft_consensus.cc:697] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.510226 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.510557 12867 consensus_queue.cc:237] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.492954 12758 leader_election.cc:304] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
W20260709 14:56:10.517343 12684 leader_election.cc:343] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.521279 12642 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.523639 12868 raft_consensus.cc:515] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.525049 12643 raft_consensus.cc:2468] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.527046 12871 raft_consensus.cc:2804] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.527707 12871 raft_consensus.cc:493] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.527657 12684 leader_election.cc:304] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.528402 12871 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.529870 12849 raft_consensus.cc:2804] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.530637 12849 raft_consensus.cc:697] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.531766 12758 leader_election.cc:304] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:10.531759 12849 consensus_queue.cc:237] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.533262 12844 raft_consensus.cc:2804] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.533941 12844 raft_consensus.cc:493] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.534348 12844 raft_consensus.cc:3060] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.534790 12876 raft_consensus.cc:2804] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.537214 12876 raft_consensus.cc:493] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.538618 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.537539 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
W20260709 14:56:10.540716 12682 leader_election.cc:343] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.538921 12876 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.544708 12871 raft_consensus.cc:515] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.538069 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.546137 12791 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.550525 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.551678 12871 leader_election.cc:290] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601)
I20260709 14:56:10.551573 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.554875 12844 raft_consensus.cc:515] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.547501 12837 tablet_bootstrap.cc:654] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.541473 12868 leader_election.cc:290] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.548301 12838 tablet_bootstrap.cc:492] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.561724 12838 ts_tablet_manager.cc:1403] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.171s	user 0.011s	sys 0.012s
I20260709 14:56:10.548487 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.567423 12715 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.543601 12642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.555557 12643 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.575222 12874 raft_consensus.cc:515] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.559983 12843 raft_consensus.cc:515] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.575317 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.583624 12718 raft_consensus.cc:3060] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.585033 12843 leader_election.cc:290] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.585186 12791 raft_consensus.cc:2468] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.559528 12639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.586732 12682 leader_election.cc:304] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.588178 12867 raft_consensus.cc:2804] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.588279 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.588778 12867 raft_consensus.cc:697] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.589062 12641 raft_consensus.cc:2393] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 2e8307f11bd74ea49e419aac823055e4 in this term.
I20260709 14:56:10.590034 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242b524686de4239a46c51ee07876660" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.589831 12867 consensus_queue.cc:237] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.590914 12792 raft_consensus.cc:2393] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2e8307f11bd74ea49e419aac823055e4 in current term 1: Already voted for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in this term.
I20260709 14:56:10.577626 12643 raft_consensus.cc:2468] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.576743 12539 catalog_manager.cc:5697] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.579547 12838 raft_consensus.cc:359] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.581849 12876 raft_consensus.cc:515] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.592976 12838 raft_consensus.cc:385] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.593859 12838 raft_consensus.cc:740] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.593190 12844 leader_election.cc:290] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:10.596248 12609 leader_election.cc:304] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054
I20260709 14:56:10.597939 12684 leader_election.cc:304] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.598675 12876 leader_election.cc:290] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.599120 12868 raft_consensus.cc:2804] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.599537 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.599881 12868 raft_consensus.cc:697] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.600279 12716 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.600910 12868 consensus_queue.cc:237] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.598794 12843 raft_consensus.cc:2749] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.601711 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
W20260709 14:56:10.604367 12607 leader_election.cc:343] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.607574 12640 raft_consensus.cc:2393] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 2e8307f11bd74ea49e419aac823055e4 in this term.
I20260709 14:56:10.608702 12718 raft_consensus.cc:2468] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:10.611161 12758 leader_election.cc:304] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.610652 12838 consensus_queue.cc:260] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.612635 12844 raft_consensus.cc:2804] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.612901 12837 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.613488 12837 ts_tablet_manager.cc:1403] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.143s	user 0.016s	sys 0.029s
I20260709 14:56:10.617890 12837 raft_consensus.cc:359] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.622253 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.624547 12874 leader_election.cc:290] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.626377 12715 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:10.632344 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.637346 12837 raft_consensus.cc:385] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.637770 12837 raft_consensus.cc:740] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.637950 12792 raft_consensus.cc:2393] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2e8307f11bd74ea49e419aac823055e4 in current term 1: Already voted for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in this term.
I20260709 14:56:10.638571 12758 leader_election.cc:304] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.638738 12837 consensus_queue.cc:260] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.640153 12871 raft_consensus.cc:2804] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.641673 12837 ts_tablet_manager.cc:1434] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.003s
I20260709 14:56:10.642990 12837 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:10.643674 12607 leader_election.cc:304] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054
I20260709 14:56:10.643018 12874 raft_consensus.cc:493] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.646338 12874 raft_consensus.cc:515] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.649896 12837 tablet_bootstrap.cc:654] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.650759 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.651257 12874 leader_election.cc:290] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.651715 12715 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.652232 12874 raft_consensus.cc:2749] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.653540 12609 leader_election.cc:304] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.654877 12843 raft_consensus.cc:2804] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.655457 12843 raft_consensus.cc:493] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.655920 12843 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.664022 12837 tablet_bootstrap.cc:492] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:10.665092 12837 ts_tablet_manager.cc:1403] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 14:56:10.664917 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.666128 12792 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:10.669848 12837 raft_consensus.cc:359] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.670990 12837 raft_consensus.cc:385] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.671466 12837 raft_consensus.cc:740] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.672607 12837 consensus_queue.cc:260] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.675876 12837 ts_tablet_manager.cc:1434] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 14:56:10.684625 12844 raft_consensus.cc:697] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:10.682971 12838 ts_tablet_manager.cc:1434] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.121s	user 0.012s	sys 0.000s
I20260709 14:56:10.685808 12844 consensus_queue.cc:237] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.688645 12838 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.689700 12843 raft_consensus.cc:515] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.692312 12843 leader_election.cc:290] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.693073 12871 raft_consensus.cc:697] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:10.697650 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.700155 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.702039 12715 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.702539 12792 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.711953 12871 consensus_queue.cc:237] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:10.701386 12838 tablet_bootstrap.cc:654] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.695463 12716 raft_consensus.cc:2468] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:10.718416 12715 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:10.723361 12609 leader_election.cc:304] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.724442 12843 raft_consensus.cc:2804] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.725092 12843 raft_consensus.cc:697] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:10.725245 12609 leader_election.cc:304] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:10.726547 12876 raft_consensus.cc:2804] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.727167 12876 raft_consensus.cc:697] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:10.726148 12843 consensus_queue.cc:237] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.728125 12876 consensus_queue.cc:237] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.732956 12792 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:10.743106 12540 catalog_manager.cc:5697] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.749128 12868 raft_consensus.cc:493] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.749974 12868 raft_consensus.cc:515] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.752534 12868 leader_election.cc:290] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.753249 12838 tablet_bootstrap.cc:492] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.754092 12838 ts_tablet_manager.cc:1403] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.007s
I20260709 14:56:10.754206 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:10.754629 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.755549 12640 raft_consensus.cc:2468] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
W20260709 14:56:10.756233 12682 leader_election.cc:343] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.757735 12684 leader_election.cc:304] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.758694 12838 raft_consensus.cc:359] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.759317 12868 raft_consensus.cc:2804] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:10.759732 12838 raft_consensus.cc:385] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.759807 12868 raft_consensus.cc:493] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:10.760157 12838 raft_consensus.cc:740] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.760295 12868 raft_consensus.cc:3060] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.761171 12838 consensus_queue.cc:260] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.764050 12838 ts_tablet_manager.cc:1434] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 14:56:10.765439 12838 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.770350 12868 raft_consensus.cc:515] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.773502 12838 tablet_bootstrap.cc:654] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.775136 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:10.776024 12640 raft_consensus.cc:3060] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.788878 12640 raft_consensus.cc:2468] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:10.789384 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.790378 12538 catalog_manager.cc:5697] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } }
W20260709 14:56:10.792407 12682 leader_election.cc:343] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:10.792701 12538 catalog_manager.cc:5697] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:10.793879 12684 leader_election.cc:304] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:10.795162 12868 leader_election.cc:290] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.797369 12868 raft_consensus.cc:2804] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:10.797899 12868 raft_consensus.cc:697] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.798915 12868 consensus_queue.cc:237] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.800400 12838 tablet_bootstrap.cc:492] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.801113 12838 ts_tablet_manager.cc:1403] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.016s
I20260709 14:56:10.802292 12539 catalog_manager.cc:5697] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.804821 12838 raft_consensus.cc:359] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.806035 12838 raft_consensus.cc:385] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.806463 12838 raft_consensus.cc:740] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.807308 12540 catalog_manager.cc:5697] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.807749 12838 consensus_queue.cc:260] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.811636 12838 ts_tablet_manager.cc:1434] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 14:56:10.809377 12540 catalog_manager.cc:5697] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.814030 12838 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:10.814386 12540 catalog_manager.cc:5697] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.822886 12838 tablet_bootstrap.cc:654] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:10.829005 12838 tablet_bootstrap.cc:492] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:10.829711 12838 ts_tablet_manager.cc:1403] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.006s
I20260709 14:56:10.832952 12838 raft_consensus.cc:359] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.833746 12838 raft_consensus.cc:385] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:10.834039 12838 raft_consensus.cc:740] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:10.834877 12838 consensus_queue.cc:260] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.835932 12539 catalog_manager.cc:5697] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:10.837337 12838 ts_tablet_manager.cc:1434] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 14:56:10.840175 12871 raft_consensus.cc:493] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.840919 12871 raft_consensus.cc:515] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.843398 12871 leader_election.cc:290] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:10.844452 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.844856 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd619b7389e14e05859cb4dc49118e0b" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.845590 12640 raft_consensus.cc:2393] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 15534220b49c48e4a153b4563479d054 in this term.
I20260709 14:56:10.847395 12758 leader_election.cc:304] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3; no voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.848412 12871 raft_consensus.cc:3060] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.856575 12871 raft_consensus.cc:2749] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.878922 12792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:36482
I20260709 14:56:10.879839 12792 raft_consensus.cc:606] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.896746 12716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:59704
I20260709 14:56:10.897475 12716 raft_consensus.cc:606] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.904304 12716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a86407143c684a358b111eaada3f75d5"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:59704
I20260709 14:56:10.904992 12716 raft_consensus.cc:606] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.910981 12716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:59704
I20260709 14:56:10.911625 12716 raft_consensus.cc:606] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.921223 12871 raft_consensus.cc:493] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.921998 12871 raft_consensus.cc:515] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.924578 12871 leader_election.cc:290] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:10.925518 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.926177 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.926577 12640 raft_consensus.cc:2393] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 15534220b49c48e4a153b4563479d054 in this term.
I20260709 14:56:10.928431 12758 leader_election.cc:304] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3; no voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.928905 12643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:43504
I20260709 14:56:10.929684 12643 raft_consensus.cc:606] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Received request to transfer leadership to TS 15534220b49c48e4a153b4563479d054
I20260709 14:56:10.929891 12871 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.930050 12868 consensus_queue.cc:1048] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:10.939981 12871 raft_consensus.cc:2749] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.942435 12867 raft_consensus.cc:993] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:10.943207 12849 raft_consensus.cc:1081] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:10.942574 12643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:43504
I20260709 14:56:10.944180 12643 raft_consensus.cc:606] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Received request to transfer leadership to TS 15534220b49c48e4a153b4563479d054
I20260709 14:56:10.946625 12643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43490
I20260709 14:56:10.952729 12643 raft_consensus.cc:493] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:10.953341 12643 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:10.956169 12792 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.958235 12871 raft_consensus.cc:493] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:10.963057 12643 raft_consensus.cc:515] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.958957 12871 raft_consensus.cc:515] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.967176 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:10.968807 12867 consensus_queue.cc:1048] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:10.970121 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:10.971158 12716 raft_consensus.cc:2393] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 2e8307f11bd74ea49e419aac823055e4 in this term.
I20260709 14:56:10.972892 12758 leader_election.cc:304] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3; no voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:10.972096 12871 leader_election.cc:290] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:10.973913 12844 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:10.982656 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:10.983088 12643 leader_election.cc:290] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:10.983672 12716 raft_consensus.cc:3055] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:10.984088 12716 raft_consensus.cc:740] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:10.982523 12867 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:10.985076 12716 consensus_queue.cc:260] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:10.987326 12716 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:10.987783 12844 raft_consensus.cc:2749] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:10.984846 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb41a14b8ed34f7d9fffa0a7cb555dc2" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:10.989056 12791 raft_consensus.cc:3060] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:10.998413 12791 raft_consensus.cc:2468] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.000029 12868 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.004546 12607 leader_election.cc:304] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:11.005579 12876 raft_consensus.cc:2804] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.006182 12876 raft_consensus.cc:697] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.007241 12876 consensus_queue.cc:237] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.011904 12716 raft_consensus.cc:2468] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.014633 12849 raft_consensus.cc:993] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.015229 12849 raft_consensus.cc:1081] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.017735 12643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43490
I20260709 14:56:11.018455 12643 raft_consensus.cc:493] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:11.018906 12643 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.021402 12867 consensus_queue.cc:1048] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.026510 12643 raft_consensus.cc:515] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.030157 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:11.030970 12716 raft_consensus.cc:3055] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:11.031332 12716 raft_consensus.cc:740] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:11.034759 12716 consensus_queue.cc:260] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.036286 12716 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.038060 12867 consensus_queue.cc:1048] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.034462 12540 catalog_manager.cc:5697] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:11.040367 12869 consensus_queue.cc:1048] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.045117 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:11.045895 12791 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.046562 12716 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.052495 12643 leader_election.cc:290] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.052942 12609 leader_election.cc:304] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:11.054136 12876 raft_consensus.cc:2804] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.055749 12791 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.054719 12876 raft_consensus.cc:697] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.061102 12876 consensus_queue.cc:237] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.071269 12640 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:11.076903 12540 catalog_manager.cc:5697] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:11.082573 12876 raft_consensus.cc:493] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:11.083457 12876 raft_consensus.cc:515] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.087152 12867 consensus_queue.cc:1048] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.089632 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:11.088209 12876 leader_election.cc:290] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.089998 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:11.092391 12607 leader_election.cc:304] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054
I20260709 14:56:11.098357 12876 raft_consensus.cc:2749] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 14:56:11.102360 12868 raft_consensus.cc:993] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.103116 12867 raft_consensus.cc:1081] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.105181 12640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a86407143c684a358b111eaada3f75d5"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43490
I20260709 14:56:11.106007 12640 raft_consensus.cc:493] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:11.106468 12640 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.113869 12640 raft_consensus.cc:515] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.116426 12640 leader_election.cc:290] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.117888 12716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:11.118760 12716 raft_consensus.cc:3055] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:11.118654 12876 consensus_queue.cc:1048] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.119271 12716 raft_consensus.cc:740] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:11.118101 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86407143c684a358b111eaada3f75d5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:11.120607 12790 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.120203 12716 consensus_queue.cc:260] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.121836 12716 raft_consensus.cc:3060] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.131409 12790 raft_consensus.cc:2468] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.131477 12716 raft_consensus.cc:2468] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.133219 12607 leader_election.cc:304] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:11.134373 12876 raft_consensus.cc:2804] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.134999 12876 raft_consensus.cc:697] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.136008 12876 consensus_queue.cc:237] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.142515 12876 consensus_queue.cc:1048] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.149287 12911 consensus_queue.cc:1048] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.151427 12538 catalog_manager.cc:5697] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:11.158818 12876 raft_consensus.cc:993] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:11.159561 12911 raft_consensus.cc:1081] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Signalling peer 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:11.166534 12715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739"
dest_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:59686
I20260709 14:56:11.167325 12715 raft_consensus.cc:493] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:11.167855 12715 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.174986 12876 consensus_queue.cc:1048] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.181571 12715 raft_consensus.cc:515] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.185664 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:11.186486 12876 raft_consensus.cc:993] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:11.187132 12876 raft_consensus.cc:1081] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Signalling peer 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:11.186493 12640 raft_consensus.cc:3055] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:11.188633 12640 raft_consensus.cc:740] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.188755 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95f790959a342c5bfea1c293cf1a739" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:11.189528 12791 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.189672 12640 consensus_queue.cc:260] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.191185 12640 raft_consensus.cc:3060] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.193665 12718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12"
dest_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:59686
I20260709 14:56:11.194394 12718 raft_consensus.cc:493] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:11.194797 12718 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.198712 12791 raft_consensus.cc:2468] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:11.198985 12715 leader_election.cc:290] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.200723 12682 leader_election.cc:304] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:11.200801 12640 raft_consensus.cc:2468] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:11.201737 12867 raft_consensus.cc:2804] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.202353 12867 raft_consensus.cc:697] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:11.203374 12867 consensus_queue.cc:237] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.204483 12718 raft_consensus.cc:515] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.206992 12718 leader_election.cc:290] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.207826 12643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:11.208581 12643 raft_consensus.cc:3055] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:11.208959 12643 raft_consensus.cc:740] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.209749 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7fdd7742cb4669b10b276d7e672f12" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:11.209663 12844 consensus_queue.cc:1048] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.209903 12643 consensus_queue.cc:260] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.210589 12791 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.211838 12643 raft_consensus.cc:3060] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.222978 12643 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:11.222232 12539 catalog_manager.cc:5697] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 1 to 2, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:11.225198 12684 leader_election.cc:304] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:11.226495 12867 raft_consensus.cc:2804] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.227126 12867 raft_consensus.cc:697] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:11.228134 12867 consensus_queue.cc:237] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:11.234568 12791 raft_consensus.cc:2468] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:11.241375 12844 consensus_queue.cc:1048] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.250137 12844 consensus_queue.cc:1048] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.255149 12540 catalog_manager.cc:5697] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 1 to 2, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:11.260295 12916 consensus_queue.cc:1048] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.274850 12871 raft_consensus.cc:993] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.275904 12916 raft_consensus.cc:1081] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:11.280517 12643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43492
I20260709 14:56:11.287211 12643 raft_consensus.cc:493] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:11.287926 12643 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.293255 12867 consensus_queue.cc:1048] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.298964 12643 raft_consensus.cc:515] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:11.302886 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:11.303591 12643 leader_election.cc:290] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:11.303701 12718 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.303828 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea32fc7c4e7f41cca6beaa1694925a61" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:11.304736 12790 raft_consensus.cc:3055] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:11.305086 12790 raft_consensus.cc:740] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:11.306041 12790 consensus_queue.cc:260] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:11.307389 12790 raft_consensus.cc:3060] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:11.314435 12868 consensus_queue.cc:1048] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:11.316803 12790 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.318307 12607 leader_election.cc:304] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:11.319458 12874 raft_consensus.cc:2804] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:11.320170 12874 raft_consensus.cc:697] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:11.321429 12874 consensus_queue.cc:237] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } }
I20260709 14:56:11.324651 12718 raft_consensus.cc:2468] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:11.338521 12539 catalog_manager.cc:5697] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:11.399662 12790 raft_consensus.cc:1275] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.401438 12874 consensus_queue.cc:1048] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.410871 12718 raft_consensus.cc:1275] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.413419 12874 consensus_queue.cc:1048] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:11.546432 12790 raft_consensus.cc:1275] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.548336 12874 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.557896 12718 raft_consensus.cc:1275] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.560657 12874 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:11.623232 12643 raft_consensus.cc:1275] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.625061 12868 consensus_queue.cc:1048] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.634940 12790 raft_consensus.cc:1275] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.636699 12867 consensus_queue.cc:1048] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.677858 12790 raft_consensus.cc:1275] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.679481 12849 consensus_queue.cc:1048] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.688850 12643 raft_consensus.cc:1275] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.690725 12867 consensus_queue.cc:1048] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:11.742734 12718 raft_consensus.cc:1275] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.744488 12874 consensus_queue.cc:1048] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.745579 12790 raft_consensus.cc:1275] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.747323 12874 consensus_queue.cc:1048] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.781582 12718 raft_consensus.cc:1275] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.783167 12874 consensus_queue.cc:1048] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:11.792816 12790 raft_consensus.cc:1275] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:11.796401 12874 consensus_queue.cc:1048] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:13.989135 12540 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33422:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:13.991798 12540 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:14.050904 12631 tablet_service.cc:1511] Processing CreateTablet for tablet cf5594b24f1e4d0396677cbe2bb0f9de (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:14.052037 12703 tablet_service.cc:1511] Processing CreateTablet for tablet cf5594b24f1e4d0396677cbe2bb0f9de (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:14.052486 12631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5594b24f1e4d0396677cbe2bb0f9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.053539 12703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5594b24f1e4d0396677cbe2bb0f9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.053381 12774 tablet_service.cc:1511] Processing CreateTablet for tablet cf5594b24f1e4d0396677cbe2bb0f9de (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:14.054790 12774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5594b24f1e4d0396677cbe2bb0f9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.060608 12779 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec3eee1d9064a1d8c288f1ea6139109 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:14.062444 12779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec3eee1d9064a1d8c288f1ea6139109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.062624 12700 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec3eee1d9064a1d8c288f1ea6139109 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:14.062953 12704 tablet_service.cc:1511] Processing CreateTablet for tablet 01d5a98a9bb846f88c319b3d57f58814 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:14.064354 12700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec3eee1d9064a1d8c288f1ea6139109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.067047 12627 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec3eee1d9064a1d8c288f1ea6139109 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:14.067948 12777 tablet_service.cc:1511] Processing CreateTablet for tablet 01d5a98a9bb846f88c319b3d57f58814 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:14.068668 12627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec3eee1d9064a1d8c288f1ea6139109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.069433 12777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d5a98a9bb846f88c319b3d57f58814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.070624 12628 tablet_service.cc:1511] Processing CreateTablet for tablet 01d5a98a9bb846f88c319b3d57f58814 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:14.071895 12704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d5a98a9bb846f88c319b3d57f58814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.072057 12628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d5a98a9bb846f88c319b3d57f58814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.092738 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc82df68e504347972d5622545eb022 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:14.096421 12778 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc82df68e504347972d5622545eb022 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:14.097384 12626 tablet_service.cc:1511] Processing CreateTablet for tablet 8862ef85c4304d71b788f5bd839a56fd (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:14.098905 12626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8862ef85c4304d71b788f5bd839a56fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 14:56:14.106755 12589 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:14.090000 12632 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc82df68e504347972d5622545eb022 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:14.108666 12632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc82df68e504347972d5622545eb022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.110682 12707 tablet_service.cc:1511] Processing CreateTablet for tablet 8862ef85c4304d71b788f5bd839a56fd (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:14.122855 12775 tablet_service.cc:1511] Processing CreateTablet for tablet 8862ef85c4304d71b788f5bd839a56fd (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:14.124074 12935 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.126478 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc82df68e504347972d5622545eb022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.127319 12707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8862ef85c4304d71b788f5bd839a56fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.128770 12778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc82df68e504347972d5622545eb022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.131354 12775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8862ef85c4304d71b788f5bd839a56fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.154346 12935 tablet_bootstrap.cc:654] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.154527 12625 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfd814c6064479d8750d6aa26b24bfe (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:14.156131 12625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfd814c6064479d8750d6aa26b24bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.160866 12782 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfd814c6064479d8750d6aa26b24bfe (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:14.165132 12782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfd814c6064479d8750d6aa26b24bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.168169 12703 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfd814c6064479d8750d6aa26b24bfe (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:14.169786 12703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfd814c6064479d8750d6aa26b24bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.178903 12630 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3b23b4ade94168b7ae1c604de2f5ee (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:14.180691 12630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3b23b4ade94168b7ae1c604de2f5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.182116 12705 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3b23b4ade94168b7ae1c604de2f5ee (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:14.183935 12705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3b23b4ade94168b7ae1c604de2f5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.198395 12781 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3b23b4ade94168b7ae1c604de2f5ee (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:14.200207 12781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3b23b4ade94168b7ae1c604de2f5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.201968 12935 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.202729 12935 ts_tablet_manager.cc:1403] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.003s
I20260709 14:56:14.202744 12937 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.216970 12628 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6b75345f024f5c9fd885daa98a6dd7 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:14.218803 12628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6b75345f024f5c9fd885daa98a6dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.219578 12937 tablet_bootstrap.cc:654] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.206421 12935 raft_consensus.cc:359] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.224717 12935 raft_consensus.cc:385] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.225126 12935 raft_consensus.cc:740] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.226135 12935 consensus_queue.cc:260] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.232024 12935 ts_tablet_manager.cc:1434] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 14:56:14.233371 12935 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.233520 12937 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.234208 12937 ts_tablet_manager.cc:1403] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.008s
I20260709 14:56:14.238030 12937 raft_consensus.cc:359] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.239142 12937 raft_consensus.cc:385] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.239475 12937 raft_consensus.cc:740] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.240298 12937 consensus_queue.cc:260] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.242775 12935 tablet_bootstrap.cc:654] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.211324 12936 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.243105 12937 ts_tablet_manager.cc:1434] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 14:56:14.244402 12937 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.252468 12937 tablet_bootstrap.cc:654] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.258993 12703 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6b75345f024f5c9fd885daa98a6dd7 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:14.260766 12703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6b75345f024f5c9fd885daa98a6dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.301508 12775 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6b75345f024f5c9fd885daa98a6dd7 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:14.308764 12625 tablet_service.cc:1511] Processing CreateTablet for tablet 22e2aa545c75491b842dad6f579733e5 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:14.310649 12625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e2aa545c75491b842dad6f579733e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.324257 12703 tablet_service.cc:1511] Processing CreateTablet for tablet 22e2aa545c75491b842dad6f579733e5 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:14.326089 12703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e2aa545c75491b842dad6f579733e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.331044 12775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6b75345f024f5c9fd885daa98a6dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.337586 12936 tablet_bootstrap.cc:654] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.345804 12779 tablet_service.cc:1511] Processing CreateTablet for tablet 22e2aa545c75491b842dad6f579733e5 (DEFAULT_TABLE table=soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:14.347613 12779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e2aa545c75491b842dad6f579733e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:14.348718 12935 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.349437 12935 ts_tablet_manager.cc:1403] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.116s	user 0.025s	sys 0.016s
I20260709 14:56:14.358439 12937 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.353245 12935 raft_consensus.cc:359] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.359256 12937 ts_tablet_manager.cc:1403] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.115s	user 0.038s	sys 0.055s
I20260709 14:56:14.359524 12935 raft_consensus.cc:385] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.359905 12935 raft_consensus.cc:740] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.363366 12937 raft_consensus.cc:359] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.364234 12937 raft_consensus.cc:385] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.364574 12937 raft_consensus.cc:740] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.363252 12935 consensus_queue.cc:260] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.365460 12937 consensus_queue.cc:260] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.369022 12935 ts_tablet_manager.cc:1434] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 14:56:14.370347 12935 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.372725 12937 ts_tablet_manager.cc:1434] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 14:56:14.382237 12937 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.409431 12867 raft_consensus.cc:493] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.410437 12867 raft_consensus.cc:515] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.447894 12936 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.449635 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:14.454005 12936 ts_tablet_manager.cc:1403] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.243s	user 0.034s	sys 0.013s
W20260709 14:56:14.455428 12684 leader_election.cc:343] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:14.456529 12937 tablet_bootstrap.cc:654] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.458505 12936 raft_consensus.cc:359] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.459710 12936 raft_consensus.cc:385] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.460068 12936 raft_consensus.cc:740] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.460990 12936 consensus_queue.cc:260] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.461755 12867 leader_election.cc:290] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.464870 12936 ts_tablet_manager.cc:1434] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.006s
I20260709 14:56:14.465154 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:14.466049 12791 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:14.466252 12936 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.470783 12935 tablet_bootstrap.cc:654] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.473419 12682 leader_election.cc:304] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:14.477061 12936 tablet_bootstrap.cc:654] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.475980 12867 raft_consensus.cc:2804] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.484092 12867 raft_consensus.cc:493] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.484746 12867 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.489308 12937 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.490062 12937 ts_tablet_manager.cc:1403] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.108s	user 0.020s	sys 0.013s
I20260709 14:56:14.491261 12936 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.491963 12936 ts_tablet_manager.cc:1403] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.010s
I20260709 14:56:14.493009 12935 tablet_bootstrap.cc:492] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.493692 12935 ts_tablet_manager.cc:1403] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.124s	user 0.010s	sys 0.015s
I20260709 14:56:14.495883 12936 raft_consensus.cc:359] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.496855 12936 raft_consensus.cc:385] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.497217 12936 raft_consensus.cc:740] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.497488 12935 raft_consensus.cc:359] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.498145 12936 consensus_queue.cc:260] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.500061 12935 raft_consensus.cc:385] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.500520 12935 raft_consensus.cc:740] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.501055 12937 raft_consensus.cc:359] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.501730 12935 consensus_queue.cc:260] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.502398 12937 raft_consensus.cc:385] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.502768 12937 raft_consensus.cc:740] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.503460 12936 ts_tablet_manager.cc:1434] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.002s
I20260709 14:56:14.504506 12935 ts_tablet_manager.cc:1434] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 14:56:14.504908 12936 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.506311 12937 consensus_queue.cc:260] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.507242 12935 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.509815 12916 raft_consensus.cc:493] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.511139 12916 raft_consensus.cc:515] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.513521 12937 ts_tablet_manager.cc:1434] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 14:56:14.515218 12937 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.516882 12936 tablet_bootstrap.cc:654] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.518419 12935 tablet_bootstrap.cc:654] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.530136 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:14.531421 12640 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:14.533334 12937 tablet_bootstrap.cc:654] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.533967 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:14.535162 12718 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:14.538334 12758 leader_election.cc:304] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.542919 12935 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.543710 12935 ts_tablet_manager.cc:1403] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.016s
I20260709 14:56:14.544189 12936 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.544260 12937 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.545051 12937 ts_tablet_manager.cc:1403] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.005s
I20260709 14:56:14.513605 12916 leader_election.cc:290] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:14.544899 12936 ts_tablet_manager.cc:1403] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.009s
I20260709 14:56:14.546484 12916 raft_consensus.cc:2804] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.547101 12916 raft_consensus.cc:493] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.547573 12916 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.549610 12937 raft_consensus.cc:359] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.550879 12937 raft_consensus.cc:385] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.550340 12936 raft_consensus.cc:359] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.551266 12937 raft_consensus.cc:740] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.551467 12936 raft_consensus.cc:385] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.551971 12936 raft_consensus.cc:740] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.552409 12937 consensus_queue.cc:260] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.553057 12936 consensus_queue.cc:260] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.555343 12937 ts_tablet_manager.cc:1434] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 14:56:14.556499 12936 ts_tablet_manager.cc:1434] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 14:56:14.556365 12935 raft_consensus.cc:359] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.557389 12935 raft_consensus.cc:385] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.557730 12935 raft_consensus.cc:740] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.558027 12936 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.558991 12937 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.558681 12935 consensus_queue.cc:260] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.561821 12935 ts_tablet_manager.cc:1434] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.003s
I20260709 14:56:14.563459 12935 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.568778 12937 tablet_bootstrap.cc:654] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.568840 12936 tablet_bootstrap.cc:654] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.572366 12935 tablet_bootstrap.cc:654] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.579725 12936 tablet_bootstrap.cc:492] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.580364 12936 ts_tablet_manager.cc:1403] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.011s
I20260709 14:56:14.584427 12936 raft_consensus.cc:359] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.585453 12936 raft_consensus.cc:385] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.585834 12936 raft_consensus.cc:740] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.588923 12924 raft_consensus.cc:493] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.589923 12924 raft_consensus.cc:515] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.591300 12936 consensus_queue.cc:260] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.593389 12937 tablet_bootstrap.cc:492] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.594129 12937 ts_tablet_manager.cc:1403] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.010s
I20260709 14:56:14.595831 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:14.597018 12790 raft_consensus.cc:2468] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.597793 12937 raft_consensus.cc:359] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.598657 12937 raft_consensus.cc:385] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.599025 12937 raft_consensus.cc:740] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.599210 12607 leader_election.cc:304] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.599908 12937 consensus_queue.cc:260] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.602205 12924 leader_election.cc:290] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.602771 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:14.604957 12935 tablet_bootstrap.cc:492] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
W20260709 14:56:14.604833 12609 leader_election.cc:343] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:14.603055 12924 raft_consensus.cc:2804] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.605677 12935 ts_tablet_manager.cc:1403] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.020s
I20260709 14:56:14.606205 12924 raft_consensus.cc:493] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.606721 12924 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.608490 12867 raft_consensus.cc:515] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.613190 12935 raft_consensus.cc:359] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.617482 12924 raft_consensus.cc:515] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.621052 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:14.614533 12935 raft_consensus.cc:385] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.621945 12935 raft_consensus.cc:740] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.622493 12924 leader_election.cc:290] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.623167 12935 consensus_queue.cc:260] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.626456 12718 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.626710 12935 ts_tablet_manager.cc:1434] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 14:56:14.643748 12937 ts_tablet_manager.cc:1434] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.049s	user 0.003s	sys 0.012s
I20260709 14:56:14.643985 12936 ts_tablet_manager.cc:1434] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.063s	user 0.014s	sys 0.008s
I20260709 14:56:14.644193 12935 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.641022 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:14.648713 12790 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.645236 12937 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.637492 12957 raft_consensus.cc:493] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.649575 12957 raft_consensus.cc:515] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.652171 12957 leader_election.cc:290] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.642791 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:14.653342 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec3eee1d9064a1d8c288f1ea6139109" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:14.653908 12791 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.654189 12715 raft_consensus.cc:2468] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.637848 12718 raft_consensus.cc:2468] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:14.645684 12936 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.634207 12916 raft_consensus.cc:515] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.655898 12792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec3eee1d9064a1d8c288f1ea6139109" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:14.656675 12792 raft_consensus.cc:2468] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.657686 12916 leader_election.cc:290] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:14.641897 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:14.658154 12607 leader_election.cc:304] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.658633 12640 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:56:14.629269 12758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:34601, user_credentials={real_user=slave}} blocked reactor thread for 91119.7us
I20260709 14:56:14.660025 12958 raft_consensus.cc:2804] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.660553 12958 raft_consensus.cc:493] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.660929 12958 raft_consensus.cc:3060] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.661259 12937 tablet_bootstrap.cc:654] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.668018 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:14.668607 12609 leader_election.cc:304] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.671195 12641 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.669214 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:14.675631 12715 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.677196 12935 tablet_bootstrap.cc:654] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.685148 12715 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:14.642664 12867 leader_election.cc:290] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.686890 12957 raft_consensus.cc:2804] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:14.686956 12758 leader_election.cc:304] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:14.687677 12957 raft_consensus.cc:697] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:14.688961 12916 raft_consensus.cc:2804] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:14.688849 12957 consensus_queue.cc:237] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.689843 12916 raft_consensus.cc:697] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:14.691164 12916 consensus_queue.cc:237] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.713297 12935 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.714293 12935 ts_tablet_manager.cc:1403] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.011s
I20260709 14:56:14.724599 12958 raft_consensus.cc:515] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.724857 12935 raft_consensus.cc:359] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.726114 12935 raft_consensus.cc:385] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.726590 12935 raft_consensus.cc:740] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.728037 12935 consensus_queue.cc:260] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.730103 12641 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:14.732801 12935 ts_tablet_manager.cc:1434] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.018s	user 0.013s	sys 0.001s
I20260709 14:56:14.733230 12958 leader_election.cc:290] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.733879 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec3eee1d9064a1d8c288f1ea6139109" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:14.735025 12789 raft_consensus.cc:3060] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.732475 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec3eee1d9064a1d8c288f1ea6139109" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:14.737479 12715 raft_consensus.cc:3060] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.740150 12538 catalog_manager.cc:5697] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:14.746399 12790 raft_consensus.cc:2468] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:14.744763 12936 tablet_bootstrap.cc:654] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.735121 12791 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:14.751629 12935 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.752333 12682 leader_election.cc:304] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:14.753593 12867 raft_consensus.cc:2804] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:14.754199 12867 raft_consensus.cc:697] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:14.755183 12867 consensus_queue.cc:237] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.745424 12640 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:14.758690 12715 raft_consensus.cc:2468] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:14.767463 12609 leader_election.cc:304] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.760247 12789 raft_consensus.cc:2468] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:14.771348 12957 raft_consensus.cc:2804] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:14.772222 12957 raft_consensus.cc:697] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:14.773281 12957 consensus_queue.cc:237] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.775449 12935 tablet_bootstrap.cc:654] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.781301 12937 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.780629 12540 catalog_manager.cc:5697] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:14.781950 12937 ts_tablet_manager.cc:1403] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.137s	user 0.046s	sys 0.065s
I20260709 14:56:14.785684 12937 raft_consensus.cc:359] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.786550 12937 raft_consensus.cc:385] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.786900 12937 raft_consensus.cc:740] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.787741 12937 consensus_queue.cc:260] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.790410 12937 ts_tablet_manager.cc:1434] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:56:14.794118 12937 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.806959 12936 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.807057 12937 tablet_bootstrap.cc:654] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.807840 12936 ts_tablet_manager.cc:1403] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.162s	user 0.012s	sys 0.014s
I20260709 14:56:14.810961 12935 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.812224 12935 ts_tablet_manager.cc:1403] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.010s
I20260709 14:56:14.815768 12935 raft_consensus.cc:359] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.816685 12935 raft_consensus.cc:385] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.817057 12935 raft_consensus.cc:740] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.817924 12935 consensus_queue.cc:260] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.818305 12539 catalog_manager.cc:5697] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:14.820873 12935 ts_tablet_manager.cc:1434] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:56:14.822116 12935 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.828336 12936 raft_consensus.cc:359] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.829385 12936 raft_consensus.cc:385] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.829712 12936 raft_consensus.cc:740] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.830735 12936 consensus_queue.cc:260] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.833802 12936 ts_tablet_manager.cc:1434] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 14:56:14.835068 12936 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.837893 12937 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.838894 12937 ts_tablet_manager.cc:1403] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.012s
I20260709 14:56:14.841300 12935 tablet_bootstrap.cc:654] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.842974 12936 tablet_bootstrap.cc:654] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.842759 12937 raft_consensus.cc:359] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.844468 12937 raft_consensus.cc:385] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.844928 12937 raft_consensus.cc:740] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.846396 12937 consensus_queue.cc:260] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.850059 12937 ts_tablet_manager.cc:1434] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 14:56:14.852057 12937 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.855947 12935 tablet_bootstrap.cc:492] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.856719 12935 ts_tablet_manager.cc:1403] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.000s
I20260709 14:56:14.861018 12937 tablet_bootstrap.cc:654] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.861339 12935 raft_consensus.cc:359] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.862254 12935 raft_consensus.cc:385] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.862524 12935 raft_consensus.cc:740] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.863300 12935 consensus_queue.cc:260] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.863643 12538 catalog_manager.cc:5697] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:14.865700 12935 ts_tablet_manager.cc:1434] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 14:56:14.870429 12936 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.871202 12936 ts_tablet_manager.cc:1403] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.003s
I20260709 14:56:14.872284 12935 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: Bootstrap starting.
I20260709 14:56:14.874781 12936 raft_consensus.cc:359] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.875720 12936 raft_consensus.cc:385] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.876046 12936 raft_consensus.cc:740] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.876906 12936 consensus_queue.cc:260] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.883662 12935 tablet_bootstrap.cc:654] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.883872 12936 ts_tablet_manager.cc:1434] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 14:56:14.885250 12937 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.885979 12937 ts_tablet_manager.cc:1403] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.009s
I20260709 14:56:14.890041 12937 raft_consensus.cc:359] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.891077 12937 raft_consensus.cc:385] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.891433 12937 raft_consensus.cc:740] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.885268 12936 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.900034 12937 consensus_queue.cc:260] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.902171 12937 ts_tablet_manager.cc:1434] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 14:56:14.903439 12937 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: Bootstrap starting.
I20260709 14:56:14.904633 12936 tablet_bootstrap.cc:654] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.913060 12937 tablet_bootstrap.cc:654] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.918502 12936 tablet_bootstrap.cc:492] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.919186 12935 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: No bootstrap required, opened a new log
I20260709 14:56:14.919209 12936 ts_tablet_manager.cc:1403] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.009s
I20260709 14:56:14.920136 12935 ts_tablet_manager.cc:1403] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.009s
I20260709 14:56:14.923761 12935 raft_consensus.cc:359] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.923691 12936 raft_consensus.cc:359] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.924674 12935 raft_consensus.cc:385] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.924755 12936 raft_consensus.cc:385] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.925088 12935 raft_consensus.cc:740] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.925184 12936 raft_consensus.cc:740] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.925429 12937 tablet_bootstrap.cc:492] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: No bootstrap required, opened a new log
I20260709 14:56:14.926424 12937 ts_tablet_manager.cc:1403] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 14:56:14.926098 12936 consensus_queue.cc:260] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.928941 12936 ts_tablet_manager.cc:1434] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:56:14.930462 12936 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.930279 12937 raft_consensus.cc:359] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.931252 12937 raft_consensus.cc:385] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.931573 12937 raft_consensus.cc:740] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.932397 12937 consensus_queue.cc:260] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.935011 12937 ts_tablet_manager.cc:1434] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 14:56:14.926115 12935 consensus_queue.cc:260] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.940382 12936 tablet_bootstrap.cc:654] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.941398 12935 ts_tablet_manager.cc:1434] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 14:56:14.942147 12957 raft_consensus.cc:493] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.942751 12957 raft_consensus.cc:515] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.946730 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:14.947521 12957 leader_election.cc:290] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.947595 12715 raft_consensus.cc:2468] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.951643 12936 tablet_bootstrap.cc:492] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.952291 12936 ts_tablet_manager.cc:1403] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.012s
I20260709 14:56:14.952760 12609 leader_election.cc:304] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.954082 12957 raft_consensus.cc:2804] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.954499 12957 raft_consensus.cc:493] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.954576 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:14.955672 12936 raft_consensus.cc:359] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.956446 12936 raft_consensus.cc:385] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 14:56:14.956661 12607 leader_election.cc:343] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:14.956725 12936 raft_consensus.cc:740] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.957073 12957 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.958051 12936 consensus_queue.cc:260] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.960984 12936 ts_tablet_manager.cc:1434] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 14:56:14.962260 12936 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Bootstrap starting.
I20260709 14:56:14.968973 12936 tablet_bootstrap.cc:654] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:14.970553 12957 raft_consensus.cc:515] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.974177 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:14.974457 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:14.975035 12715 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.975543 12936 tablet_bootstrap.cc:492] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: No bootstrap required, opened a new log
I20260709 14:56:14.976337 12936 ts_tablet_manager.cc:1403] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.008s
W20260709 14:56:14.976531 12607 leader_election.cc:343] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:14.977340 12957 leader_election.cc:290] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.980343 12936 raft_consensus.cc:359] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.981312 12936 raft_consensus.cc:385] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:14.981597 12936 raft_consensus.cc:740] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Initialized, Role: FOLLOWER
I20260709 14:56:14.981810 12957 raft_consensus.cc:493] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:14.982440 12957 raft_consensus.cc:515] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.982434 12936 consensus_queue.cc:260] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:14.984190 12715 raft_consensus.cc:2468] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:14.985214 12936 ts_tablet_manager.cc:1434] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 14:56:14.986045 12957 leader_election.cc:290] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:14.987005 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:14.985961 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:14.987632 12609 leader_election.cc:304] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:14.987855 12789 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.988119 12718 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 0.
I20260709 14:56:14.989535 12609 leader_election.cc:304] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:14.990267 12957 raft_consensus.cc:2804] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:14.990639 12958 raft_consensus.cc:2804] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:14.990994 12957 raft_consensus.cc:697] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:14.991176 12958 raft_consensus.cc:493] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:14.991607 12958 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:14.992115 12957 consensus_queue.cc:237] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.001185 12958 raft_consensus.cc:515] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.004304 12538 catalog_manager.cc:5697] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.005122 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.005916 12718 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.006901 12958 leader_election.cc:290] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.008914 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.009728 12789 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.015623 12718 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:15.017221 12609 leader_election.cc:304] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.018219 12958 raft_consensus.cc:2804] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:15.018877 12958 raft_consensus.cc:697] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.019189 12789 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 1.
I20260709 14:56:15.019853 12958 consensus_queue.cc:237] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.028229 12916 raft_consensus.cc:493] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:15.029047 12916 raft_consensus.cc:515] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.029760 12538 catalog_manager.cc:5697] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.031606 12916 leader_election.cc:290] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.033272 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:15.033593 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:15.034168 12640 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:15.034240 12718 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:15.036041 12758 leader_election.cc:304] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.037019 12916 raft_consensus.cc:2804] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:15.037473 12916 raft_consensus.cc:493] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:15.037837 12916 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.045164 12956 raft_consensus.cc:493] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:15.045974 12956 raft_consensus.cc:515] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.045962 12916 raft_consensus.cc:515] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.048792 12956 leader_election.cc:290] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.048997 12916 leader_election.cc:290] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.050499 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:15.050758 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.051566 12715 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.051620 12718 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:15.051972 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:15.052793 12640 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 0.
I20260709 14:56:15.054373 12758 leader_election.cc:304] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.054575 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.055469 12641 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.055497 12916 raft_consensus.cc:2804] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:15.056172 12916 raft_consensus.cc:493] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:15.056512 12916 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.061777 12715 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:15.063534 12758 leader_election.cc:304] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.064496 12641 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:15.064870 12956 raft_consensus.cc:2804] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:15.064620 12916 raft_consensus.cc:515] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.065531 12956 raft_consensus.cc:697] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.066944 12956 consensus_queue.cc:237] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.067492 12916 leader_election.cc:290] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.068360 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.069098 12641 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.070472 12867 raft_consensus.cc:493] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:15.071079 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.071195 12867 raft_consensus.cc:515] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.073817 12867 leader_election.cc:290] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.075153 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6b75345f024f5c9fd885daa98a6dd7" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" is_pre_election: true
I20260709 14:56:15.075542 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6b75345f024f5c9fd885daa98a6dd7" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:15.076072 12789 raft_consensus.cc:2468] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:15.076316 12640 raft_consensus.cc:2468] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 0.
I20260709 14:56:15.077759 12682 leader_election.cc:304] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.078778 12867 raft_consensus.cc:2804] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:15.078181 12715 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.079203 12641 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:15.079268 12867 raft_consensus.cc:493] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:15.080052 12867 raft_consensus.cc:3060] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.080061 12538 catalog_manager.cc:5697] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:15.083931 12758 leader_election.cc:304] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.084830 12956 raft_consensus.cc:2804] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:15.085342 12956 raft_consensus.cc:697] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.086171 12956 consensus_queue.cc:237] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.091909 12715 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 1.
I20260709 14:56:15.091864 12867 raft_consensus.cc:515] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.096063 12867 leader_election.cc:290] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.097015 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6b75345f024f5c9fd885daa98a6dd7" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.097319 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6b75345f024f5c9fd885daa98a6dd7" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.097877 12641 raft_consensus.cc:3060] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.098022 12789 raft_consensus.cc:3060] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.099807 12538 catalog_manager.cc:5697] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:15.105693 12641 raft_consensus.cc:2468] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:15.107226 12684 leader_election.cc:304] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.107312 12789 raft_consensus.cc:2468] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 1.
I20260709 14:56:15.108698 12867 raft_consensus.cc:2804] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:15.109483 12867 raft_consensus.cc:697] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:15.110414 12867 consensus_queue.cc:237] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.123137 12538 catalog_manager.cc:5697] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
W20260709 14:56:15.123925 12589 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:15.183403 12867 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:15.184181 12944 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:15.196509 12956 consensus_queue.cc:1048] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.233770 12715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:59704
I20260709 14:56:15.234509 12715 raft_consensus.cc:606] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:15.234670 12956 consensus_queue.cc:1048] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.243484 12867 raft_consensus.cc:993] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.244532 12867 raft_consensus.cc:1081] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.245970 12958 consensus_queue.cc:1048] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.247304 12641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43490
I20260709 14:56:15.248030 12641 raft_consensus.cc:493] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.248507 12641 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.253500 12791 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:36482
I20260709 14:56:15.254383 12791 raft_consensus.cc:606] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:15.259912 12641 raft_consensus.cc:515] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.262854 12641 leader_election.cc:290] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.263877 12718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.264830 12718 raft_consensus.cc:3055] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.265261 12718 raft_consensus.cc:740] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:15.268435 12791 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dc82df68e504347972d5622545eb022"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
mode: GRACEFUL
new_leader_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:36482
I20260709 14:56:15.268632 12790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.269402 12791 raft_consensus.cc:606] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Received request to transfer leadership to TS 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:15.269650 12790 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.270152 12956 raft_consensus.cc:993] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.266372 12718 consensus_queue.cc:260] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.271390 12956 raft_consensus.cc:1081] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.275028 12640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43492
I20260709 14:56:15.277063 12718 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.281462 12641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:43504
I20260709 14:56:15.282317 12641 raft_consensus.cc:606] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Received request to transfer leadership to TS 15534220b49c48e4a153b4563479d054
I20260709 14:56:15.275854 12640 raft_consensus.cc:493] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.287828 12790 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.287719 12957 consensus_queue.cc:1048] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.288348 12640 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.289737 12607 leader_election.cc:304] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.290782 12958 raft_consensus.cc:2804] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.291483 12958 raft_consensus.cc:697] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.292490 12958 consensus_queue.cc:237] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.290586 12957 consensus_queue.cc:1048] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.294512 12956 raft_consensus.cc:493] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:15.295205 12956 raft_consensus.cc:515] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.298812 12956 leader_election.cc:290] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.300164 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15534220b49c48e4a153b4563479d054" is_pre_election: true
I20260709 14:56:15.300274 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8307f11bd74ea49e419aac823055e4" is_pre_election: true
I20260709 14:56:15.301214 12717 raft_consensus.cc:2393] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in current term 1: Already voted for candidate 2e8307f11bd74ea49e419aac823055e4 in this term.
I20260709 14:56:15.302651 12641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:43504
I20260709 14:56:15.303449 12641 raft_consensus.cc:606] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Received request to transfer leadership to TS 15534220b49c48e4a153b4563479d054
I20260709 14:56:15.303519 12758 leader_election.cc:304] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3; no voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4
I20260709 14:56:15.306633 12718 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.313795 12640 raft_consensus.cc:515] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.316951 12956 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:15.318055 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.318892 12717 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.324024 12957 raft_consensus.cc:993] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.324783 12957 raft_consensus.cc:1081] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Signalling peer 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.325131 12540 catalog_manager.cc:5697] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.327365 12715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd"
dest_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:59686
I20260709 14:56:15.328183 12715 raft_consensus.cc:493] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.328827 12789 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
mode: GRACEFUL
new_leader_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:36482
I20260709 14:56:15.329725 12789 raft_consensus.cc:606] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Received request to transfer leadership to TS 15534220b49c48e4a153b4563479d054
I20260709 14:56:15.331871 12715 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.338152 12924 consensus_queue.cc:1048] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.342374 12641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:43504
I20260709 14:56:15.343194 12641 raft_consensus.cc:606] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Received request to transfer leadership to TS 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:15.344725 12717 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.346315 12640 leader_election.cc:290] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.345966 12715 raft_consensus.cc:515] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.348256 12791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d5a98a9bb846f88c319b3d57f58814" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.352017 12791 raft_consensus.cc:3055] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.347685 12609 leader_election.cc:304] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.352624 12791 raft_consensus.cc:740] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.353634 12791 consensus_queue.cc:260] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.354173 12958 raft_consensus.cc:2804] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.354982 12958 raft_consensus.cc:697] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.355985 12958 consensus_queue.cc:237] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.359556 12791 raft_consensus.cc:3060] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.346058 12956 raft_consensus.cc:2749] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:15.367714 12715 leader_election.cc:290] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.368413 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.369223 12640 raft_consensus.cc:3055] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.369553 12640 raft_consensus.cc:740] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.372910 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8862ef85c4304d71b788f5bd839a56fd" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.373730 12789 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.375129 12791 raft_consensus.cc:2468] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.370482 12640 consensus_queue.cc:260] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.378630 12640 raft_consensus.cc:3060] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.384850 12538 catalog_manager.cc:5697] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.386754 12789 raft_consensus.cc:2468] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.388283 12682 leader_election.cc:304] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.389236 12867 raft_consensus.cc:2804] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.389851 12640 raft_consensus.cc:2468] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.390129 12867 raft_consensus.cc:697] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:15.391433 12867 consensus_queue.cc:237] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.400046 12538 catalog_manager.cc:5697] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 1 to 2, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.427917 12958 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:15.441812 12924 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.451258 12957 raft_consensus.cc:993] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.451766 12924 raft_consensus.cc:1081] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Signalling peer 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.453549 12715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe"
dest_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:59686
I20260709 14:56:15.454118 12715 raft_consensus.cc:493] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.454456 12715 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.461194 12715 raft_consensus.cc:515] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.463838 12715 leader_election.cc:290] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.464323 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.464529 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.465134 12640 raft_consensus.cc:3055] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.465695 12640 raft_consensus.cc:740] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.465777 12789 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.466740 12640 consensus_queue.cc:260] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.468763 12640 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.475032 12789 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.476580 12682 leader_election.cc:304] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:15.477535 12867 raft_consensus.cc:2804] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.477965 12640 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.477998 12867 raft_consensus.cc:697] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:15.479204 12867 consensus_queue.cc:237] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.486922 12538 catalog_manager.cc:5697] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 1 to 2, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.491274 12956 consensus_queue.cc:1048] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.493072 12867 consensus_queue.cc:1048] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:15.509009 12916 consensus_queue.cc:1048] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.517341 12867 consensus_queue.cc:1048] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:15.534592 12956 raft_consensus.cc:993] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: : Instructing follower 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.536238 12956 raft_consensus.cc:1081] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Signalling peer 2e8307f11bd74ea49e419aac823055e4 to start an election
I20260709 14:56:15.538990 12641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dc82df68e504347972d5622545eb022"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
 from {username='slave'} at 127.0.0.1:43492
I20260709 14:56:15.539673 12641 raft_consensus.cc:493] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.540123 12641 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.547714 12641 raft_consensus.cc:515] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.550340 12641 leader_election.cc:290] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Requested vote from peers 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.550921 12715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.551043 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc82df68e504347972d5622545eb022" candidate_uuid: "2e8307f11bd74ea49e419aac823055e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.551882 12715 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.551994 12789 raft_consensus.cc:3055] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.552557 12789 raft_consensus.cc:740] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.553484 12789 consensus_queue.cc:260] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.555012 12789 raft_consensus.cc:3060] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.559851 12715 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.561604 12609 leader_election.cc:304] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.562592 12924 raft_consensus.cc:2804] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.563321 12924 raft_consensus.cc:697] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Leader. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.564292 12789 raft_consensus.cc:2468] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e8307f11bd74ea49e419aac823055e4 in term 2.
I20260709 14:56:15.564289 12924 consensus_queue.cc:237] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.572599 12539 catalog_manager.cc:5697] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 reported cstate change: term changed from 1 to 2, leader changed from 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195) to 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2e8307f11bd74ea49e419aac823055e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.574179 12956 consensus_queue.cc:1048] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.591150 12951 consensus_queue.cc:1048] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.592417 12998 consensus_queue.cc:1048] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.603648 12957 raft_consensus.cc:993] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:15.604204 12957 raft_consensus.cc:1081] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Signalling peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:15.608210 12791 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:36474
I20260709 14:56:15.608496 12951 raft_consensus.cc:993] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: : Instructing follower 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.608958 12791 raft_consensus.cc:493] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.609127 12951 raft_consensus.cc:1081] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Signalling peer 15534220b49c48e4a153b4563479d054 to start an election
I20260709 14:56:15.609449 12791 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.613582 12715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee"
dest_uuid: "15534220b49c48e4a153b4563479d054"
 from {username='slave'} at 127.0.0.1:59688
I20260709 14:56:15.614423 12715 raft_consensus.cc:493] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:15.614918 12715 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.616014 12958 consensus_queue.cc:1048] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:15.624629 12715 raft_consensus.cc:515] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.624893 12791 raft_consensus.cc:515] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.628278 12641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.629046 12641 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.629272 12640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:15.630097 12640 raft_consensus.cc:3055] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.630532 12640 raft_consensus.cc:740] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:15.631516 12640 consensus_queue.cc:260] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.632213 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e2aa545c75491b842dad6f579733e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:15.632896 12640 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.632931 12717 raft_consensus.cc:3060] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.641204 12789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3b23b4ade94168b7ae1c604de2f5ee" candidate_uuid: "15534220b49c48e4a153b4563479d054" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
I20260709 14:56:15.641932 12789 raft_consensus.cc:3055] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:15.642256 12789 raft_consensus.cc:740] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.630132 12791 leader_election.cc:290] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:15.643152 12789 consensus_queue.cc:260] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.644408 12789 raft_consensus.cc:3060] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:15.647025 12715 leader_election.cc:290] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195:45519)
I20260709 14:56:15.648461 12641 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.649856 12640 raft_consensus.cc:2468] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 2.
I20260709 14:56:15.650178 12684 leader_election.cc:304] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15534220b49c48e4a153b4563479d054, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.650573 12717 raft_consensus.cc:2468] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 2.
I20260709 14:56:15.651330 13002 raft_consensus.cc:2804] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.651350 12758 leader_election.cc:304] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:15.652148 13002 raft_consensus.cc:697] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Leader. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:15.652551 12956 raft_consensus.cc:2804] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:15.652966 12789 raft_consensus.cc:2468] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15534220b49c48e4a153b4563479d054 in term 2.
I20260709 14:56:15.653137 12956 raft_consensus.cc:697] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:15.653352 13002 consensus_queue.cc:237] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.654592 12956 consensus_queue.cc:237] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:15.663934 12540 catalog_manager.cc:5697] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 1 to 2, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:15.667641 12539 catalog_manager.cc:5697] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 reported cstate change: term changed from 1 to 2, leader changed from 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195) to 15534220b49c48e4a153b4563479d054 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "15534220b49c48e4a153b4563479d054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:15.802652 12789 raft_consensus.cc:1275] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.804214 12958 consensus_queue.cc:1048] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:15.817939 12717 raft_consensus.cc:1275] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.819794 12958 consensus_queue.cc:1048] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:15.837508 12789 raft_consensus.cc:1275] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.839855 12958 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:15.849000 12717 raft_consensus.cc:1275] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.851832 12958 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:15.912660 12789 raft_consensus.cc:1275] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.914422 13002 consensus_queue.cc:1048] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:15.921905 12791 raft_consensus.cc:1275] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.923770 12640 raft_consensus.cc:1275] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.923938 13002 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:15.925917 12944 consensus_queue.cc:1048] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:15.941079 12640 raft_consensus.cc:1275] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:15.943576 13002 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:16.169263 12717 raft_consensus.cc:1275] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.171221 12958 consensus_queue.cc:1048] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:16.178342 12789 raft_consensus.cc:1275] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e8307f11bd74ea49e419aac823055e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.185581 12957 consensus_queue.cc:1048] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:16.200919 12640 raft_consensus.cc:1275] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.203792 13009 consensus_queue.cc:1048] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:16.204241 12640 raft_consensus.cc:1275] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.206043 12867 consensus_queue.cc:1048] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:16.217173 12789 raft_consensus.cc:1275] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Refusing update from remote peer 15534220b49c48e4a153b4563479d054: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.218258 12717 raft_consensus.cc:1275] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:16.219148 12867 consensus_queue.cc:1048] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:16.220850 13009 consensus_queue.cc:1048] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:18.348105 12538 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33422:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 14:56:18.382077 12538 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33422:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 14:56:18.453112   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 14:56:18.767640 13049 log.cc:927] Time spent T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Append to log took a long time: real 0.090s	user 0.001s	sys 0.001s
W20260709 14:56:18.833739 13060 log.cc:927] Time spent T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 14:56:18.863591 13064 tablet.cc:1724] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:18.867393 13065 log.cc:927] Time spent T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 14:56:18.879837 13056 log.cc:927] Time spent T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 14:56:18.862353 13068 tablet.cc:1724] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:18.904553 13071 tablet.cc:1724] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:18.925022 13075 tablet.cc:1724] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:18.925521 13067 tablet.cc:1724] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:18.937851 13073 tablet.cc:1724] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:18.939136 12744 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:01d5a98a9bb846f88c319b3d57f58814)
I20260709 14:56:18.939005 13077 tablet.cc:1724] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:18.983263 12682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45519, user_credentials={real_user=slave}} blocked reactor thread for 49704.4us
W20260709 14:56:18.982529 13070 log.cc:927] Time spent T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 14:56:18.998010 12758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33531, user_credentials={real_user=slave}} blocked reactor thread for 56872.1us
I20260709 14:56:18.998580 13082 tablet.cc:1724] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.012203 12718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:59718
W20260709 14:56:19.016707 12744 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:01d5a98a9bb846f88c319b3d57f58814)
W20260709 14:56:19.022734 12744 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:01d5a98a9bb846f88c319b3d57f58814)
W20260709 14:56:19.029898 13072 log.cc:927] Time spent T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 14:56:19.032668 12718 raft_consensus.cc:606] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Received request to transfer leadership to TS 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:19.046461 13079 tablet.cc:1724] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.079221 13080 tablet.cc:1724] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.063000 13077 tablet.cc:1724] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.088804 13073 tablet.cc:1724] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.097245 13087 tablet.cc:1724] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:19.105513 13084 log.cc:927] Time spent T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 14:56:19.116243 13086 tablet.cc:1724] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.118211 13082 tablet.cc:1724] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.118777 13087 tablet.cc:1724] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.138401 13086 tablet.cc:1724] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.140130 13073 tablet.cc:1724] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.145040 13080 tablet.cc:1724] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.163146 13064 tablet.cc:1724] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.177415 13067 tablet.cc:1724] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.199100 13068 tablet.cc:1724] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.189441 13073 tablet.cc:1724] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.210095 13052 raft_consensus.cc:993] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: : Instructing follower 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.210907 13052 raft_consensus.cc:1081] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Signalling peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.211205 13086 tablet.cc:1724] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.213475 12785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:36458
I20260709 14:56:19.215080 13077 tablet.cc:1724] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.223421 12638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:43516
I20260709 14:56:19.220184 13080 tablet.cc:1724] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.235280 12638 raft_consensus.cc:606] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Received request to transfer leadership to TS 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:19.237284   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 14:56:19.238087   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:19.225528 12785 raft_consensus.cc:493] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:19.247901 12785 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.229524 13071 tablet.cc:1724] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:19.268469 12717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303"
dest_uuid: "15534220b49c48e4a153b4563479d054"
mode: GRACEFUL
new_leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:59718
I20260709 14:56:19.270164 12717 raft_consensus.cc:606] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Received request to transfer leadership to TS 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:19.282119 12638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5"
dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
mode: GRACEFUL
new_leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:43516
I20260709 14:56:19.283305 12638 raft_consensus.cc:606] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Received request to transfer leadership to TS 13aa2e233ebd4ec0b82f53f7cad0eeb3
I20260709 14:56:19.284277 12785 raft_consensus.cc:515] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.289026 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:19.290282 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfd814c6064479d8750d6aa26b24bfe" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:19.292150   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:19.292879 12717 raft_consensus.cc:3055] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:19.293956 12717 raft_consensus.cc:740] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:19.293047   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:19.296527 12717 consensus_queue.cc:260] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.298975 12717 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.290654 12638 raft_consensus.cc:3060] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.289832 12785 leader_election.cc:290] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:19.312996 12717 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.313117 12638 raft_consensus.cc:2468] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.315160 12758 leader_election.cc:304] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:19.317235 13083 raft_consensus.cc:2804] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:19.317865 13083 raft_consensus.cc:697] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:19.319028 13083 consensus_queue.cc:237] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.319446 12541 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: soft_deleted_table [id=556155079afd4f99aafaa569f8f84cee] alter complete (version 1)
I20260709 14:56:19.331656 12541 catalog_manager.cc:5697] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 2 to 3, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:19.540724 13045 raft_consensus.cc:993] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.541355 13046 raft_consensus.cc:1081] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Signalling peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.543237 12790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:36474
I20260709 14:56:19.543824 12790 raft_consensus.cc:493] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:19.544158 12790 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.552186 12790 raft_consensus.cc:515] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.554719 12790 leader_election.cc:290] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:19.556370 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:19.556416 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78ce6c1a4c44290bdd90177bde180e5" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:19.557617 12717 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.557623 12638 raft_consensus.cc:3055] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:19.558295 12638 raft_consensus.cc:740] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:19.559408 12638 consensus_queue.cc:260] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.561043 12638 raft_consensus.cc:3060] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.567749 12717 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.569566 12758 leader_election.cc:304] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:19.570567 13083 raft_consensus.cc:2804] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:19.570971 12638 raft_consensus.cc:2468] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.571149 13083 raft_consensus.cc:697] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:19.572461 13083 consensus_queue.cc:237] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.583014 12541 catalog_manager.cc:5697] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 2 to 3, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:19.583812 13014 raft_consensus.cc:993] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: : Instructing follower 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.584574 13052 raft_consensus.cc:1081] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Signalling peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.586620 12790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:36458
I20260709 14:56:19.587352 12790 raft_consensus.cc:493] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:19.587766 12790 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:19.596184 12790 raft_consensus.cc:515] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.598261 12790 leader_election.cc:290] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 2 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:19.599974 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:19.600801 12717 raft_consensus.cc:3055] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:19.599974 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d91efd7d2344d2e930909ffe9b79303" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:19.601434 12717 raft_consensus.cc:740] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15534220b49c48e4a153b4563479d054, State: Running, Role: LEADER
I20260709 14:56:19.601800 12638 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:19.602638 12717 consensus_queue.cc:260] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.604269 12717 raft_consensus.cc:3060] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:19.611250 12638 raft_consensus.cc:2468] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 2.
I20260709 14:56:19.612649 12758 leader_election.cc:304] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 2e8307f11bd74ea49e419aac823055e4; no voters: 
I20260709 14:56:19.613597 13083 raft_consensus.cc:2804] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:19.614017 12717 raft_consensus.cc:2468] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 2.
I20260709 14:56:19.614185 13083 raft_consensus.cc:697] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:19.615411 13083 consensus_queue.cc:237] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.624063 12541 catalog_manager.cc:5697] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 1 to 2, leader changed from 15534220b49c48e4a153b4563479d054 (127.0.107.194) to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:19.687168 13045 raft_consensus.cc:993] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: : Instructing follower 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.687791 13046 raft_consensus.cc:1081] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Signalling peer 13aa2e233ebd4ec0b82f53f7cad0eeb3 to start an election
I20260709 14:56:19.689440 12790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de"
dest_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3"
 from {username='slave'} at 127.0.0.1:36474
I20260709 14:56:19.690044 12790 raft_consensus.cc:493] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:19.690322 12790 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.696161 12790 raft_consensus.cc:515] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.697952 12790 leader_election.cc:290] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Requested vote from peers 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601), 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531)
I20260709 14:56:19.699054 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "2e8307f11bd74ea49e419aac823055e4"
I20260709 14:56:19.699364 12717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5594b24f1e4d0396677cbe2bb0f9de" candidate_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "15534220b49c48e4a153b4563479d054"
I20260709 14:56:19.699832 12638 raft_consensus.cc:3055] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:19.700098 12717 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.700310 12638 raft_consensus.cc:740] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2e8307f11bd74ea49e419aac823055e4, State: Running, Role: LEADER
I20260709 14:56:19.701315 12638 consensus_queue.cc:260] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.702534 12638 raft_consensus.cc:3060] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:19.707067 12717 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.708670 12758 leader_election.cc:304] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13aa2e233ebd4ec0b82f53f7cad0eeb3, 15534220b49c48e4a153b4563479d054; no voters: 
I20260709 14:56:19.709272 12638 raft_consensus.cc:2468] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13aa2e233ebd4ec0b82f53f7cad0eeb3 in term 3.
I20260709 14:56:19.709632 13083 raft_consensus.cc:2804] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:19.710255 13083 raft_consensus.cc:697] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Becoming Leader. State: Replica: 13aa2e233ebd4ec0b82f53f7cad0eeb3, State: Running, Role: LEADER
I20260709 14:56:19.711364 13083 consensus_queue.cc:237] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } }
I20260709 14:56:19.723654 12541 catalog_manager.cc:5697] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 reported cstate change: term changed from 2 to 3, leader changed from 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193) to 13aa2e233ebd4ec0b82f53f7cad0eeb3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13aa2e233ebd4ec0b82f53f7cad0eeb3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:19.770663 12638 raft_consensus.cc:1275] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 14:56:19.772311 13083 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:56:19.783404 12717 raft_consensus.cc:1275] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 14:56:19.789484 13076 consensus_queue.cc:1048] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 14:56:20.136828 12717 raft_consensus.cc:1275] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:20.138780 13095 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:56:20.142483 12643 raft_consensus.cc:1275] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:20.144918 13076 consensus_queue.cc:1048] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 14:56:20.153403 12638 raft_consensus.cc:1275] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:20.155439 13076 consensus_queue.cc:1048] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:20.181559 12718 raft_consensus.cc:1275] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:20.187556 12643 raft_consensus.cc:1275] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 14:56:20.189069 13095 consensus_queue.cc:1048] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:20.192338 13095 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8307f11bd74ea49e419aac823055e4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:56:20.211084 12718 raft_consensus.cc:1275] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Refusing update from remote peer 13aa2e233ebd4ec0b82f53f7cad0eeb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 14:56:20.213044 13101 consensus_queue.cc:1048] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15534220b49c48e4a153b4563479d054" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 14:56:20.379849   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 14:56:20.386751   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 14:56:20.387400   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:20.393182   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 14:56:20.393764   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:21.430712   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 14:56:21.440601 12684 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 14:56:21.448956 12684 consensus_peers.cc:597] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.467398   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:21.468467   431 tablet_replica.cc:333] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.469501   431 raft_consensus.cc:2243] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.470170   431 raft_consensus.cc:2272] T 22e2aa545c75491b842dad6f579733e5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.473642   431 tablet_replica.cc:333] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.474746   431 raft_consensus.cc:2243] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.476033   431 raft_consensus.cc:2272] T 9dc82df68e504347972d5622545eb022 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.479017   431 tablet_replica.cc:333] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.479784   431 raft_consensus.cc:2243] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.480391   431 raft_consensus.cc:2272] T 0dfd814c6064479d8750d6aa26b24bfe P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.483016   431 tablet_replica.cc:333] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.484234   431 raft_consensus.cc:2243] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.484943   431 raft_consensus.cc:2272] T 8862ef85c4304d71b788f5bd839a56fd P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.488248   431 tablet_replica.cc:333] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.489143   431 raft_consensus.cc:2243] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.489701   431 raft_consensus.cc:2272] T fd619b7389e14e05859cb4dc49118e0b P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.492933   431 tablet_replica.cc:333] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
W20260709 14:56:21.493042 12758 consensus_peers.cc:597] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.493867   431 raft_consensus.cc:2243] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.494465   431 raft_consensus.cc:2272] T cf5594b24f1e4d0396677cbe2bb0f9de P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.497447   431 tablet_replica.cc:333] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.498298   431 raft_consensus.cc:2243] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.499398   431 raft_consensus.cc:2272] T 01d5a98a9bb846f88c319b3d57f58814 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.502167   431 tablet_replica.cc:333] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.503015   431 raft_consensus.cc:2243] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.504101   431 raft_consensus.cc:2272] T a86407143c684a358b111eaada3f75d5 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.507212   431 tablet_replica.cc:333] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.508036   431 raft_consensus.cc:2243] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.508657   431 raft_consensus.cc:2272] T 3e6b75345f024f5c9fd885daa98a6dd7 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.511405   431 tablet_replica.cc:333] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.512249   431 raft_consensus.cc:2243] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.512871   431 raft_consensus.cc:2272] T e95f790959a342c5bfea1c293cf1a739 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.515698   431 tablet_replica.cc:333] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.516395   431 raft_consensus.cc:2243] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.517024   431 raft_consensus.cc:2272] T de7fdd7742cb4669b10b276d7e672f12 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.519896   431 tablet_replica.cc:333] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.520604   431 raft_consensus.cc:2243] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.521175   431 raft_consensus.cc:2272] T 5b3b23b4ade94168b7ae1c604de2f5ee P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.523815   431 tablet_replica.cc:333] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.524523   431 raft_consensus.cc:2243] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.525518   431 raft_consensus.cc:2272] T 6ec3eee1d9064a1d8c288f1ea6139109 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.528208   431 tablet_replica.cc:333] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.528898   431 raft_consensus.cc:2243] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.529505   431 raft_consensus.cc:2272] T 6d91efd7d2344d2e930909ffe9b79303 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.532074   431 tablet_replica.cc:333] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.532701   431 raft_consensus.cc:2243] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.533735   431 raft_consensus.cc:2272] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.536607   431 tablet_replica.cc:333] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.537312   431 raft_consensus.cc:2243] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.537890   431 raft_consensus.cc:2272] T 242b524686de4239a46c51ee07876660 P 2e8307f11bd74ea49e419aac823055e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.541234   431 tablet_replica.cc:333] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.542141   431 raft_consensus.cc:2243] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.543512   431 raft_consensus.cc:2272] T ea32fc7c4e7f41cca6beaa1694925a61 P 2e8307f11bd74ea49e419aac823055e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.547137   431 tablet_replica.cc:333] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4: stopping tablet replica
I20260709 14:56:21.547662   431 raft_consensus.cc:2243] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.548116   431 raft_consensus.cc:2272] T c78ce6c1a4c44290bdd90177bde180e5 P 2e8307f11bd74ea49e419aac823055e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.600095   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:56:21.613046 12684 consensus_peers.cc:597] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:21.614156 12684 consensus_peers.cc:597] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:21.631054 12758 consensus_peers.cc:597] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.650784   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 14:56:21.691613   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:21.692543   431 tablet_replica.cc:333] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.693641   431 raft_consensus.cc:2243] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.694276   431 raft_consensus.cc:2272] T 22e2aa545c75491b842dad6f579733e5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.696877   431 tablet_replica.cc:333] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.697796   431 raft_consensus.cc:2243] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.698356   431 raft_consensus.cc:2272] T 9dc82df68e504347972d5622545eb022 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.700927   431 tablet_replica.cc:333] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.701555   431 raft_consensus.cc:2243] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.702539   431 raft_consensus.cc:2272] T 8862ef85c4304d71b788f5bd839a56fd P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.705674   431 tablet_replica.cc:333] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.706555   431 raft_consensus.cc:2243] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.707206   431 raft_consensus.cc:2272] T 0dfd814c6064479d8750d6aa26b24bfe P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:21.708771 12758 consensus_peers.cc:597] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.709991   431 tablet_replica.cc:333] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.710821   431 raft_consensus.cc:2243] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.711879   431 raft_consensus.cc:2272] T fd619b7389e14e05859cb4dc49118e0b P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.714452   431 tablet_replica.cc:333] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.715164   431 raft_consensus.cc:2243] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.716112   431 raft_consensus.cc:2272] T 5b3b23b4ade94168b7ae1c604de2f5ee P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.718611   431 tablet_replica.cc:333] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.719331   431 raft_consensus.cc:2243] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.719862   431 raft_consensus.cc:2272] T 6d91efd7d2344d2e930909ffe9b79303 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.722393   431 tablet_replica.cc:333] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.723116   431 raft_consensus.cc:2243] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.723676   431 raft_consensus.cc:2272] T 6ec3eee1d9064a1d8c288f1ea6139109 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.726179   431 tablet_replica.cc:333] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.726905   431 raft_consensus.cc:2243] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.727414   431 raft_consensus.cc:2272] T 242b524686de4239a46c51ee07876660 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:21.730338 12758 consensus_peers.cc:597] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531): Couldn't send request to peer 15534220b49c48e4a153b4563479d054. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.731014   431 tablet_replica.cc:333] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.731978   431 raft_consensus.cc:2243] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.733078   431 raft_consensus.cc:2272] T 3e6b75345f024f5c9fd885daa98a6dd7 P 15534220b49c48e4a153b4563479d054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.735894   431 tablet_replica.cc:333] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.736668   431 raft_consensus.cc:2243] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.737593   431 raft_consensus.cc:2272] T de7fdd7742cb4669b10b276d7e672f12 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.740525   431 tablet_replica.cc:333] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.741264   431 raft_consensus.cc:2243] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.742274   431 raft_consensus.cc:2272] T e95f790959a342c5bfea1c293cf1a739 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.744910   431 tablet_replica.cc:333] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.745595   431 raft_consensus.cc:2243] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.746157   431 raft_consensus.cc:2272] T 01d5a98a9bb846f88c319b3d57f58814 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.748733   431 tablet_replica.cc:333] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.749424   431 raft_consensus.cc:2243] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.750017   431 raft_consensus.cc:2272] T a86407143c684a358b111eaada3f75d5 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.752652   431 tablet_replica.cc:333] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.753346   431 raft_consensus.cc:2243] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.753981   431 raft_consensus.cc:2272] T cf5594b24f1e4d0396677cbe2bb0f9de P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.756616   431 tablet_replica.cc:333] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.757256   431 raft_consensus.cc:2243] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.757767   431 raft_consensus.cc:2272] T ea32fc7c4e7f41cca6beaa1694925a61 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.760274   431 tablet_replica.cc:333] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.760865   431 raft_consensus.cc:2243] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.761392   431 raft_consensus.cc:2272] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 15534220b49c48e4a153b4563479d054 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.763913   431 tablet_replica.cc:333] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054: stopping tablet replica
I20260709 14:56:21.764473   431 raft_consensus.cc:2243] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.765002   431 raft_consensus.cc:2272] T c78ce6c1a4c44290bdd90177bde180e5 P 15534220b49c48e4a153b4563479d054 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:21.776855 12758 consensus_peers.cc:597] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.807934   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 14:56:21.809387 12758 consensus_peers.cc:597] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531): Couldn't send request to peer 15534220b49c48e4a153b4563479d054. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.853120   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 14:56:21.898506 12758 consensus_peers.cc:597] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 2e8307f11bd74ea49e419aac823055e4 (127.0.107.193:34601): Couldn't send request to peer 2e8307f11bd74ea49e419aac823055e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34601: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.919323   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:21.920276   431 tablet_replica.cc:333] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.921042   431 raft_consensus.cc:2243] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.922050   431 raft_consensus.cc:2272] T 0dfd814c6064479d8750d6aa26b24bfe P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.924665   431 tablet_replica.cc:333] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.925420   431 raft_consensus.cc:2243] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.927053   431 raft_consensus.cc:2272] T 22e2aa545c75491b842dad6f579733e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.929796   431 tablet_replica.cc:333] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.930608   431 raft_consensus.cc:2243] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.931247   431 raft_consensus.cc:2272] T 9dc82df68e504347972d5622545eb022 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.933647   431 tablet_replica.cc:333] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.934382   431 raft_consensus.cc:2243] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.935029   431 raft_consensus.cc:2272] T 8862ef85c4304d71b788f5bd839a56fd P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.937529   431 tablet_replica.cc:333] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.938216   431 raft_consensus.cc:2243] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.938941   431 raft_consensus.cc:2272] T fd619b7389e14e05859cb4dc49118e0b P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.942061   431 tablet_replica.cc:333] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.942727   431 raft_consensus.cc:2243] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.943315   431 raft_consensus.cc:2272] T 3e6b75345f024f5c9fd885daa98a6dd7 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.945621   431 tablet_replica.cc:333] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.946360   431 raft_consensus.cc:2243] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.947002   431 raft_consensus.cc:2272] T de7fdd7742cb4669b10b276d7e672f12 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.949514   431 tablet_replica.cc:333] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.950197   431 raft_consensus.cc:2243] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.950709   431 raft_consensus.cc:2272] T e95f790959a342c5bfea1c293cf1a739 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.953240   431 tablet_replica.cc:333] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.953900   431 raft_consensus.cc:2243] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.954470   431 raft_consensus.cc:2272] T 5b3b23b4ade94168b7ae1c604de2f5ee P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.956938   431 tablet_replica.cc:333] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.957543   431 raft_consensus.cc:2243] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.958043   431 raft_consensus.cc:2272] T 6ec3eee1d9064a1d8c288f1ea6139109 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.960345   431 tablet_replica.cc:333] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.961056   431 raft_consensus.cc:2243] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.962023   431 raft_consensus.cc:2272] T 6d91efd7d2344d2e930909ffe9b79303 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.965131   431 tablet_replica.cc:333] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.965842   431 raft_consensus.cc:2243] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.966420   431 raft_consensus.cc:2272] T eb41a14b8ed34f7d9fffa0a7cb555dc2 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:21.969028 12758 consensus_peers.cc:597] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 -> Peer 15534220b49c48e4a153b4563479d054 (127.0.107.194:33531): Couldn't send request to peer 15534220b49c48e4a153b4563479d054. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:21.969276   431 tablet_replica.cc:333] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.970127   431 raft_consensus.cc:2243] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.971215   431 raft_consensus.cc:2272] T c78ce6c1a4c44290bdd90177bde180e5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.974139   431 tablet_replica.cc:333] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.974947   431 raft_consensus.cc:2243] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.975534   431 raft_consensus.cc:2272] T a86407143c684a358b111eaada3f75d5 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.978329   431 tablet_replica.cc:333] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.979101   431 raft_consensus.cc:2243] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.979710   431 raft_consensus.cc:2272] T 01d5a98a9bb846f88c319b3d57f58814 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.982187   431 tablet_replica.cc:333] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.982920   431 raft_consensus.cc:2243] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.984004   431 raft_consensus.cc:2272] T cf5594b24f1e4d0396677cbe2bb0f9de P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.986716   431 tablet_replica.cc:333] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.987498   431 raft_consensus.cc:2243] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:21.988103   431 raft_consensus.cc:2272] T ea32fc7c4e7f41cca6beaa1694925a61 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:21.990638   431 tablet_replica.cc:333] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3: stopping tablet replica
I20260709 14:56:21.991283   431 raft_consensus.cc:2243] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:21.992437   431 raft_consensus.cc:2272] T 242b524686de4239a46c51ee07876660 P 13aa2e233ebd4ec0b82f53f7cad0eeb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:22.037753   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:56:22.080904   431 master.cc:562] Master@127.0.107.254:46621 shutting down...
I20260709 14:56:22.109920   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:22.110587   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:22.111013   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7547cd1ca894443f892fa6f941dda1d9: stopping tablet replica
I20260709 14:56:22.135144   431 master.cc:584] Master@127.0.107.254:46621 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13334 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 14:56:22.198264   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34179
I20260709 14:56:22.199682   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:22.206398 13109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:22.206458 13110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.208420   431 server_base.cc:1061] running on GCE node
W20260709 14:56:22.208166 13112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.214723   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:22.215093   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:22.215318   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608982215296 us; error 0 us; skew 500 ppm
I20260709 14:56:22.215976   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:22.218969   431 webserver.cc:533] Webserver started at http://127.0.107.254:36547/ using document root <none> and password file <none>
I20260709 14:56:22.219571   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:22.219794   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:22.220121   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:22.221405   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/master-0-root/instance:
uuid: "f97fae1c76d3474ca9e61e0940d3bedc"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.226511   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 14:56:22.230523 13117 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.231545   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 14:56:22.231949   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/master-0-root
uuid: "f97fae1c76d3474ca9e61e0940d3bedc"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.232293   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:22.254133   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:22.255844   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:22.302681   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34179
I20260709 14:56:22.302812 13168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34179 every 8 connection(s)
I20260709 14:56:22.307426 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:22.320885 13169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: Bootstrap starting.
I20260709 14:56:22.326879 13169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:22.332116 13169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: No bootstrap required, opened a new log
I20260709 14:56:22.334936 13169 raft_consensus.cc:359] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER }
I20260709 14:56:22.335427 13169 raft_consensus.cc:385] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:22.335693 13169 raft_consensus.cc:740] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97fae1c76d3474ca9e61e0940d3bedc, State: Initialized, Role: FOLLOWER
I20260709 14:56:22.336365 13169 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER }
I20260709 14:56:22.336879 13169 raft_consensus.cc:399] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 14:56:22.337113 13169 raft_consensus.cc:493] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 14:56:22.337423 13169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:22.343999 13169 raft_consensus.cc:515] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER }
I20260709 14:56:22.344695 13169 leader_election.cc:304] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97fae1c76d3474ca9e61e0940d3bedc; no voters: 
I20260709 14:56:22.346019 13169 leader_election.cc:290] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 14:56:22.346493 13172 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:22.348222 13172 raft_consensus.cc:697] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 1 LEADER]: Becoming Leader. State: Replica: f97fae1c76d3474ca9e61e0940d3bedc, State: Running, Role: LEADER
I20260709 14:56:22.349110 13172 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER }
I20260709 14:56:22.349817 13169 sys_catalog.cc:565] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 14:56:22.354532 13174 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97fae1c76d3474ca9e61e0940d3bedc. Latest consensus state: current_term: 1 leader_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER } }
I20260709 14:56:22.354945 13173 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97fae1c76d3474ca9e61e0940d3bedc" member_type: VOTER } }
I20260709 14:56:22.355567 13174 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:22.356150 13173 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [sys.catalog]: This master's current role is: LEADER
I20260709 14:56:22.359431 13178 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 14:56:22.368618 13178 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 14:56:22.370896   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 14:56:22.379814 13178 catalog_manager.cc:1383] Generated new cluster ID: d5314e2c95e74502997085be50311849
I20260709 14:56:22.380213 13178 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 14:56:22.430640 13178 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 14:56:22.432205 13178 catalog_manager.cc:1540] Loading token signing keys...
I20260709 14:56:22.452085 13178 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: Generated new TSK 0
I20260709 14:56:22.452829 13178 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 14:56:22.503217   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:22.510751 13190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:22.511631 13191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.514123   431 server_base.cc:1061] running on GCE node
W20260709 14:56:22.514158 13193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.515475   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:22.515744   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:22.515946   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608982515921 us; error 0 us; skew 500 ppm
I20260709 14:56:22.516520   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:22.519330   431 webserver.cc:533] Webserver started at http://127.0.107.193:44829/ using document root <none> and password file <none>
I20260709 14:56:22.519876   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:22.520098   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:22.520394   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:22.521689   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-0-root/instance:
uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.527195   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 14:56:22.531213 13198 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.532131   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 14:56:22.532510   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-0-root
uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.532861   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:22.552178   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:22.553696   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:22.555506   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:22.558174   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:22.558426   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.558682   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:22.558918   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 14:56:22.606453   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46565
I20260709 14:56:22.606572 13260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46565 every 8 connection(s)
I20260709 14:56:22.612725   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:22.621905 13265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:22.623109 13266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.626924   431 server_base.cc:1061] running on GCE node
W20260709 14:56:22.627090 13268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.629216   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:22.629557   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:22.629789   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608982629743 us; error 0 us; skew 500 ppm
I20260709 14:56:22.630455   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:22.631449 13261 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34179
I20260709 14:56:22.631917 13261 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:22.632876 13261 heartbeater.cc:507] Master 127.0.107.254:34179 requested a full tablet report, sending...
I20260709 14:56:22.633630   431 webserver.cc:533] Webserver started at http://127.0.107.194:46003/ using document root <none> and password file <none>
I20260709 14:56:22.634443   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:22.634668   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:22.635030   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:22.635951 13134 ts_manager.cc:194] Registered new tserver with Master: 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565)
I20260709 14:56:22.636924   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-1-root/instance:
uuid: "7e759d511dbf48d389a26747d452fbae"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.638795 13134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40744
I20260709 14:56:22.643528   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 14:56:22.647889 13273 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.648859   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 14:56:22.649268   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-1-root
uuid: "7e759d511dbf48d389a26747d452fbae"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.649588   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:22.667632   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:22.669384   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:22.671355   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:22.674431   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:22.674769   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.675100   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:22.675302   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.724828   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:37319
I20260709 14:56:22.724961 13335 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:37319 every 8 connection(s)
I20260709 14:56:22.730957   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 14:56:22.740676 13339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:22.743899 13340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 14:56:22.744452 13342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 14:56:22.744760   431 server_base.cc:1061] running on GCE node
I20260709 14:56:22.745489 13336 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34179
I20260709 14:56:22.745945 13336 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:22.746002   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 14:56:22.746351   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 14:56:22.746554   431 hybrid_clock.cc:648] HybridClock initialized: now 1783608982746539 us; error 0 us; skew 500 ppm
I20260709 14:56:22.746999 13336 heartbeater.cc:507] Master 127.0.107.254:34179 requested a full tablet report, sending...
I20260709 14:56:22.747346   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 14:56:22.749704 13134 ts_manager.cc:194] Registered new tserver with Master: 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:22.751587 13134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40760
I20260709 14:56:22.753084   431 webserver.cc:533] Webserver started at http://127.0.107.195:44999/ using document root <none> and password file <none>
I20260709 14:56:22.753837   431 fs_manager.cc:362] Metadata directory not provided
I20260709 14:56:22.754081   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 14:56:22.754392   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 14:56:22.755762   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-2-root/instance:
uuid: "65dabe7272bd41ec849103a082ddd59a"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.761104   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 14:56:22.765359 13347 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.766372   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 14:56:22.766913   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-2-root
uuid: "65dabe7272bd41ec849103a082ddd59a"
format_stamp: "Formatted at 2026-07-09 14:56:22 on dist-test-slave-wg8n"
I20260709 14:56:22.767275   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWTqNXQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783608598766170-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 14:56:22.778290   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 14:56:22.779842   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 14:56:22.781590   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 14:56:22.784121   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 14:56:22.784394   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.784663   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 14:56:22.784857   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 14:56:22.833021   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:39583
I20260709 14:56:22.833140 13409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:39583 every 8 connection(s)
I20260709 14:56:22.849906 13410 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34179
I20260709 14:56:22.850520 13410 heartbeater.cc:461] Registering TS with master...
I20260709 14:56:22.851542 13410 heartbeater.cc:507] Master 127.0.107.254:34179 requested a full tablet report, sending...
I20260709 14:56:22.853963 13134 ts_manager.cc:194] Registered new tserver with Master: 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:22.854768   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017712441s
I20260709 14:56:22.855690 13134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40764
I20260709 14:56:22.888013 13134 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40778:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:22.891206 13134 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:22.988617 13373 tablet_service.cc:1511] Processing CreateTablet for tablet 4c7a9981ec004884ab14f39794183aba (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:22.989023 13372 tablet_service.cc:1511] Processing CreateTablet for tablet bee8ad3334284671a1e3a35ed587356a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:22.989499 13374 tablet_service.cc:1511] Processing CreateTablet for tablet 19cf6020ce324c1ab6a5ff208b019565 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:22.990751 13372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee8ad3334284671a1e3a35ed587356a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:22.996470 13373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7a9981ec004884ab14f39794183aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:22.996178 13299 tablet_service.cc:1511] Processing CreateTablet for tablet 4c7a9981ec004884ab14f39794183aba (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:22.997834 13299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7a9981ec004884ab14f39794183aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.000183 13298 tablet_service.cc:1511] Processing CreateTablet for tablet bee8ad3334284671a1e3a35ed587356a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:23.002045 13295 tablet_service.cc:1511] Processing CreateTablet for tablet a12707f8815f4068b98ef47558aab544 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:23.002987 13297 tablet_service.cc:1511] Processing CreateTablet for tablet be6baf7543824ff5a503aa00c55adc45 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:23.004891 13226 tablet_service.cc:1511] Processing CreateTablet for tablet 1e69b1ce56f94e03b714a57aa6e0d983 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:23.005554 13296 tablet_service.cc:1511] Processing CreateTablet for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:23.006539 13226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e69b1ce56f94e03b714a57aa6e0d983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.008097 13222 tablet_service.cc:1511] Processing CreateTablet for tablet be6baf7543824ff5a503aa00c55adc45 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:23.009382 13224 tablet_service.cc:1511] Processing CreateTablet for tablet 4c7a9981ec004884ab14f39794183aba (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:23.010563 13221 tablet_service.cc:1511] Processing CreateTablet for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:23.011863 13223 tablet_service.cc:1511] Processing CreateTablet for tablet bee8ad3334284671a1e3a35ed587356a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:23.012974 13220 tablet_service.cc:1511] Processing CreateTablet for tablet a12707f8815f4068b98ef47558aab544 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:23.014426 13220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12707f8815f4068b98ef47558aab544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.015120 13224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7a9981ec004884ab14f39794183aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.016157 13375 tablet_service.cc:1511] Processing CreateTablet for tablet 1e69b1ce56f94e03b714a57aa6e0d983 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:23.017634 13375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e69b1ce56f94e03b714a57aa6e0d983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.019225 13223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee8ad3334284671a1e3a35ed587356a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.008118 13225 tablet_service.cc:1511] Processing CreateTablet for tablet 19cf6020ce324c1ab6a5ff208b019565 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:23.020962 13225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19cf6020ce324c1ab6a5ff208b019565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:22.991983 13301 tablet_service.cc:1511] Processing CreateTablet for tablet 1e69b1ce56f94e03b714a57aa6e0d983 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:22.994120 13300 tablet_service.cc:1511] Processing CreateTablet for tablet 19cf6020ce324c1ab6a5ff208b019565 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:22.990772 13371 tablet_service.cc:1511] Processing CreateTablet for tablet be6baf7543824ff5a503aa00c55adc45 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:23.024389 13371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6baf7543824ff5a503aa00c55adc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.025803 13221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.026645 13374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19cf6020ce324c1ab6a5ff208b019565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.028090 13370 tablet_service.cc:1511] Processing CreateTablet for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:23.030386 13370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.028121 13222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6baf7543824ff5a503aa00c55adc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.039307 13298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee8ad3334284671a1e3a35ed587356a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.043504 13297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6baf7543824ff5a503aa00c55adc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.047323 13296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ff4b0d5d0544e378da6c1d1b5f1f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.046525 13425 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.052609 13301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e69b1ce56f94e03b714a57aa6e0d983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.055476 13425 tablet_bootstrap.cc:654] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.056345 13300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19cf6020ce324c1ab6a5ff208b019565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.064167 13295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12707f8815f4068b98ef47558aab544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.083091 13373 tablet_service.cc:1511] Processing CreateTablet for tablet a12707f8815f4068b98ef47558aab544 (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:23.084913 13373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12707f8815f4068b98ef47558aab544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.088397 13219 tablet_service.cc:1511] Processing CreateTablet for tablet 60b48dbc27304cf29011afb100eb77bb (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:23.090134 13219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b48dbc27304cf29011afb100eb77bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.110754 13294 tablet_service.cc:1511] Processing CreateTablet for tablet 60b48dbc27304cf29011afb100eb77bb (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:23.112625 13294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b48dbc27304cf29011afb100eb77bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.166090 13425 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.166810 13425 ts_tablet_manager.cc:1403] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.121s	user 0.012s	sys 0.007s
I20260709 14:56:23.171202 13427 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.178963 13427 tablet_bootstrap.cc:654] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.191529 13425 raft_consensus.cc:359] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.192956 13425 raft_consensus.cc:385] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.193409 13425 raft_consensus.cc:740] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.203733 13428 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.207078 13219 tablet_service.cc:1511] Processing CreateTablet for tablet 05bf66306ab9413d9bba98818c0b6b5a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:23.209549 13296 tablet_service.cc:1511] Processing CreateTablet for tablet 05bf66306ab9413d9bba98818c0b6b5a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:23.203140 13373 tablet_service.cc:1511] Processing CreateTablet for tablet 60b48dbc27304cf29011afb100eb77bb (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:23.211700 13373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b48dbc27304cf29011afb100eb77bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.216667 13296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05bf66306ab9413d9bba98818c0b6b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.222195 13370 tablet_service.cc:1511] Processing CreateTablet for tablet 05bf66306ab9413d9bba98818c0b6b5a (DEFAULT_TABLE table=test-workload [id=12ae1bd0a3564a3ebbac7d3c4ac2e5c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:23.224326 13219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05bf66306ab9413d9bba98818c0b6b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.224794 13370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05bf66306ab9413d9bba98818c0b6b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:23.215269 13425 consensus_queue.cc:260] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.232264 13428 tablet_bootstrap.cc:654] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.237643 13427 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.238550 13427 ts_tablet_manager.cc:1403] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.068s	user 0.025s	sys 0.021s
I20260709 14:56:23.244012 13427 raft_consensus.cc:359] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.245265 13427 raft_consensus.cc:385] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.245685 13427 raft_consensus.cc:740] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.247205 13427 consensus_queue.cc:260] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.252923 13410 heartbeater.cc:499] Master 127.0.107.254:34179 was elected leader, sending a full tablet report...
I20260709 14:56:23.266180 13430 raft_consensus.cc:493] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.267117 13430 raft_consensus.cc:515] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.287189 13261 heartbeater.cc:499] Master 127.0.107.254:34179 was elected leader, sending a full tablet report...
I20260709 14:56:23.288511 13427 ts_tablet_manager.cc:1434] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.049s	user 0.023s	sys 0.007s
I20260709 14:56:23.290016 13427 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.294001 13425 ts_tablet_manager.cc:1434] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.127s	user 0.025s	sys 0.019s
I20260709 14:56:23.295727 13425 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.298645 13427 tablet_bootstrap.cc:654] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.310220 13425 tablet_bootstrap.cc:654] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.312410 13427 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.313246 13427 ts_tablet_manager.cc:1403] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 14:56:23.318341 13427 raft_consensus.cc:359] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.319952 13427 raft_consensus.cc:385] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.320392 13427 raft_consensus.cc:740] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.321619 13427 consensus_queue.cc:260] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.323810 13425 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.324615 13425 ts_tablet_manager.cc:1403] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.011s
I20260709 14:56:23.324728 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:23.324628 13427 ts_tablet_manager.cc:1434] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.007s
W20260709 14:56:23.327214 13351 leader_election.cc:343] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.327994 13427 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.328732 13425 raft_consensus.cc:359] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.329645 13425 raft_consensus.cc:385] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.329991 13425 raft_consensus.cc:740] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.331380 13425 consensus_queue.cc:260] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
W20260709 14:56:23.338335 13411 tablet.cc:2406] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 14:56:23.339551 13427 tablet_bootstrap.cc:654] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.344843 13432 raft_consensus.cc:493] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.345860 13432 raft_consensus.cc:515] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.355820 13430 leader_election.cc:290] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
W20260709 14:56:23.365681 13182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:23.371344 13432 leader_election.cc:290] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.371742 13428 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.372566 13428 ts_tablet_manager.cc:1403] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.169s	user 0.060s	sys 0.064s
I20260709 14:56:23.376317 13428 raft_consensus.cc:359] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.378002 13428 raft_consensus.cc:385] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.378460 13428 raft_consensus.cc:740] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.379515 13428 consensus_queue.cc:260] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.384148 13427 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.385293 13427 ts_tablet_manager.cc:1403] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.002s
I20260709 14:56:23.389825 13427 raft_consensus.cc:359] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.395474 13427 raft_consensus.cc:385] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.396580 13427 raft_consensus.cc:740] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.400903 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
W20260709 14:56:23.402788 13351 leader_election.cc:343] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.403353 13351 leader_election.cc:304] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.398003 13427 consensus_queue.cc:260] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.404441 13336 heartbeater.cc:499] Master 127.0.107.254:34179 was elected leader, sending a full tablet report...
I20260709 14:56:23.404996 13430 raft_consensus.cc:2749] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:23.415165 13428 ts_tablet_manager.cc:1434] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.008s
I20260709 14:56:23.416725 13428 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.417414 13427 ts_tablet_manager.cc:1434] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 14:56:23.421422 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.424667 13428 tablet_bootstrap.cc:654] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
W20260709 14:56:23.430583 13202 leader_election.cc:343] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.433863 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
W20260709 14:56:23.437026 13200 leader_election.cc:343] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.437856 13200 leader_election.cc:304] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.439296 13432 raft_consensus.cc:2749] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:23.441622 13427 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.452847 13425 ts_tablet_manager.cc:1434] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.128s	user 0.058s	sys 0.058s
I20260709 14:56:23.454434 13425 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.458017 13427 tablet_bootstrap.cc:654] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.461431 13428 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.462244 13428 ts_tablet_manager.cc:1403] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.014s
I20260709 14:56:23.462436 13425 tablet_bootstrap.cc:654] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.466529 13428 raft_consensus.cc:359] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.467530 13428 raft_consensus.cc:385] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.467844 13428 raft_consensus.cc:740] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.468678 13428 consensus_queue.cc:260] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.469756 13427 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.470679 13427 ts_tablet_manager.cc:1403] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.008s
I20260709 14:56:23.471570 13428 ts_tablet_manager.cc:1434] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:56:23.473445 13428 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.475807 13427 raft_consensus.cc:359] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.477485 13427 raft_consensus.cc:385] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.477795 13427 raft_consensus.cc:740] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.478395 13432 raft_consensus.cc:493] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.479030 13432 raft_consensus.cc:515] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.479061 13427 consensus_queue.cc:260] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.482613 13428 tablet_bootstrap.cc:654] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.483472 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
W20260709 14:56:23.486385 13202 leader_election.cc:343] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.490943 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:23.492110 13385 raft_consensus.cc:2468] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:23.493230 13425 tablet_bootstrap.cc:492] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.494283 13425 ts_tablet_manager.cc:1403] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.019s
I20260709 14:56:23.494516 13200 leader_election.cc:304] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.496307 13432 leader_election.cc:290] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.499369 13425 raft_consensus.cc:359] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.500717 13425 raft_consensus.cc:385] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.501119 13432 raft_consensus.cc:2804] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.501204 13425 raft_consensus.cc:740] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.503232 13432 raft_consensus.cc:493] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.503876 13425 consensus_queue.cc:260] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.504412 13432 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.506642 13425 ts_tablet_manager.cc:1434] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 14:56:23.507936 13425 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.516209 13425 tablet_bootstrap.cc:654] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.529718 13425 tablet_bootstrap.cc:492] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.529906 13427 ts_tablet_manager.cc:1434] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.059s	user 0.019s	sys 0.037s
I20260709 14:56:23.530553 13425 ts_tablet_manager.cc:1403] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.003s
I20260709 14:56:23.531701 13428 tablet_bootstrap.cc:492] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.532459 13428 ts_tablet_manager.cc:1403] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.014s
I20260709 14:56:23.532008 13432 raft_consensus.cc:515] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.535270 13427 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.534785 13425 raft_consensus.cc:359] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.535678 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:23.535892 13425 raft_consensus.cc:385] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.536373 13425 raft_consensus.cc:740] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.537513 13425 consensus_queue.cc:260] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.540870 13428 raft_consensus.cc:359] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.538446 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:23.542500 13385 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.543810 13425 ts_tablet_manager.cc:1434] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 14:56:23.543862 13427 tablet_bootstrap.cc:654] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.545377 13425 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.551033 13432 leader_election.cc:290] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.554011 13425 tablet_bootstrap.cc:654] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.541937 13428 raft_consensus.cc:385] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.556710 13428 raft_consensus.cc:740] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.557380 13385 raft_consensus.cc:2468] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:23.558107 13428 consensus_queue.cc:260] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.559129 13200 leader_election.cc:304] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:23.559927 13311 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.560246 13432 raft_consensus.cc:2804] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.561988 13432 raft_consensus.cc:697] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:23.563629 13432 consensus_queue.cc:237] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.567785 13425 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.569000 13425 ts_tablet_manager.cc:1403] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 14:56:23.572224 13311 raft_consensus.cc:2468] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:23.572942 13425 raft_consensus.cc:359] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.573964 13425 raft_consensus.cc:385] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.574368 13425 raft_consensus.cc:740] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.574801 13427 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.575624 13427 ts_tablet_manager.cc:1403] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.012s
I20260709 14:56:23.575620 13425 consensus_queue.cc:260] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.579337 13425 ts_tablet_manager.cc:1434] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.005s
I20260709 14:56:23.579674 13427 raft_consensus.cc:359] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.580826 13425 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.580595 13427 raft_consensus.cc:385] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.583256 13427 raft_consensus.cc:740] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.583364 13133 catalog_manager.cc:5697] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:23.584378 13427 consensus_queue.cc:260] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.587512 13427 ts_tablet_manager.cc:1434] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 14:56:23.589202 13427 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.594288 13425 tablet_bootstrap.cc:654] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.599083 13428 ts_tablet_manager.cc:1434] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.004s
I20260709 14:56:23.600728 13428 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.599112 13427 tablet_bootstrap.cc:654] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.608666 13425 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.609081 13428 tablet_bootstrap.cc:654] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.609368 13425 ts_tablet_manager.cc:1403] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.005s
I20260709 14:56:23.611104 13432 raft_consensus.cc:493] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.611941 13432 raft_consensus.cc:515] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.613813 13425 raft_consensus.cc:359] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.615404 13425 raft_consensus.cc:385] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.615862 13425 raft_consensus.cc:740] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.616755 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
W20260709 14:56:23.619077 13200 leader_election.cc:343] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.621441 13432 leader_election.cc:290] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.620813 13425 consensus_queue.cc:260] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.621915 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
W20260709 14:56:23.623911 13202 leader_election.cc:343] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.624437 13202 leader_election.cc:304] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.625249 13427 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.625905 13432 raft_consensus.cc:2749] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:23.625957 13427 ts_tablet_manager.cc:1403] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.012s
I20260709 14:56:23.631084 13425 ts_tablet_manager.cc:1434] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 14:56:23.632469 13427 raft_consensus.cc:359] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.633682 13427 raft_consensus.cc:385] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.634569 13425 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.635347 13428 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.636145 13428 ts_tablet_manager.cc:1403] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.012s
I20260709 14:56:23.635438 13427 raft_consensus.cc:740] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.640300 13427 consensus_queue.cc:260] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.643283 13427 ts_tablet_manager.cc:1434] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 14:56:23.644618 13427 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.644796 13425 tablet_bootstrap.cc:654] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.647733 13428 raft_consensus.cc:359] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.648874 13428 raft_consensus.cc:385] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.649313 13428 raft_consensus.cc:740] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.650384 13428 consensus_queue.cc:260] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.653586 13428 ts_tablet_manager.cc:1434] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 14:56:23.654213 13425 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.654929 13425 ts_tablet_manager.cc:1403] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.001s
I20260709 14:56:23.658746 13427 tablet_bootstrap.cc:654] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.658623 13425 raft_consensus.cc:359] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.659603 13425 raft_consensus.cc:385] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.659950 13425 raft_consensus.cc:740] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.661162 13425 consensus_queue.cc:260] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.654929 13428 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.666077 13435 raft_consensus.cc:493] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.666708 13435 raft_consensus.cc:515] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.667771 13427 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.668449 13427 ts_tablet_manager.cc:1403] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.011s
I20260709 14:56:23.669525 13432 raft_consensus.cc:493] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.670152 13432 raft_consensus.cc:515] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.672026 13427 raft_consensus.cc:359] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.672492 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:23.673007 13427 raft_consensus.cc:385] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.673262 13236 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:23.673398 13427 raft_consensus.cc:740] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.673893 13428 tablet_bootstrap.cc:654] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.673698 13432 leader_election.cc:290] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.669586 13435 leader_election.cc:290] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565)
I20260709 14:56:23.674561 13427 consensus_queue.cc:260] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.670521 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.676110 13310 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:23.677361 13425 ts_tablet_manager.cc:1434] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 14:56:23.678277 13351 leader_election.cc:304] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:23.680852 13435 raft_consensus.cc:2804] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.680974 13427 ts_tablet_manager.cc:1434] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 14:56:23.681447 13435 raft_consensus.cc:493] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.681867 13435 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.682564 13427 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.683389 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.685595 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:23.686889 13385 raft_consensus.cc:2468] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:23.688406 13200 leader_election.cc:304] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:23.689468 13432 raft_consensus.cc:2804] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.689880 13432 raft_consensus.cc:493] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.690204 13432 raft_consensus.cc:3060] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.691167 13427 tablet_bootstrap.cc:654] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.696269 13440 raft_consensus.cc:493] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 14:56:23.696986 13202 leader_election.cc:343] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.696986 13440 raft_consensus.cc:515] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.707892 13425 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.708959 13440 leader_election.cc:290] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.707957 13435 raft_consensus.cc:515] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.714039 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:23.714883 13311 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.718349 13425 tablet_bootstrap.cc:654] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.725737 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:23.726898 13435 leader_election.cc:290] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565)
I20260709 14:56:23.726943 13236 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.742031 13435 raft_consensus.cc:493] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.749469 13435 raft_consensus.cc:515] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.750367 13432 raft_consensus.cc:515] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.754019 13435 leader_election.cc:290] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:23.755509 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.755359 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:23.756016 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:23.756564 13310 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:23.756625 13235 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:23.758081 13351 leader_election.cc:304] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
W20260709 14:56:23.758262 13202 leader_election.cc:343] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.758258 13444 raft_consensus.cc:493] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.759393 13444 raft_consensus.cc:515] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.759747 13435 raft_consensus.cc:2804] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.760356 13435 raft_consensus.cc:493] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.760774 13435 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.761907 13444 leader_election.cc:290] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.763862 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.764379 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:23.765885 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:23.766324 13385 raft_consensus.cc:3060] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.766686 13384 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:23.767009 13428 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.768117 13428 ts_tablet_manager.cc:1403] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.114s	user 0.036s	sys 0.037s
I20260709 14:56:23.768458 13200 leader_election.cc:304] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
W20260709 14:56:23.769547 13202 leader_election.cc:343] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.772696 13444 raft_consensus.cc:2804] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.772425 13428 raft_consensus.cc:359] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.773274 13444 raft_consensus.cc:493] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.773550 13428 raft_consensus.cc:385] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.773731 13444 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.774075 13428 raft_consensus.cc:740] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.774497 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:23.775177 13428 consensus_queue.cc:260] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
W20260709 14:56:23.776333 13275 leader_election.cc:343] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.777161 13385 raft_consensus.cc:2468] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:23.778041 13428 ts_tablet_manager.cc:1434] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:56:23.778502 13200 leader_election.cc:304] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.779752 13432 leader_election.cc:290] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.779492 13446 raft_consensus.cc:2804] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.782469 13446 raft_consensus.cc:697] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:23.783614 13444 raft_consensus.cc:515] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.783964 13236 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:23.783751 13446 consensus_queue.cc:237] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.785511 13351 leader_election.cc:304] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:23.786005 13444 leader_election.cc:290] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.786413 13430 raft_consensus.cc:2804] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.789058 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:23.789736 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:23.790601 13235 raft_consensus.cc:2468] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
W20260709 14:56:23.790984 13202 leader_election.cc:343] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.782648 13425 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.791786 13425 ts_tablet_manager.cc:1403] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.012s
I20260709 14:56:23.792460 13277 leader_election.cc:304] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:23.779481 13428 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.794111 13440 raft_consensus.cc:2804] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.794593 13440 raft_consensus.cc:493] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.794982 13440 raft_consensus.cc:3060] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.789269 13311 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:23.795336 13425 raft_consensus.cc:359] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.798097 13425 raft_consensus.cc:385] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.798544 13425 raft_consensus.cc:740] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.799620 13425 consensus_queue.cc:260] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.767045 13427 tablet_bootstrap.cc:492] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.802228 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:23.801551 13134 catalog_manager.cc:5697] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:23.803035 13385 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.801630 13427 ts_tablet_manager.cc:1403] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.018s
I20260709 14:56:23.815407 13428 tablet_bootstrap.cc:654] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.809962 13435 raft_consensus.cc:515] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.809628 13440 raft_consensus.cc:515] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.817811 13435 leader_election.cc:290] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:23.820006 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:23.820006 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:23.820851 13236 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.821717 13311 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.821640 13440 leader_election.cc:290] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.822906 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:23.820803 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:23.823782 13235 raft_consensus.cc:3060] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:56:23.824796 13275 leader_election.cc:343] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.825758 13430 raft_consensus.cc:697] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:23.827298 13430 consensus_queue.cc:237] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:23.831306 13236 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:23.818523 13427 raft_consensus.cc:359] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.833699 13351 leader_election.cc:304] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:23.834118 13427 raft_consensus.cc:385] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.835386 13464 raft_consensus.cc:2804] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.836048 13427 raft_consensus.cc:740] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.836408 13464 raft_consensus.cc:697] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:23.837849 13464 consensus_queue.cc:237] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.849306 13428 tablet_bootstrap.cc:492] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:23.850152 13428 ts_tablet_manager.cc:1403] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.007s
I20260709 14:56:23.850806 13385 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:23.857394 13200 leader_election.cc:304] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.858218 13432 raft_consensus.cc:2804] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.858740 13432 raft_consensus.cc:697] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:23.859822 13432 consensus_queue.cc:237] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.859390 13134 catalog_manager.cc:5697] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:23.853631 13425 ts_tablet_manager.cc:1434] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.004s
I20260709 14:56:23.866429 13425 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:23.871124 13132 catalog_manager.cc:5697] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:23.855034 13428 raft_consensus.cc:359] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.877454 13428 raft_consensus.cc:385] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.856009 13427 consensus_queue.cc:260] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.889802 13427 ts_tablet_manager.cc:1434] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.004s
I20260709 14:56:23.884222 13428 raft_consensus.cc:740] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.891116 13427 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:23.890964 13428 consensus_queue.cc:260] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.892519 13311 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:23.893579 13428 ts_tablet_manager.cc:1434] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 14:56:23.883174 13235 raft_consensus.cc:2468] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:23.898046 13277 leader_election.cc:304] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:23.898818 13427 tablet_bootstrap.cc:654] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.899763 13428 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:23.903292 13440 raft_consensus.cc:2804] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:23.889886 13425 tablet_bootstrap.cc:654] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.904237 13440 raft_consensus.cc:697] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:23.905637 13440 consensus_queue.cc:237] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.917321 13464 raft_consensus.cc:493] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.918453 13464 raft_consensus.cc:515] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.924985 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
W20260709 14:56:23.926640 13351 leader_election.cc:343] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.929694 13428 tablet_bootstrap.cc:654] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:23.935513 13464 leader_election.cc:290] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:23.935645 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:23.936304 13472 raft_consensus.cc:493] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:23.936898 13472 raft_consensus.cc:515] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
W20260709 14:56:23.937626 13351 leader_election.cc:343] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:23.938242 13351 leader_election.cc:304] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:23.939640 13464 raft_consensus.cc:2749] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:23.940227 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ff4b0d5d0544e378da6c1d1b5f1f4ec" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:23.941004 13235 raft_consensus.cc:2468] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:23.941249 13427 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:23.942569 13277 leader_election.cc:304] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:23.943112 13427 ts_tablet_manager.cc:1403] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.052s	user 0.030s	sys 0.007s
I20260709 14:56:23.944207 13134 catalog_manager.cc:5697] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:23.945924 13475 raft_consensus.cc:2804] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:23.947240 13475 raft_consensus.cc:493] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:23.948378 13475 raft_consensus.cc:3060] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.957455 13425 tablet_bootstrap.cc:492] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:23.946305 13427 raft_consensus.cc:359] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.959892 13425 ts_tablet_manager.cc:1403] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.007s
I20260709 14:56:23.959767 13432 consensus_queue.cc:1048] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:23.960321 13427 raft_consensus.cc:385] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.960762 13427 raft_consensus.cc:740] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.963552 13425 raft_consensus.cc:359] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.964766 13425 raft_consensus.cc:385] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:23.965158 13425 raft_consensus.cc:740] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:23.965260 13427 consensus_queue.cc:260] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.967988 13472 leader_election.cc:290] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.969673 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ff4b0d5d0544e378da6c1d1b5f1f4ec" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:23.970495 13385 raft_consensus.cc:2468] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:23.970970 13427 ts_tablet_manager.cc:1434] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 14:56:23.974278 13475 raft_consensus.cc:515] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:23.977393 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ff4b0d5d0544e378da6c1d1b5f1f4ec" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:23.977540 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ff4b0d5d0544e378da6c1d1b5f1f4ec" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:23.978366 13235 raft_consensus.cc:3060] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.978513 13385 raft_consensus.cc:3060] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:23.981916 13475 leader_election.cc:290] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:23.973991 13133 catalog_manager.cc:5697] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:23.995659 13385 raft_consensus.cc:2468] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:23.999672 13275 leader_election.cc:304] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:24.002784 13235 raft_consensus.cc:2468] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:24.003891 13475 raft_consensus.cc:2804] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:24.004735 13475 raft_consensus.cc:697] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:24.007072 13472 raft_consensus.cc:493] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:24.007899 13472 raft_consensus.cc:515] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.006112 13475 consensus_queue.cc:237] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.012622 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:24.019541 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19cf6020ce324c1ab6a5ff208b019565" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:24.020864 13385 raft_consensus.cc:2393] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e759d511dbf48d389a26747d452fbae in current term 1: Already voted for candidate 7c7aef1187334e5bb60453d1d4c07ebe in this term.
I20260709 14:56:24.021214 13472 leader_election.cc:290] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:24.022704 13275 leader_election.cc:304] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:24.022871 13425 consensus_queue.cc:260] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.024807 13428 tablet_bootstrap.cc:492] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:24.025446 13425 ts_tablet_manager.cc:1434] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 14:56:24.025645 13428 ts_tablet_manager.cc:1403] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.126s	user 0.025s	sys 0.041s
I20260709 14:56:24.026109 13472 raft_consensus.cc:3060] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.031870 13428 raft_consensus.cc:359] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.032991 13428 raft_consensus.cc:385] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:24.039444 13428 raft_consensus.cc:740] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:24.040764 13472 raft_consensus.cc:2749] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:24.040443 13428 consensus_queue.cc:260] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.043578 13428 ts_tablet_manager.cc:1434] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.002s
I20260709 14:56:24.044737 13428 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:24.048733 13133 catalog_manager.cc:5697] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:24.054734 13432 consensus_queue.cc:1048] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.056970 13428 tablet_bootstrap.cc:654] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:24.068038 13428 tablet_bootstrap.cc:492] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:24.068625 13428 ts_tablet_manager.cc:1403] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.012s
I20260709 14:56:24.071013 13464 raft_consensus.cc:493] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:24.071668 13464 raft_consensus.cc:515] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.071745 13428 raft_consensus.cc:359] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.072585 13428 raft_consensus.cc:385] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:24.072881 13428 raft_consensus.cc:740] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:24.073817 13428 consensus_queue.cc:260] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.075342 13464 leader_election.cc:290] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:24.076159 13432 raft_consensus.cc:493] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:24.076363 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:24.076783 13432 raft_consensus.cc:515] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.077263 13428 ts_tablet_manager.cc:1434] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 14:56:24.076388 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b48dbc27304cf29011afb100eb77bb" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:24.078583 13428 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:24.079376 13432 leader_election.cc:290] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:24.080885 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:24.081646 13311 raft_consensus.cc:2468] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:24.082984 13202 leader_election.cc:304] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:24.077373 13235 raft_consensus.cc:2393] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 65dabe7272bd41ec849103a082ddd59a in current term 1: Already voted for candidate 7e759d511dbf48d389a26747d452fbae in this term.
I20260709 14:56:24.084015 13432 raft_consensus.cc:2804] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:24.084475 13432 raft_consensus.cc:493] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:24.084882 13432 raft_consensus.cc:3060] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.085234 13351 leader_election.cc:304] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:24.086068 13428 tablet_bootstrap.cc:654] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:24.086319 13464 raft_consensus.cc:3060] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.093848 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:24.094786 13385 raft_consensus.cc:2468] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:24.096891 13432 raft_consensus.cc:515] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.100801 13432 leader_election.cc:290] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:24.101701 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:24.102525 13311 raft_consensus.cc:3060] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.107095 13464 raft_consensus.cc:2749] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:24.108520 13428 tablet_bootstrap.cc:492] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:24.108409 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee8ad3334284671a1e3a35ed587356a" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:24.109194 13428 ts_tablet_manager.cc:1403] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 14:56:24.109218 13385 raft_consensus.cc:3060] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.111462 13311 raft_consensus.cc:2468] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:24.112660 13428 raft_consensus.cc:359] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.112923 13202 leader_election.cc:304] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:24.113922 13432 raft_consensus.cc:2804] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:24.113941 13428 raft_consensus.cc:385] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:24.114547 13428 raft_consensus.cc:740] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:24.115425 13428 consensus_queue.cc:260] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.115700 13432 raft_consensus.cc:697] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:24.116868 13432 consensus_queue.cc:237] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.118145 13428 ts_tablet_manager.cc:1434] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 14:56:24.118345 13385 raft_consensus.cc:2468] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:24.128037 13133 catalog_manager.cc:5697] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:24.226879 13464 consensus_queue.cc:1048] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.241315 13310 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.243338 13466 consensus_queue.cc:1048] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.255059 13432 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.269176 13446 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.283428 13432 consensus_queue.cc:1048] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.284055 13466 consensus_queue.cc:1048] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.301103 13464 consensus_queue.cc:1048] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.316167 13494 consensus_queue.cc:1048] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.368376 13472 raft_consensus.cc:493] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:24.369138 13472 raft_consensus.cc:515] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.371658 13472 leader_election.cc:290] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:24.372699 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:24.373129 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:24.373586 13235 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:24.374133 13385 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:24.375140 13277 leader_election.cc:304] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
W20260709 14:56:24.375463 13182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:24.376340 13472 raft_consensus.cc:2804] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:24.376827 13472 raft_consensus.cc:493] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:24.377123 13472 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.385789 13472 raft_consensus.cc:515] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.387928 13472 leader_election.cc:290] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:24.388934 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:24.389045 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:24.389833 13385 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.390372 13235 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:24.396569 13385 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:24.396730 13235 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:24.398226 13277 leader_election.cc:304] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:24.399096 13472 raft_consensus.cc:2804] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:24.399561 13472 raft_consensus.cc:697] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:24.400496 13472 consensus_queue.cc:237] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.408927 13134 catalog_manager.cc:5697] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:24.435848 13472 consensus_queue.cc:1048] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.448772 13477 consensus_queue.cc:1048] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.489382 13477 consensus_queue.cc:1048] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.506814 13472 consensus_queue.cc:1048] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.509012 13446 consensus_queue.cc:1048] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.527894 13446 consensus_queue.cc:1048] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.714380 13384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
mode: GRACEFUL
new_leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
 from {username='slave'} at 127.0.0.1:49926
I20260709 14:56:24.715088 13384 raft_consensus.cc:606] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Received request to transfer leadership to TS 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:24.731396 13384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:49926
I20260709 14:56:24.732085 13384 raft_consensus.cc:606] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:24.750713 13235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a12707f8815f4068b98ef47558aab544"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:59412
I20260709 14:56:24.751544 13235 raft_consensus.cc:606] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:24.763351 13464 raft_consensus.cc:993] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: : Instructing follower 7c7aef1187334e5bb60453d1d4c07ebe to start an election
I20260709 14:56:24.763932 13464 raft_consensus.cc:1081] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Signalling peer 7c7aef1187334e5bb60453d1d4c07ebe to start an election
I20260709 14:56:24.766680 13235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
 from {username='slave'} at 127.0.0.1:59396
I20260709 14:56:24.767330 13235 raft_consensus.cc:493] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:24.767660 13235 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.774564 13310 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
mode: GRACEFUL
new_leader_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:34256
I20260709 14:56:24.775460 13310 raft_consensus.cc:606] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Received request to transfer leadership to TS 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:24.775513 13235 raft_consensus.cc:515] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:24.778662 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:24.779346 13384 raft_consensus.cc:3055] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:24.779735 13384 raft_consensus.cc:740] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:24.779749 13235 leader_election.cc:290] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 2 election: Requested vote from peers 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:24.780331 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e69b1ce56f94e03b714a57aa6e0d983" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:24.781018 13311 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.780740 13384 consensus_queue.cc:260] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:24.782218 13384 raft_consensus.cc:3060] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.787868 13311 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 2.
I20260709 14:56:24.788856 13384 raft_consensus.cc:2468] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 2.
I20260709 14:56:24.789855 13202 leader_election.cc:304] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:24.790993 13446 raft_consensus.cc:2804] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:24.791507 13446 raft_consensus.cc:697] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:24.792440 13446 consensus_queue.cc:237] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } }
I20260709 14:56:24.800165 13134 catalog_manager.cc:5697] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 1 to 2, leader changed from 65dabe7272bd41ec849103a082ddd59a (127.0.107.195) to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 2 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:24.812913 13472 consensus_queue.cc:1048] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:24.824939 13475 consensus_queue.cc:1048] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:24.834918 13475 raft_consensus.cc:993] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: : Instructing follower 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:24.835464 13440 raft_consensus.cc:1081] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Signalling peer 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:24.837339 13384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:49914
I20260709 14:56:24.837986 13384 raft_consensus.cc:493] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:24.838371 13384 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.845127 13384 raft_consensus.cc:515] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.847476 13384 leader_election.cc:290] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:24.848362 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:24.848727 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05bf66306ab9413d9bba98818c0b6b5a" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:24.849078 13235 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.849524 13311 raft_consensus.cc:3055] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:24.849774 13311 raft_consensus.cc:740] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:24.850507 13311 consensus_queue.cc:260] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.851792 13311 raft_consensus.cc:3060] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:24.857154 13235 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:24.857892 13311 raft_consensus.cc:2468] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:24.858500 13351 leader_election.cc:304] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:24.859488 13435 raft_consensus.cc:2804] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:24.859993 13435 raft_consensus.cc:697] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:24.860929 13435 consensus_queue.cc:237] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:24.871060 13134 catalog_manager.cc:5697] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 1 to 2, leader changed from 7e759d511dbf48d389a26747d452fbae (127.0.107.194) to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:25.123060 13430 raft_consensus.cc:993] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:25.123656 13435 raft_consensus.cc:1081] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:25.125607 13311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34232
I20260709 14:56:25.126317 13311 raft_consensus.cc:493] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:25.126749 13311 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.135982 13311 raft_consensus.cc:515] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.137928 13311 leader_election.cc:290] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:25.138795 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:25.139077 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6baf7543824ff5a503aa00c55adc45" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:25.139568 13235 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.139803 13384 raft_consensus.cc:3055] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:25.140178 13384 raft_consensus.cc:740] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:25.140974 13384 consensus_queue.cc:260] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.142153 13384 raft_consensus.cc:3060] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.145959 13235 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:25.147334 13277 leader_election.cc:304] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:25.148144 13440 raft_consensus.cc:2804] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:25.148582 13440 raft_consensus.cc:697] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:25.149464 13440 consensus_queue.cc:237] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.149991 13384 raft_consensus.cc:2468] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:25.157591 13134 catalog_manager.cc:5697] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 65dabe7272bd41ec849103a082ddd59a (127.0.107.195) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:25.193365 13384 raft_consensus.cc:1275] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7c7aef1187334e5bb60453d1d4c07ebe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.195210 13446 consensus_queue.cc:1048] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.207579 13311 raft_consensus.cc:1275] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Refusing update from remote peer 7c7aef1187334e5bb60453d1d4c07ebe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.209317 13444 consensus_queue.cc:1048] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:25.239951 13509 raft_consensus.cc:993] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:25.240509 13444 raft_consensus.cc:1081] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:25.242204 13311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a12707f8815f4068b98ef47558aab544"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34248
I20260709 14:56:25.242794 13311 raft_consensus.cc:493] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:25.243177 13311 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.249502 13311 raft_consensus.cc:515] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.251811 13311 leader_election.cc:290] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:25.252817 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:25.252992 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:25.253651 13235 raft_consensus.cc:3055] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:25.253803 13384 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.254114 13235 raft_consensus.cc:740] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:25.255100 13235 consensus_queue.cc:260] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.256354 13235 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:25.261379 13384 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:25.262781 13235 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:25.263134 13275 leader_election.cc:304] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:25.264307 13440 raft_consensus.cc:2804] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:25.264751 13440 raft_consensus.cc:697] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:25.265494 13440 consensus_queue.cc:237] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:25.273923 13133 catalog_manager.cc:5697] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:25.296645 13235 raft_consensus.cc:1275] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.298306 13435 consensus_queue.cc:1048] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.307310 13311 raft_consensus.cc:1275] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.309063 13464 consensus_queue.cc:1048] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.594864 13384 raft_consensus.cc:1275] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.596246 13440 consensus_queue.cc:1048] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.605605 13235 raft_consensus.cc:1275] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.607470 13477 consensus_queue.cc:1048] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.665283 13235 raft_consensus.cc:1275] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.666715 13477 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:25.676920 13384 raft_consensus.cc:1275] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:25.679256 13477 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:27.821573 13134 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40782:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 14:56:27.826277 13134 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 14:56:27.891569 13219 tablet_service.cc:1511] Processing CreateTablet for tablet e20d324c4a82498a9b9319063c6b8c6d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:27.894711 13296 tablet_service.cc:1511] Processing CreateTablet for tablet e20d324c4a82498a9b9319063c6b8c6d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:27.895172 13373 tablet_service.cc:1511] Processing CreateTablet for tablet e20d324c4a82498a9b9319063c6b8c6d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 14:56:27.897112 13296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d324c4a82498a9b9319063c6b8c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.897114 13373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d324c4a82498a9b9319063c6b8c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.901155 13370 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca8c258194e44d895257978f0a2ee07 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:27.903445 13370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca8c258194e44d895257978f0a2ee07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.897104 13219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d324c4a82498a9b9319063c6b8c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.910337 13220 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca8c258194e44d895257978f0a2ee07 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:27.911064 13295 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca8c258194e44d895257978f0a2ee07 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 14:56:27.912801 13295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca8c258194e44d895257978f0a2ee07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.912385 13223 tablet_service.cc:1511] Processing CreateTablet for tablet 2a55dd7f3a5846e4a887775d892130ce (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:27.912817 13220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca8c258194e44d895257978f0a2ee07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.918463 13223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a55dd7f3a5846e4a887775d892130ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.952170 13374 tablet_service.cc:1511] Processing CreateTablet for tablet 2a55dd7f3a5846e4a887775d892130ce (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:27.955653 13300 tablet_service.cc:1511] Processing CreateTablet for tablet 2a55dd7f3a5846e4a887775d892130ce (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 14:56:27.957684 13300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a55dd7f3a5846e4a887775d892130ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.958230 13374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a55dd7f3a5846e4a887775d892130ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.969660 13221 tablet_service.cc:1511] Processing CreateTablet for tablet 189870e93a894e529fc792e024b798c3 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:27.971776 13221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189870e93a894e529fc792e024b798c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.972992 13301 tablet_service.cc:1511] Processing CreateTablet for tablet 189870e93a894e529fc792e024b798c3 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:27.976987 13375 tablet_service.cc:1511] Processing CreateTablet for tablet 189870e93a894e529fc792e024b798c3 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 14:56:27.978333 13371 tablet_service.cc:1511] Processing CreateTablet for tablet 396a5271c7b54b88b7e7fd0cabd1399d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:27.978982 13375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189870e93a894e529fc792e024b798c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.980067 13371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396a5271c7b54b88b7e7fd0cabd1399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.977022 13301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189870e93a894e529fc792e024b798c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.981508 13298 tablet_service.cc:1511] Processing CreateTablet for tablet 396a5271c7b54b88b7e7fd0cabd1399d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:27.983137 13298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396a5271c7b54b88b7e7fd0cabd1399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:27.984706 13225 tablet_service.cc:1511] Processing CreateTablet for tablet 396a5271c7b54b88b7e7fd0cabd1399d (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 14:56:27.986541 13225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396a5271c7b54b88b7e7fd0cabd1399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.001657 13222 tablet_service.cc:1511] Processing CreateTablet for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:28.002545 13294 tablet_service.cc:1511] Processing CreateTablet for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:28.003643 13222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.004176 13294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.031436 13372 tablet_service.cc:1511] Processing CreateTablet for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 14:56:28.033295 13372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b11d13b5fd47c0bb7dc0e5a56344c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.050974 13226 tablet_service.cc:1511] Processing CreateTablet for tablet 38f2066c01c84ab2830b97582940f560 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:28.052742 13226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f2066c01c84ab2830b97582940f560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.062485 13369 tablet_service.cc:1511] Processing CreateTablet for tablet 38f2066c01c84ab2830b97582940f560 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:28.065542 13369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f2066c01c84ab2830b97582940f560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.065627 13300 tablet_service.cc:1511] Processing CreateTablet for tablet 38f2066c01c84ab2830b97582940f560 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 14:56:28.067241 13300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f2066c01c84ab2830b97582940f560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.098491 13532 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.100847 13225 tablet_service.cc:1511] Processing CreateTablet for tablet 88ad4624d080415b867fb71340293995 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:28.102511 13225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ad4624d080415b867fb71340293995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.106534 13531 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.111373 13295 tablet_service.cc:1511] Processing CreateTablet for tablet 88ad4624d080415b867fb71340293995 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:28.112998 13295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ad4624d080415b867fb71340293995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.116434 13371 tablet_service.cc:1511] Processing CreateTablet for tablet 88ad4624d080415b867fb71340293995 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 14:56:28.118108 13371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ad4624d080415b867fb71340293995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.121376 13223 tablet_service.cc:1511] Processing CreateTablet for tablet 609caffbdcf5479597a579e719e7d023 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:28.123523 13223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609caffbdcf5479597a579e719e7d023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.129351 13294 tablet_service.cc:1511] Processing CreateTablet for tablet 609caffbdcf5479597a579e719e7d023 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:28.131048 13294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609caffbdcf5479597a579e719e7d023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.137660 13533 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.134663 13375 tablet_service.cc:1511] Processing CreateTablet for tablet 609caffbdcf5479597a579e719e7d023 (DEFAULT_TABLE table=soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 14:56:28.134624 13532 tablet_bootstrap.cc:654] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.145529 13375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609caffbdcf5479597a579e719e7d023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 14:56:28.155895 13533 tablet_bootstrap.cc:654] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.160935 13531 tablet_bootstrap.cc:654] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.168092 13532 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.168936 13532 ts_tablet_manager.cc:1403] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.071s	user 0.004s	sys 0.014s
I20260709 14:56:28.172955 13532 raft_consensus.cc:359] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.183858 13532 raft_consensus.cc:385] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.184376 13532 raft_consensus.cc:740] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.192030 13532 consensus_queue.cc:260] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.195676 13532 ts_tablet_manager.cc:1434] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.008s
I20260709 14:56:28.196631 13531 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.197039 13532 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.197332 13531 ts_tablet_manager.cc:1403] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.000s
I20260709 14:56:28.205513 13532 tablet_bootstrap.cc:654] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.209909 13533 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.210721 13533 ts_tablet_manager.cc:1403] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.074s	user 0.023s	sys 0.000s
I20260709 14:56:28.213984 13532 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.214721 13532 ts_tablet_manager.cc:1403] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 14:56:28.214500 13533 raft_consensus.cc:359] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.215451 13533 raft_consensus.cc:385] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.215857 13533 raft_consensus.cc:740] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.216364 13531 raft_consensus.cc:359] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.217233 13531 raft_consensus.cc:385] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.216815 13533 consensus_queue.cc:260] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.217536 13531 raft_consensus.cc:740] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.218443 13531 consensus_queue.cc:260] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.218881 13532 raft_consensus.cc:359] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.219908 13533 ts_tablet_manager.cc:1434] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 14:56:28.220230 13532 raft_consensus.cc:385] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.220554 13532 raft_consensus.cc:740] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.221351 13533 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.221436 13532 consensus_queue.cc:260] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.224681 13532 ts_tablet_manager.cc:1434] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:56:28.226061 13532 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.230298 13533 tablet_bootstrap.cc:654] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.235430 13532 tablet_bootstrap.cc:654] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.244834 13533 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.245608 13533 ts_tablet_manager.cc:1403] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 14:56:28.248354 13532 tablet_bootstrap.cc:492] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.249156 13532 ts_tablet_manager.cc:1403] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 14:56:28.249583 13533 raft_consensus.cc:359] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.250551 13533 raft_consensus.cc:385] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.250892 13533 raft_consensus.cc:740] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.251883 13533 consensus_queue.cc:260] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.253254 13532 raft_consensus.cc:359] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.254042 13533 ts_tablet_manager.cc:1434] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.003s
I20260709 14:56:28.254168 13532 raft_consensus.cc:385] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.254576 13532 raft_consensus.cc:740] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.255434 13533 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.255604 13532 consensus_queue.cc:260] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.261866 13532 ts_tablet_manager.cc:1434] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 14:56:28.263504 13532 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.265409 13533 tablet_bootstrap.cc:654] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.273845 13532 tablet_bootstrap.cc:654] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.282708 13533 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.283629 13533 ts_tablet_manager.cc:1403] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.012s
I20260709 14:56:28.288386 13533 raft_consensus.cc:359] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.289319 13533 raft_consensus.cc:385] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.289664 13533 raft_consensus.cc:740] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.290094 13532 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.290865 13532 ts_tablet_manager.cc:1403] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.003s
I20260709 14:56:28.290601 13533 consensus_queue.cc:260] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.293839 13533 ts_tablet_manager.cc:1434] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:56:28.295368 13533 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.295255 13532 raft_consensus.cc:359] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.296370 13532 raft_consensus.cc:385] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.296749 13532 raft_consensus.cc:740] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.297726 13532 consensus_queue.cc:260] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.300798 13532 ts_tablet_manager.cc:1434] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 14:56:28.302269 13532 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.303189 13515 raft_consensus.cc:493] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.304342 13515 raft_consensus.cc:515] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.305018 13533 tablet_bootstrap.cc:654] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.311311 13532 tablet_bootstrap.cc:654] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.315043 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.320330 13533 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.321377 13533 ts_tablet_manager.cc:1403] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.000s
W20260709 14:56:28.322562 13202 leader_election.cc:343] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.324834 13532 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.325510 13532 ts_tablet_manager.cc:1403] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.017s
I20260709 14:56:28.329226 13532 raft_consensus.cc:359] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.329128 13533 raft_consensus.cc:359] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.330389 13532 raft_consensus.cc:385] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.330935 13532 raft_consensus.cc:740] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.332188 13532 consensus_queue.cc:260] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.335644 13517 raft_consensus.cc:493] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.335918 13532 ts_tablet_manager.cc:1434] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.008s
I20260709 14:56:28.336517 13517 raft_consensus.cc:515] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.337342 13532 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.340405 13517 leader_election.cc:290] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.345548 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.346772 13532 tablet_bootstrap.cc:654] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
W20260709 14:56:28.349007 13351 leader_election.cc:343] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.330435 13533 raft_consensus.cc:385] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.351928 13533 raft_consensus.cc:740] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.356626 13533 consensus_queue.cc:260] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.353196 13515 leader_election.cc:290] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.361394 13515 raft_consensus.cc:493] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.362195 13515 raft_consensus.cc:515] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.365475 13515 leader_election.cc:290] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.367344 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca8c258194e44d895257978f0a2ee07" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.368572 13310 raft_consensus.cc:2468] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.368960 13533 ts_tablet_manager.cc:1434] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.003s
I20260709 14:56:28.370901 13533 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.350795 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.376483 13531 ts_tablet_manager.cc:1434] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.179s	user 0.068s	sys 0.082s
I20260709 14:56:28.378001 13531 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.378409 13202 leader_election.cc:304] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.379806 13520 raft_consensus.cc:2804] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.380267 13520 raft_consensus.cc:493] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.380640 13520 raft_consensus.cc:3060] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
W20260709 14:56:28.383633 13351 leader_election.cc:343] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.393008 13533 tablet_bootstrap.cc:654] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.394210 13531 tablet_bootstrap.cc:654] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.400171 13532 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.401028 13532 ts_tablet_manager.cc:1403] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.064s	user 0.039s	sys 0.012s
I20260709 14:56:28.400879 13520 raft_consensus.cc:515] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.404760 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca8c258194e44d895257978f0a2ee07" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.405601 13311 raft_consensus.cc:3060] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.406003 13532 raft_consensus.cc:359] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.408633 13532 raft_consensus.cc:385] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.409036 13532 raft_consensus.cc:740] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.410317 13532 consensus_queue.cc:260] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.390374 13351 leader_election.cc:304] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.412470 13520 leader_election.cc:290] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.413327 13517 raft_consensus.cc:2749] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:28.414793 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.415714 13385 raft_consensus.cc:2468] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.416977 13532 ts_tablet_manager.cc:1434] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 14:56:28.417423 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca8c258194e44d895257978f0a2ee07" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.418207 13384 raft_consensus.cc:2468] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
W20260709 14:56:28.420989 13182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 14:56:28.421243 13532 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.422446 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca8c258194e44d895257978f0a2ee07" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.423187 13384 raft_consensus.cc:3060] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.423786 13311 raft_consensus.cc:2468] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.425444 13202 leader_election.cc:304] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.426401 13515 raft_consensus.cc:2804] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.426921 13515 raft_consensus.cc:697] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.427891 13515 consensus_queue.cc:237] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.428797 13533 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.429558 13533 ts_tablet_manager.cc:1403] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.011s
I20260709 14:56:28.433544 13384 raft_consensus.cc:2468] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.433329 13533 raft_consensus.cc:359] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.434448 13533 raft_consensus.cc:385] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.434887 13533 raft_consensus.cc:740] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.436493 13533 consensus_queue.cc:260] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.437609 13531 tablet_bootstrap.cc:492] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.438302 13531 ts_tablet_manager.cc:1403] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.010s
I20260709 14:56:28.439549 13533 ts_tablet_manager.cc:1434] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 14:56:28.440912 13533 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.441887 13532 tablet_bootstrap.cc:654] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.442389 13531 raft_consensus.cc:359] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.443488 13531 raft_consensus.cc:385] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.443852 13531 raft_consensus.cc:740] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.444859 13531 consensus_queue.cc:260] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.446203 13200 leader_election.cc:304] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.449996 13520 raft_consensus.cc:2804] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.450676 13520 raft_consensus.cc:493] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.451198 13520 raft_consensus.cc:3060] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.452605 13531 ts_tablet_manager.cc:1434] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 14:56:28.453922 13533 tablet_bootstrap.cc:654] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.461349 13532 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.462037 13532 ts_tablet_manager.cc:1403] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.041s	user 0.027s	sys 0.001s
I20260709 14:56:28.462198 13517 raft_consensus.cc:493] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.463340 13546 raft_consensus.cc:493] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.463068 13517 raft_consensus.cc:515] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.463917 13546 raft_consensus.cc:515] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.453998 13531 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.466336 13546 leader_election.cc:290] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.468468 13552 raft_consensus.cc:493] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.469060 13552 raft_consensus.cc:515] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.468039 13532 raft_consensus.cc:359] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.471709 13532 raft_consensus.cc:385] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.474792 13532 raft_consensus.cc:740] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.475274 13531 tablet_bootstrap.cc:654] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.476529 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.476153 13532 consensus_queue.cc:260] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.476564 13517 leader_election.cc:290] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.477921 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.478608 13235 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:28.479947 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.472381 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.473201 13552 leader_election.cc:290] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.474294 13533 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.484397 13533 ts_tablet_manager.cc:1403] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.004s
W20260709 14:56:28.486112 13202 leader_election.cc:343] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.488636 13531 tablet_bootstrap.cc:492] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.488355 13533 raft_consensus.cc:359] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.489260 13531 ts_tablet_manager.cc:1403] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.011s
I20260709 14:56:28.489344 13533 raft_consensus.cc:385] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.489974 13533 raft_consensus.cc:740] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.491050 13533 consensus_queue.cc:260] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.493968 13533 ts_tablet_manager.cc:1434] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 14:56:28.495401 13533 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.496488 13520 raft_consensus.cc:515] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.499619 13131 catalog_manager.cc:5697] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.503325 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.503731 13351 leader_election.cc:304] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:28.504525 13384 raft_consensus.cc:2468] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
W20260709 14:56:28.504905 13351 leader_election.cc:343] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Illegal state: must be running to vote when last-logged opid is not known
W20260709 14:56:28.505752 13351 leader_election.cc:343] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.506250 13351 leader_election.cc:304] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.506932 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.507871 13553 raft_consensus.cc:2804] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.508383 13553 raft_consensus.cc:493] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.508417 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.508868 13553 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.509233 13311 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:28.509382 13531 raft_consensus.cc:359] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.510320 13531 raft_consensus.cc:385] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 14:56:28.511473 13202 leader_election.cc:343] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.512313 13552 raft_consensus.cc:2749] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:28.511509 13533 tablet_bootstrap.cc:654] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.512455 13520 leader_election.cc:290] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.510696 13531 raft_consensus.cc:740] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.514459 13531 consensus_queue.cc:260] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.516084 13200 leader_election.cc:304] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.517475 13520 raft_consensus.cc:2804] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.517678 13531 ts_tablet_manager.cc:1434] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 14:56:28.517900 13520 raft_consensus.cc:493] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.518414 13520 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.527887 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.528757 13385 raft_consensus.cc:3060] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.530649 13531 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.535879 13553 raft_consensus.cc:515] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.543301 13531 tablet_bootstrap.cc:654] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.542763 13385 raft_consensus.cc:2468] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.549793 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:28.551434 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.552891 13236 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.553172 13200 leader_election.cc:304] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.553052 13520 raft_consensus.cc:515] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.554761 13515 raft_consensus.cc:2804] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.555404 13515 raft_consensus.cc:697] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.551153 13532 ts_tablet_manager.cc:1434] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.089s	user 0.013s	sys 0.027s
I20260709 14:56:28.556442 13515 consensus_queue.cc:237] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.557941 13532 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.548758 13553 leader_election.cc:290] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.559365 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.559653 13311 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.560127 13310 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.560276 13520 leader_election.cc:290] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.567516 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.568392 13385 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.549614 13533 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.575317 13311 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:28.577096 13351 leader_election.cc:304] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.577977 13553 raft_consensus.cc:2804] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.578476 13553 raft_consensus.cc:697] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:28.579325 13553 consensus_queue.cc:237] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.582813 13532 tablet_bootstrap.cc:654] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.584164 13533 ts_tablet_manager.cc:1403] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.010s
I20260709 14:56:28.588017 13515 raft_consensus.cc:493] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.591041 13515 raft_consensus.cc:515] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.594513 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
W20260709 14:56:28.596058 13202 leader_election.cc:343] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.590507 13531 tablet_bootstrap.cc:492] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.597123 13531 ts_tablet_manager.cc:1403] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.067s	user 0.029s	sys 0.001s
I20260709 14:56:28.599509 13134 catalog_manager.cc:5697] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.602308 13533 raft_consensus.cc:359] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.603372 13533 raft_consensus.cc:385] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.603708 13533 raft_consensus.cc:740] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.604748 13310 raft_consensus.cc:2468] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.607236 13202 leader_election.cc:304] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.608583 13520 raft_consensus.cc:2804] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.609108 13520 raft_consensus.cc:697] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.609792 13515 leader_election.cc:290] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.610038 13520 consensus_queue.cc:237] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.621942 13532 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.622550 13532 ts_tablet_manager.cc:1403] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.001s
I20260709 14:56:28.611270 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.609583 13131 catalog_manager.cc:5697] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:28.623394 13384 raft_consensus.cc:2468] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.625658 13200 leader_election.cc:304] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.626173 13532 raft_consensus.cc:359] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.627079 13532 raft_consensus.cc:385] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.627368 13532 raft_consensus.cc:740] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.627666 13515 raft_consensus.cc:2804] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.628228 13515 raft_consensus.cc:493] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.613657 13531 raft_consensus.cc:359] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.628832 13515 raft_consensus.cc:3060] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.629168 13531 raft_consensus.cc:385] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.629523 13531 raft_consensus.cc:740] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.616314 13236 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:28.604553 13533 consensus_queue.cc:260] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.619333 13385 raft_consensus.cc:2468] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.631335 13531 consensus_queue.cc:260] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.628196 13532 consensus_queue.cc:260] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.634528 13531 ts_tablet_manager.cc:1434] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.036s	user 0.001s	sys 0.008s
I20260709 14:56:28.635291 13532 ts_tablet_manager.cc:1434] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 14:56:28.636222 13531 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.636597 13532 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: Bootstrap starting.
I20260709 14:56:28.641487 13515 raft_consensus.cc:515] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.644670 13531 tablet_bootstrap.cc:654] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.644677 13532 tablet_bootstrap.cc:654] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.647998 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
W20260709 14:56:28.649762 13202 leader_election.cc:343] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.652494 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.653630 13533 ts_tablet_manager.cc:1434] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 14:56:28.653523 13385 raft_consensus.cc:3060] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.655006 13533 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.652336 13515 leader_election.cc:290] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.664848 13385 raft_consensus.cc:2468] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.666437 13531 tablet_bootstrap.cc:492] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.665541 13134 catalog_manager.cc:5697] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.666589 13200 leader_election.cc:304] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.667273 13531 ts_tablet_manager.cc:1403] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.012s
I20260709 14:56:28.667765 13520 raft_consensus.cc:2804] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.668375 13520 raft_consensus.cc:697] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.669339 13520 consensus_queue.cc:237] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.671114 13531 raft_consensus.cc:359] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.671970 13531 raft_consensus.cc:385] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.672303 13531 raft_consensus.cc:740] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.673174 13533 tablet_bootstrap.cc:654] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.673305 13531 consensus_queue.cc:260] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.676065 13531 ts_tablet_manager.cc:1434] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:56:28.677488 13531 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.681305 13532 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: No bootstrap required, opened a new log
I20260709 14:56:28.682163 13532 ts_tablet_manager.cc:1403] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.000s
I20260709 14:56:28.683749 13546 raft_consensus.cc:493] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.684422 13546 raft_consensus.cc:515] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.685482 13531 tablet_bootstrap.cc:654] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.686755 13532 raft_consensus.cc:359] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.687266 13546 leader_election.cc:290] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.688176 13553 raft_consensus.cc:493] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.689848 13553 raft_consensus.cc:515] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.690244 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.692476 13553 leader_election.cc:290] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
W20260709 14:56:28.692843 13202 leader_election.cc:343] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.694324 13532 raft_consensus.cc:385] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.695770 13532 raft_consensus.cc:740] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.696058 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.697046 13236 raft_consensus.cc:2468] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:28.694701 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.697355 13532 consensus_queue.cc:260] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.698388 13351 leader_election.cc:304] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:28.702471 13553 raft_consensus.cc:2804] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 14:56:28.702694 13351 leader_election.cc:343] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.702978 13553 raft_consensus.cc:493] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.703375 13553 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.706094 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.707033 13385 raft_consensus.cc:2468] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.708555 13200 leader_election.cc:304] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.710932 13532 ts_tablet_manager.cc:1434] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.004s
I20260709 14:56:28.721148 13134 catalog_manager.cc:5697] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.725462 13531 tablet_bootstrap.cc:492] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.726198 13531 ts_tablet_manager.cc:1403] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.011s
I20260709 14:56:28.729803 13531 raft_consensus.cc:359] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.729933 13553 raft_consensus.cc:515] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.730723 13531 raft_consensus.cc:385] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.734894 13553 leader_election.cc:290] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.735467 13531 raft_consensus.cc:740] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.735576 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.736737 13531 consensus_queue.cc:260] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.737871 13310 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.716084 13520 raft_consensus.cc:2804] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.739372 13520 raft_consensus.cc:493] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.739373 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:28.739959 13520 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.740267 13236 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.734997 13533 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.743055 13533 ts_tablet_manager.cc:1403] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.010s
I20260709 14:56:28.746884 13533 raft_consensus.cc:359] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.747684 13533 raft_consensus.cc:385] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.747977 13533 raft_consensus.cc:740] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.748399 13531 ts_tablet_manager.cc:1434] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.004s
I20260709 14:56:28.748761 13533 consensus_queue.cc:260] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.749837 13531 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.755916 13236 raft_consensus.cc:2468] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:28.759599 13531 tablet_bootstrap.cc:654] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.759847 13533 ts_tablet_manager.cc:1434] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 14:56:28.760646 13351 leader_election.cc:304] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:28.761114 13533 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: Bootstrap starting.
I20260709 14:56:28.766017 13310 raft_consensus.cc:2468] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 1.
I20260709 14:56:28.770169 13553 raft_consensus.cc:2804] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.772214 13553 raft_consensus.cc:697] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:28.768064 13520 raft_consensus.cc:515] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.773720 13553 consensus_queue.cc:237] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.777269 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
W20260709 14:56:28.783049 13202 leader_election.cc:343] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 14:56:28.785238 13520 leader_election.cc:290] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.784216 13533 tablet_bootstrap.cc:654] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.788424 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.789273 13385 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.797555 13531 tablet_bootstrap.cc:492] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.798466 13533 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: No bootstrap required, opened a new log
I20260709 14:56:28.798341 13531 ts_tablet_manager.cc:1403] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.017s
I20260709 14:56:28.799353 13533 ts_tablet_manager.cc:1403] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.003s
I20260709 14:56:28.800465 13385 raft_consensus.cc:2468] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.801824 13200 leader_election.cc:304] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.802630 13520 raft_consensus.cc:2804] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.803210 13520 raft_consensus.cc:697] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.802800 13533 raft_consensus.cc:359] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.803779 13533 raft_consensus.cc:385] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.804103 13533 raft_consensus.cc:740] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.804199 13520 consensus_queue.cc:237] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.805269 13533 consensus_queue.cc:260] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.805469 13531 raft_consensus.cc:359] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.806571 13531 raft_consensus.cc:385] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.806969 13531 raft_consensus.cc:740] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.807958 13533 ts_tablet_manager.cc:1434] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 14:56:28.807885 13531 consensus_queue.cc:260] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.810794 13531 ts_tablet_manager.cc:1434] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 14:56:28.811101 13134 catalog_manager.cc:5697] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:28.811993 13531 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.822774 13531 tablet_bootstrap.cc:654] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.829033 13531 tablet_bootstrap.cc:492] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.829543 13531 ts_tablet_manager.cc:1403] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 14:56:28.830435 13134 catalog_manager.cc:5697] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.832077 13531 raft_consensus.cc:359] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.832844 13531 raft_consensus.cc:385] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.833148 13531 raft_consensus.cc:740] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.833881 13531 consensus_queue.cc:260] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.838595 13531 ts_tablet_manager.cc:1434] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 14:56:28.839754 13531 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: Bootstrap starting.
I20260709 14:56:28.845721 13531 tablet_bootstrap.cc:654] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: Neither blocks nor log segments found. Creating new log.
I20260709 14:56:28.852067 13477 raft_consensus.cc:493] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.853029 13531 tablet_bootstrap.cc:492] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: No bootstrap required, opened a new log
I20260709 14:56:28.852763 13477 raft_consensus.cc:515] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.853790 13531 ts_tablet_manager.cc:1403] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.001s
I20260709 14:56:28.855558 13477 leader_election.cc:290] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.857108 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.857575 13531 raft_consensus.cc:359] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.858649 13531 raft_consensus.cc:385] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 14:56:28.858492 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189870e93a894e529fc792e024b798c3" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.859004 13520 raft_consensus.cc:493] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.859231 13531 raft_consensus.cc:740] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Initialized, Role: FOLLOWER
I20260709 14:56:28.859581 13385 raft_consensus.cc:2393] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e759d511dbf48d389a26747d452fbae in current term 1: Already voted for candidate 7c7aef1187334e5bb60453d1d4c07ebe in this term.
I20260709 14:56:28.860168 13531 consensus_queue.cc:260] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.859689 13520 raft_consensus.cc:515] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.861310 13275 leader_election.cc:304] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:28.862610 13531 ts_tablet_manager.cc:1434] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 14:56:28.863229 13477 raft_consensus.cc:3060] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.863780 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.864547 13310 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.864946 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.865578 13520 leader_election.cc:290] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.865608 13385 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 0.
I20260709 14:56:28.866319 13202 leader_election.cc:304] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.867326 13520 raft_consensus.cc:2804] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.867738 13520 raft_consensus.cc:493] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.868093 13520 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.871280 13515 consensus_queue.cc:1048] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:28.873728 13477 raft_consensus.cc:2749] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:28.877002 13520 raft_consensus.cc:515] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.879622 13520 leader_election.cc:290] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.880865 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.880897 13311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.881628 13385 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.881719 13311 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.889372 13477 raft_consensus.cc:493] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.889470 13385 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.890290 13477 raft_consensus.cc:515] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.890671 13553 raft_consensus.cc:493] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:28.891310 13553 raft_consensus.cc:515] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.892252 13477 leader_election.cc:290] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.893505 13553 leader_election.cc:290] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.894054 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:28.894851 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.895067 13385 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:28.895596 13236 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 0.
I20260709 14:56:28.896685 13275 leader_election.cc:304] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:28.898001 13477 raft_consensus.cc:2804] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.898552 13477 raft_consensus.cc:493] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.899044 13477 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.900825 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae" is_pre_election: true
I20260709 14:56:28.901378 13200 leader_election.cc:304] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:28.902599 13515 raft_consensus.cc:2804] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.908121 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:28.909097 13236 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 0.
I20260709 14:56:28.910929 13311 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 1.
I20260709 14:56:28.911199 13351 leader_election.cc:304] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:28.912985 13553 raft_consensus.cc:2804] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 14:56:28.914543 13553 raft_consensus.cc:493] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 14:56:28.915006 13553 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.915258 13477 raft_consensus.cc:515] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.916635 13309 raft_consensus.cc:2393] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 65dabe7272bd41ec849103a082ddd59a in current term 1: Already voted for candidate 7e759d511dbf48d389a26747d452fbae in this term.
I20260709 14:56:28.919582 13477 leader_election.cc:290] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:28.920585 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:28.921025 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:28.921357 13236 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:28.913575 13515 raft_consensus.cc:697] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:28.925187 13515 consensus_queue.cc:237] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.928306 13553 raft_consensus.cc:515] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.930764 13553 leader_election.cc:290] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:28.932078 13385 raft_consensus.cc:2393] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e759d511dbf48d389a26747d452fbae in current term 1: Already voted for candidate 65dabe7272bd41ec849103a082ddd59a in this term.
I20260709 14:56:28.935130 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:28.935292 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:28.935601 13236 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 1.
I20260709 14:56:28.936178 13309 raft_consensus.cc:2393] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 65dabe7272bd41ec849103a082ddd59a in current term 1: Already voted for candidate 7e759d511dbf48d389a26747d452fbae in this term.
I20260709 14:56:28.937249 13277 leader_election.cc:304] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:28.937814 13351 leader_election.cc:304] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a; no voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:28.939042 13553 raft_consensus.cc:2749] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 14:56:28.939533 13477 raft_consensus.cc:2804] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 14:56:28.940137 13477 raft_consensus.cc:697] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:28.940980 13477 consensus_queue.cc:237] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:28.945454 13515 consensus_queue.cc:1048] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:28.948249 13134 catalog_manager.cc:5697] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.952767 13133 catalog_manager.cc:5697] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 1 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:28.965492 13554 consensus_queue.cc:1048] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:28.989991 13552 consensus_queue.cc:1048] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.007076 13477 raft_consensus.cc:493] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:29.007692 13477 raft_consensus.cc:515] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.010463 13515 consensus_queue.cc:1048] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.011325 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:29.011210 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a55dd7f3a5846e4a887775d892130ce" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:29.012020 13477 leader_election.cc:290] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.013415 13385 raft_consensus.cc:2393] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e759d511dbf48d389a26747d452fbae in current term 1: Already voted for candidate 7c7aef1187334e5bb60453d1d4c07ebe in this term.
I20260709 14:56:29.014601 13275 leader_election.cc:304] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:29.016149 13477 raft_consensus.cc:3060] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:29.026867 13384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
mode: GRACEFUL
new_leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
 from {username='slave'} at 127.0.0.1:49926
I20260709 14:56:29.027251 13580 consensus_queue.cc:1048] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.027906 13384 raft_consensus.cc:606] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Received request to transfer leadership to TS 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:29.036829 13477 raft_consensus.cc:2749] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:29.042514 13580 consensus_queue.cc:1048] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.063241 13236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:59412
I20260709 14:56:29.064034 13236 raft_consensus.cc:606] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:29.068064 13579 consensus_queue.cc:1048] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.080673 13384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:49926
I20260709 14:56:29.081489 13384 raft_consensus.cc:606] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:29.085843 13546 raft_consensus.cc:993] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.086405 13546 raft_consensus.cc:1081] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.089824 13310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34248
I20260709 14:56:29.090760 13310 raft_consensus.cc:493] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.090605 13236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38f2066c01c84ab2830b97582940f560"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:59412
I20260709 14:56:29.091555 13310 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.091663 13236 raft_consensus.cc:606] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:29.099625 13236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88ad4624d080415b867fb71340293995"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
mode: GRACEFUL
new_leader_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:59412
I20260709 14:56:29.100328 13236 raft_consensus.cc:606] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Received request to transfer leadership to TS 7e759d511dbf48d389a26747d452fbae
I20260709 14:56:29.100976 13310 raft_consensus.cc:515] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.103554 13310 leader_election.cc:290] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.106120 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:29.106585 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396a5271c7b54b88b7e7fd0cabd1399d" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:29.107203 13236 raft_consensus.cc:3055] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.107461 13384 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.107673 13236 raft_consensus.cc:740] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:29.109159 13310 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "609caffbdcf5479597a579e719e7d023"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
mode: GRACEFUL
new_leader_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:34256
I20260709 14:56:29.110107 13310 raft_consensus.cc:606] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Received request to transfer leadership to TS 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:29.109195 13236 consensus_queue.cc:260] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.111526 13236 raft_consensus.cc:3060] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.118454 13236 raft_consensus.cc:2468] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.119835 13277 leader_election.cc:304] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:29.120820 13477 raft_consensus.cc:2804] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.121446 13477 raft_consensus.cc:697] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:29.122220 13477 consensus_queue.cc:237] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.123133 13384 raft_consensus.cc:2468] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.131102 13132 catalog_manager.cc:5697] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:29.185084 13546 consensus_queue.cc:1048] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:29.198871 13580 consensus_queue.cc:1048] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.201067 13553 consensus_queue.cc:1048] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.235041 13552 consensus_queue.cc:1048] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 14:56:29.250072 13553 raft_consensus.cc:993] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.250648 13552 raft_consensus.cc:1081] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.252664 13310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34232
I20260709 14:56:29.253260 13310 raft_consensus.cc:493] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.253597 13310 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.260802 13310 raft_consensus.cc:515] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.262913 13310 leader_election.cc:290] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.264094 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:29.264151 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b11d13b5fd47c0bb7dc0e5a56344c1" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:29.264986 13236 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.265075 13384 raft_consensus.cc:3055] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.265610 13384 raft_consensus.cc:740] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:29.266537 13384 consensus_queue.cc:260] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.267843 13384 raft_consensus.cc:3060] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.272363 13477 raft_consensus.cc:493] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 14:56:29.273150 13477 raft_consensus.cc:515] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.275761 13477 leader_election.cc:290] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.275605 13236 raft_consensus.cc:2468] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.277422 13384 raft_consensus.cc:2468] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.277400 13546 consensus_queue.cc:1048] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.277910 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dabe7272bd41ec849103a082ddd59a" is_pre_election: true
I20260709 14:56:29.278811 13385 raft_consensus.cc:2393] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e759d511dbf48d389a26747d452fbae in current term 1: Already voted for candidate 7c7aef1187334e5bb60453d1d4c07ebe in this term.
I20260709 14:56:29.279035 13277 leader_election.cc:304] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:29.279917 13235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" is_pre_election: true
I20260709 14:56:29.280478 13477 raft_consensus.cc:2804] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.281842 13275 leader_election.cc:304] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e759d511dbf48d389a26747d452fbae; no voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe
I20260709 14:56:29.281890 13477 raft_consensus.cc:697] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:29.283442 13477 consensus_queue.cc:237] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.283706 13538 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 14:56:29.293697 13132 catalog_manager.cc:5697] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 65dabe7272bd41ec849103a082ddd59a (127.0.107.195) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:29.303472 13538 raft_consensus.cc:2749] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 14:56:29.309584 13520 consensus_queue.cc:1048] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.319609 13580 raft_consensus.cc:993] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.320233 13520 raft_consensus.cc:1081] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.322456 13310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38f2066c01c84ab2830b97582940f560"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34248
I20260709 14:56:29.323309 13310 raft_consensus.cc:493] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.323738 13310 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.330734 13310 raft_consensus.cc:515] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.333110 13310 leader_election.cc:290] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.334616 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:29.334800 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f2066c01c84ab2830b97582940f560" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:29.335734 13385 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.335901 13236 raft_consensus.cc:3055] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.336404 13236 raft_consensus.cc:740] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:29.337416 13236 consensus_queue.cc:260] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.339120 13236 raft_consensus.cc:3060] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.343034 13385 raft_consensus.cc:2468] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.344447 13275 leader_election.cc:304] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:29.345499 13538 raft_consensus.cc:2804] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.346114 13538 raft_consensus.cc:697] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:29.346657 13236 raft_consensus.cc:2468] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.347147 13538 consensus_queue.cc:237] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.357445 13132 catalog_manager.cc:5697] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:29.388286 13538 consensus_queue.cc:1048] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 14:56:29.404943 13520 consensus_queue.cc:1048] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.406888 13538 consensus_queue.cc:1048] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.423224 13515 consensus_queue.cc:1048] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 14:56:29.429019 13477 raft_consensus.cc:993] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: : Instructing follower 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:29.429639 13477 raft_consensus.cc:1081] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Signalling peer 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:29.432967 13385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "609caffbdcf5479597a579e719e7d023"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:49914
I20260709 14:56:29.434661 13385 raft_consensus.cc:493] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.435797 13385 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.447661 13385 raft_consensus.cc:515] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.448864 13579 raft_consensus.cc:993] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: : Instructing follower 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.449473 13579 raft_consensus.cc:1081] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Signalling peer 7e759d511dbf48d389a26747d452fbae to start an election
I20260709 14:56:29.451670 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:29.452349 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "609caffbdcf5479597a579e719e7d023" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:29.452337 13236 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.452509 13385 leader_election.cc:290] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:29.453302 13309 raft_consensus.cc:3055] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.453661 13309 raft_consensus.cc:740] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:29.454559 13309 consensus_queue.cc:260] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.452261 13310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88ad4624d080415b867fb71340293995"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
 from {username='slave'} at 127.0.0.1:34248
I20260709 14:56:29.455684 13310 raft_consensus.cc:493] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.456063 13309 raft_consensus.cc:3060] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.456146 13310 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.466353 13310 raft_consensus.cc:515] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.466362 13236 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:29.469054 13351 leader_election.cc:304] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:29.469551 13309 raft_consensus.cc:2468] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:29.470177 13552 raft_consensus.cc:2804] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.470798 13552 raft_consensus.cc:697] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:29.471220 13310 leader_election.cc:290] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.471063 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:29.472008 13552 consensus_queue.cc:237] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.472806 13385 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.475404 13236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ad4624d080415b867fb71340293995" candidate_uuid: "7e759d511dbf48d389a26747d452fbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:29.476207 13236 raft_consensus.cc:3055] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.476519 13236 raft_consensus.cc:740] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:29.477357 13236 consensus_queue.cc:260] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.478731 13236 raft_consensus.cc:3060] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.479698 13385 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.481173 13275 leader_election.cc:304] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:29.482142 13538 raft_consensus.cc:2804] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.481848 13131 catalog_manager.cc:5697] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 1 to 2, leader changed from 7e759d511dbf48d389a26747d452fbae (127.0.107.194) to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:29.482800 13538 raft_consensus.cc:697] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Leader. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:29.483862 13538 consensus_queue.cc:237] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.487293 13236 raft_consensus.cc:2468] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e759d511dbf48d389a26747d452fbae in term 2.
I20260709 14:56:29.493535 13134 catalog_manager.cc:5697] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae reported cstate change: term changed from 1 to 2, leader changed from 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193) to 7e759d511dbf48d389a26747d452fbae (127.0.107.194). New cstate: current_term: 2 leader_uuid: "7e759d511dbf48d389a26747d452fbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:29.541675 13554 raft_consensus.cc:993] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: : Instructing follower 7c7aef1187334e5bb60453d1d4c07ebe to start an election
I20260709 14:56:29.542299 13552 raft_consensus.cc:1081] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Signalling peer 7c7aef1187334e5bb60453d1d4c07ebe to start an election
I20260709 14:56:29.544415 13235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
 from {username='slave'} at 127.0.0.1:59396
I20260709 14:56:29.545207 13235 raft_consensus.cc:493] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:29.545526 13235 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.552536 13235 raft_consensus.cc:515] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.554502 13235 leader_election.cc:290] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 2 election: Requested vote from peers 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319), 65dabe7272bd41ec849103a082ddd59a (127.0.107.195:39583)
I20260709 14:56:29.555658 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:29.555940 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d324c4a82498a9b9319063c6b8c6d" candidate_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
I20260709 14:56:29.556423 13310 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.556645 13385 raft_consensus.cc:3055] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:29.557111 13385 raft_consensus.cc:740] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:29.558009 13385 consensus_queue.cc:260] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.559352 13385 raft_consensus.cc:3060] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:29.564770 13310 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 2.
I20260709 14:56:29.566040 13385 raft_consensus.cc:2468] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7aef1187334e5bb60453d1d4c07ebe in term 2.
I20260709 14:56:29.566502 13202 leader_election.cc:304] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7aef1187334e5bb60453d1d4c07ebe, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:29.567523 13515 raft_consensus.cc:2804] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:29.568051 13515 raft_consensus.cc:697] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 LEADER]: Becoming Leader. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:29.568984 13515 consensus_queue.cc:237] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:29.571614 13385 raft_consensus.cc:1275] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.574528 13538 consensus_queue.cc:1048] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 14:56:29.578819 13134 catalog_manager.cc:5697] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe reported cstate change: term changed from 1 to 2, leader changed from 65dabe7272bd41ec849103a082ddd59a (127.0.107.195) to 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193). New cstate: current_term: 2 leader_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 14:56:29.589706 13235 raft_consensus.cc:1275] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.591687 13598 consensus_queue.cc:1048] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:29.728113 13385 raft_consensus.cc:1275] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.729727 13598 consensus_queue.cc:1048] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:29.740334 13236 raft_consensus.cc:1275] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.745066 13538 consensus_queue.cc:1048] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:29.854431 13385 raft_consensus.cc:1275] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.856210 13538 consensus_queue.cc:1048] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:29.868503 13236 raft_consensus.cc:1275] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.870349 13310 raft_consensus.cc:1275] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.870647 13598 consensus_queue.cc:1048] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:29.872473 13552 consensus_queue.cc:1048] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:29.883419 13236 raft_consensus.cc:1275] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:29.890434 13554 consensus_queue.cc:1048] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:30.012806 13310 raft_consensus.cc:1275] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Refusing update from remote peer 7c7aef1187334e5bb60453d1d4c07ebe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:30.014398 13515 consensus_queue.cc:1048] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:30.024899 13383 raft_consensus.cc:1275] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7c7aef1187334e5bb60453d1d4c07ebe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:30.026809 13520 consensus_queue.cc:1048] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:30.050060 13385 raft_consensus.cc:1275] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:30.051712 13598 consensus_queue.cc:1048] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 14:56:30.061411 13236 raft_consensus.cc:1275] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 7e759d511dbf48d389a26747d452fbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:30.064599 13599 consensus_queue.cc:1048] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:32.114792 13131 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40782:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 14:56:32.139489 13131 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40782:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 14:56:32.205935   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 14:56:32.485090 13655 tablet.cc:1724] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.526558 13235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba"
dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
mode: GRACEFUL
new_leader_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:44332
I20260709 14:56:32.527395 13235 raft_consensus.cc:606] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Received request to transfer leadership to TS 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:32.538707 13659 tablet.cc:1724] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:32.549055 13275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 93039.2us
W20260709 14:56:32.551028 13277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:46565, user_credentials={real_user=slave}} blocked reactor thread for 94270.3us
I20260709 14:56:32.552436 13668 tablet.cc:1724] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.607899 13658 tablet.cc:1724] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.633054 13655 tablet.cc:1724] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.637583 13669 tablet.cc:1724] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.644698 13665 tablet.cc:1724] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.702030 13673 tablet.cc:1724] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:32.727501 13657 log.cc:927] Time spent T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 14:56:32.737705 13275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 68879us
I20260709 14:56:32.741474 13307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a12707f8815f4068b98ef47558aab544"
dest_uuid: "7e759d511dbf48d389a26747d452fbae"
mode: GRACEFUL
new_leader_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:45674
I20260709 14:56:32.742556 13307 raft_consensus.cc:606] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Received request to transfer leadership to TS 65dabe7272bd41ec849103a082ddd59a
I20260709 14:56:32.746685 13659 tablet.cc:1724] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.760839   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 14:56:32.761799   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 14:56:32.770491 13664 tablet.cc:1724] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.763080 13667 tablet.cc:1724] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.713174 13670 tablet.cc:1724] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.872138 13668 tablet.cc:1724] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:32.876004 13671 log.cc:927] Time spent T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Append to log took a long time: real 0.143s	user 0.005s	sys 0.000s
I20260709 14:56:32.875346 13681 tablet.cc:1724] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.878391 13655 tablet.cc:1724] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.882794 13658 tablet.cc:1724] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.868790 13680 tablet.cc:1724] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.910548 13665 tablet.cc:1724] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.913098 13649 raft_consensus.cc:993] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: : Instructing follower 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:32.916756 13664 tablet.cc:1724] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 14:56:32.919698 13677 log.cc:927] Time spent T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 14:56:32.890106 13650 raft_consensus.cc:993] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: : Instructing follower 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:32.913801 13649 raft_consensus.cc:1081] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Signalling peer 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:32.889003 13667 tablet.cc:1724] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.922456 13650 raft_consensus.cc:1081] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Signalling peer 65dabe7272bd41ec849103a082ddd59a to start an election
I20260709 14:56:32.935257 13380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a12707f8815f4068b98ef47558aab544"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:49914
I20260709 14:56:32.936544 13380 raft_consensus.cc:493] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:32.937057 13380 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:32.932749 13378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba"
dest_uuid: "65dabe7272bd41ec849103a082ddd59a"
 from {username='slave'} at 127.0.0.1:49900
I20260709 14:56:32.941586 13378 raft_consensus.cc:493] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 14:56:32.942095 13378 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:32.950942 13673 tablet.cc:1724] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.952244 13669 tablet.cc:1724] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.965674 13655 tablet.cc:1724] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.950064 13659 tablet.cc:1724] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.970641 13680 tablet.cc:1724] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.974371 13681 tablet.cc:1724] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:32.989849 13380 raft_consensus.cc:515] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:32.999401 13378 raft_consensus.cc:515] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:33.003767 13232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:33.004582 13232 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:33.009182 13310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12707f8815f4068b98ef47558aab544" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:33.010025 13310 raft_consensus.cc:3055] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 14:56:33.010329 13310 raft_consensus.cc:740] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e759d511dbf48d389a26747d452fbae, State: Running, Role: LEADER
I20260709 14:56:33.011183 13310 consensus_queue.cc:260] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:33.012418 13310 raft_consensus.cc:3060] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Advancing to term 3
I20260709 14:56:33.017686 13378 leader_election.cc:290] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:33.021804 13234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c7aef1187334e5bb60453d1d4c07ebe"
I20260709 14:56:33.022570 13234 raft_consensus.cc:3055] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 14:56:33.022944 13234 raft_consensus.cc:740] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c7aef1187334e5bb60453d1d4c07ebe, State: Running, Role: LEADER
I20260709 14:56:33.002187 13380 leader_election.cc:290] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 3 election: Requested vote from peers 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565), 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319)
I20260709 14:56:33.023766 13234 consensus_queue.cc:260] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:33.026693 13309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c7a9981ec004884ab14f39794183aba" candidate_uuid: "65dabe7272bd41ec849103a082ddd59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7e759d511dbf48d389a26747d452fbae"
I20260709 14:56:33.027804 13309 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:33.040513 13658 tablet.cc:1724] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 14:56:33.042313 13309 raft_consensus.cc:2468] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:33.043251 13234 raft_consensus.cc:3060] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Advancing to term 2
I20260709 14:56:33.056592 13351 leader_election.cc:304] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7e759d511dbf48d389a26747d452fbae; no voters: 
I20260709 14:56:33.058130 13675 raft_consensus.cc:2804] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 14:56:33.058729 13675 raft_consensus.cc:697] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:33.059785 13675 consensus_queue.cc:237] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:33.049057 13232 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 3.
I20260709 14:56:33.064380 13351 leader_election.cc:304] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dabe7272bd41ec849103a082ddd59a, 7c7aef1187334e5bb60453d1d4c07ebe; no voters: 
I20260709 14:56:33.065521 13634 raft_consensus.cc:2804] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 14:56:33.066082 13634 raft_consensus.cc:697] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 3 LEADER]: Becoming Leader. State: Replica: 65dabe7272bd41ec849103a082ddd59a, State: Running, Role: LEADER
I20260709 14:56:33.067065 13634 consensus_queue.cc:237] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } }
I20260709 14:56:33.053119 13310 raft_consensus.cc:2468] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 3.
I20260709 14:56:33.075026 13234 raft_consensus.cc:2468] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dabe7272bd41ec849103a082ddd59a in term 2.
I20260709 14:56:33.079358 13132 catalog_manager.cc:6242] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: soft_deleted_table [id=9cc61a02316e4c86981376492a02ef9a] alter complete (version 1)
I20260709 14:56:33.094569 13132 catalog_manager.cc:5697] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 1 to 2, leader changed from 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193) to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:33.096086 13132 catalog_manager.cc:5697] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a reported cstate change: term changed from 2 to 3, leader changed from 7e759d511dbf48d389a26747d452fbae (127.0.107.194) to 65dabe7272bd41ec849103a082ddd59a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "65dabe7272bd41ec849103a082ddd59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dabe7272bd41ec849103a082ddd59a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 14:56:33.524565 13234 raft_consensus.cc:1275] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:33.526227 13634 consensus_queue.cc:1048] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:33.538885 13310 raft_consensus.cc:1275] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 14:56:33.541777 13234 raft_consensus.cc:1275] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 3 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:33.542133 13675 consensus_queue.cc:1048] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 14:56:33.543965 13634 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7aef1187334e5bb60453d1d4c07ebe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:56:33.581452 13310 raft_consensus.cc:1275] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 3 FOLLOWER]: Refusing update from remote peer 65dabe7272bd41ec849103a082ddd59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 14:56:33.583725 13696 consensus_queue.cc:1048] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e759d511dbf48d389a26747d452fbae" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 14:56:33.833256   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 14:56:33.871001   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:33.871941   431 tablet_replica.cc:333] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.873122   431 raft_consensus.cc:2243] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.873904   431 raft_consensus.cc:2272] T 38f2066c01c84ab2830b97582940f560 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.877839   431 tablet_replica.cc:333] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.878731   431 raft_consensus.cc:2243] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.879390   431 raft_consensus.cc:2272] T 396a5271c7b54b88b7e7fd0cabd1399d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.882558   431 tablet_replica.cc:333] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.883489   431 raft_consensus.cc:2243] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.884080   431 raft_consensus.cc:2272] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.887180   431 tablet_replica.cc:333] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.887946   431 raft_consensus.cc:2243] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.888978   431 raft_consensus.cc:2272] T 189870e93a894e529fc792e024b798c3 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.892057   431 tablet_replica.cc:333] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.892805   431 raft_consensus.cc:2243] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.893782   431 raft_consensus.cc:2272] T e20d324c4a82498a9b9319063c6b8c6d P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.896524   431 tablet_replica.cc:333] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.897269   431 raft_consensus.cc:2243] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 LEADER]: Raft consensus shutting down.
W20260709 14:56:33.897650 13351 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
I20260709 14:56:33.898490   431 raft_consensus.cc:2272] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:33.901346 13351 consensus_peers.cc:597] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a -> Peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Couldn't send request to peer 7c7aef1187334e5bb60453d1d4c07ebe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:33.901683   431 tablet_replica.cc:333] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.902536   431 raft_consensus.cc:2243] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.903203   431 raft_consensus.cc:2272] T 05bf66306ab9413d9bba98818c0b6b5a P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.905992   431 tablet_replica.cc:333] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.906711   431 raft_consensus.cc:2243] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.907323   431 raft_consensus.cc:2272] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.909993   431 tablet_replica.cc:333] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.910724   431 raft_consensus.cc:2243] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.911412   431 raft_consensus.cc:2272] T 609caffbdcf5479597a579e719e7d023 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.913920   431 tablet_replica.cc:333] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.914647   431 raft_consensus.cc:2243] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.915354   431 raft_consensus.cc:2272] T be6baf7543824ff5a503aa00c55adc45 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:33.917750 13351 consensus_peers.cc:597] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a -> Peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Couldn't send request to peer 7c7aef1187334e5bb60453d1d4c07ebe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:33.917939   431 tablet_replica.cc:333] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.918777   431 raft_consensus.cc:2243] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.920060   431 raft_consensus.cc:2272] T bee8ad3334284671a1e3a35ed587356a P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.923659   431 tablet_replica.cc:333] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.924430   431 raft_consensus.cc:2243] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.925071   431 raft_consensus.cc:2272] T a12707f8815f4068b98ef47558aab544 P 7c7aef1187334e5bb60453d1d4c07ebe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.928217   431 tablet_replica.cc:333] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.928946   431 raft_consensus.cc:2243] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.929505   431 raft_consensus.cc:2272] T 60b48dbc27304cf29011afb100eb77bb P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.932112   431 tablet_replica.cc:333] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.932749   431 raft_consensus.cc:2243] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.933724   431 raft_consensus.cc:2272] T 2a55dd7f3a5846e4a887775d892130ce P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.937021   431 tablet_replica.cc:333] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.937685   431 raft_consensus.cc:2243] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.938786   431 raft_consensus.cc:2272] T 19cf6020ce324c1ab6a5ff208b019565 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.941344   431 tablet_replica.cc:333] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.941917   431 raft_consensus.cc:2243] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.942474   431 raft_consensus.cc:2272] T 88ad4624d080415b867fb71340293995 P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.945748   431 tablet_replica.cc:333] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.946636   431 raft_consensus.cc:2243] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:33.947886   431 raft_consensus.cc:2272] T 5ca8c258194e44d895257978f0a2ee07 P 7c7aef1187334e5bb60453d1d4c07ebe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:33.950960   431 tablet_replica.cc:333] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe: stopping tablet replica
I20260709 14:56:33.951800   431 raft_consensus.cc:2243] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:33.952427   431 raft_consensus.cc:2272] T 4c7a9981ec004884ab14f39794183aba P 7c7aef1187334e5bb60453d1d4c07ebe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:33.972983 13351 consensus_peers.cc:597] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a -> Peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Couldn't send request to peer 7c7aef1187334e5bb60453d1d4c07ebe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:34.002259   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 14:56:34.022732 13277 consensus_peers.cc:597] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae -> Peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Couldn't send request to peer 7c7aef1187334e5bb60453d1d4c07ebe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 14:56:34.044523 13351 consensus_peers.cc:597] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a -> Peer 7c7aef1187334e5bb60453d1d4c07ebe (127.0.107.193:46565): Couldn't send request to peer 7c7aef1187334e5bb60453d1d4c07ebe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:34.048744   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 14:56:34.085964   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:34.087046   431 tablet_replica.cc:333] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.088068   431 raft_consensus.cc:2243] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.089432   431 raft_consensus.cc:2272] T 38f2066c01c84ab2830b97582940f560 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.092278   431 tablet_replica.cc:333] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.092962   431 raft_consensus.cc:2243] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.093907   431 raft_consensus.cc:2272] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.096268   431 tablet_replica.cc:333] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.096961   431 raft_consensus.cc:2243] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.097442   431 raft_consensus.cc:2272] T 189870e93a894e529fc792e024b798c3 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.099702   431 tablet_replica.cc:333] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.100432   431 raft_consensus.cc:2243] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.101331   431 raft_consensus.cc:2272] T 396a5271c7b54b88b7e7fd0cabd1399d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.103719   431 tablet_replica.cc:333] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.104637   431 raft_consensus.cc:2243] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.105135   431 raft_consensus.cc:2272] T e20d324c4a82498a9b9319063c6b8c6d P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.109222   431 tablet_replica.cc:333] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.110241   431 raft_consensus.cc:2243] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.110797   431 raft_consensus.cc:2272] T 1e69b1ce56f94e03b714a57aa6e0d983 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.113400   431 tablet_replica.cc:333] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.114005   431 raft_consensus.cc:2243] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.114470   431 raft_consensus.cc:2272] T 05bf66306ab9413d9bba98818c0b6b5a P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.117235   431 tablet_replica.cc:333] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.117828   431 raft_consensus.cc:2243] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.118278   431 raft_consensus.cc:2272] T a12707f8815f4068b98ef47558aab544 P 7e759d511dbf48d389a26747d452fbae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.121243   431 tablet_replica.cc:333] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.121906   431 raft_consensus.cc:2243] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.122397   431 raft_consensus.cc:2272] T 2a55dd7f3a5846e4a887775d892130ce P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.124679   431 tablet_replica.cc:333] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.125315   431 raft_consensus.cc:2243] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.125751   431 raft_consensus.cc:2272] T 19cf6020ce324c1ab6a5ff208b019565 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.128243   431 tablet_replica.cc:333] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.129077   431 raft_consensus.cc:2243] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.129752   431 raft_consensus.cc:2272] T 609caffbdcf5479597a579e719e7d023 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.132401   431 tablet_replica.cc:333] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.133107   431 raft_consensus.cc:2243] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.134574   431 raft_consensus.cc:2272] T be6baf7543824ff5a503aa00c55adc45 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.138096   431 tablet_replica.cc:333] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.139237   431 raft_consensus.cc:2243] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.139919   431 raft_consensus.cc:2272] T bee8ad3334284671a1e3a35ed587356a P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.143060   431 tablet_replica.cc:333] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.143729   431 raft_consensus.cc:2243] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.144719   431 raft_consensus.cc:2272] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.147719   431 tablet_replica.cc:333] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.148411   431 raft_consensus.cc:2243] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.149407   431 raft_consensus.cc:2272] T 60b48dbc27304cf29011afb100eb77bb P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.152447   431 tablet_replica.cc:333] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.153115   431 raft_consensus.cc:2243] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.154076   431 raft_consensus.cc:2272] T 88ad4624d080415b867fb71340293995 P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.156504   431 tablet_replica.cc:333] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.157125   431 raft_consensus.cc:2243] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.157670   431 raft_consensus.cc:2272] T 5ca8c258194e44d895257978f0a2ee07 P 7e759d511dbf48d389a26747d452fbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.160094   431 tablet_replica.cc:333] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae: stopping tablet replica
I20260709 14:56:34.160728   431 raft_consensus.cc:2243] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.161249   431 raft_consensus.cc:2272] T 4c7a9981ec004884ab14f39794183aba P 7e759d511dbf48d389a26747d452fbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.206538   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 14:56:34.246774   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 14:56:34.280627   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 14:56:34.281615   431 tablet_replica.cc:333] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.282693   431 raft_consensus.cc:2243] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.283838   431 raft_consensus.cc:2272] T 38f2066c01c84ab2830b97582940f560 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.286476   431 tablet_replica.cc:333] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.287307   431 raft_consensus.cc:2243] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.287851   431 raft_consensus.cc:2272] T 396a5271c7b54b88b7e7fd0cabd1399d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.290194   431 tablet_replica.cc:333] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.291055   431 raft_consensus.cc:2243] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.291757   431 raft_consensus.cc:2272] T 77b11d13b5fd47c0bb7dc0e5a56344c1 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.294130   431 tablet_replica.cc:333] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.294790   431 raft_consensus.cc:2243] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.295398   431 raft_consensus.cc:2272] T 189870e93a894e529fc792e024b798c3 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.298141   431 tablet_replica.cc:333] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.299103   431 raft_consensus.cc:2243] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.299799   431 raft_consensus.cc:2272] T 88ad4624d080415b867fb71340293995 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.302763   431 tablet_replica.cc:333] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.303531   431 raft_consensus.cc:2243] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.304064   431 raft_consensus.cc:2272] T 5ca8c258194e44d895257978f0a2ee07 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.306916   431 tablet_replica.cc:333] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.307683   431 raft_consensus.cc:2243] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.308368   431 raft_consensus.cc:2272] T e20d324c4a82498a9b9319063c6b8c6d P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 14:56:34.309268 13351 consensus_peers.cc:597] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a -> Peer 7e759d511dbf48d389a26747d452fbae (127.0.107.194:37319): Couldn't send request to peer 7e759d511dbf48d389a26747d452fbae. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 14:56:34.311108   431 tablet_replica.cc:333] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.311952   431 raft_consensus.cc:2243] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.312464   431 raft_consensus.cc:2272] T 60b48dbc27304cf29011afb100eb77bb P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.315361   431 tablet_replica.cc:333] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.316068   431 raft_consensus.cc:2243] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.316576   431 raft_consensus.cc:2272] T 1e69b1ce56f94e03b714a57aa6e0d983 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.319267   431 tablet_replica.cc:333] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.319973   431 raft_consensus.cc:2243] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.320909   431 raft_consensus.cc:2272] T 05bf66306ab9413d9bba98818c0b6b5a P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.323642   431 tablet_replica.cc:333] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.324235   431 raft_consensus.cc:2243] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.324718   431 raft_consensus.cc:2272] T 2a55dd7f3a5846e4a887775d892130ce P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.327023   431 tablet_replica.cc:333] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.327600   431 raft_consensus.cc:2243] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.328047   431 raft_consensus.cc:2272] T 19cf6020ce324c1ab6a5ff208b019565 P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.330444   431 tablet_replica.cc:333] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.331180   431 raft_consensus.cc:2243] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 3 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.332132   431 raft_consensus.cc:2272] T a12707f8815f4068b98ef47558aab544 P 65dabe7272bd41ec849103a082ddd59a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.336174   431 tablet_replica.cc:333] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.336988   431 raft_consensus.cc:2243] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.337991   431 raft_consensus.cc:2272] T 609caffbdcf5479597a579e719e7d023 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.340719   431 tablet_replica.cc:333] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.341436   431 raft_consensus.cc:2243] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.342073   431 raft_consensus.cc:2272] T be6baf7543824ff5a503aa00c55adc45 P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.344889   431 tablet_replica.cc:333] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.345487   431 raft_consensus.cc:2243] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.346020   431 raft_consensus.cc:2272] T bee8ad3334284671a1e3a35ed587356a P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.348941   431 tablet_replica.cc:333] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.349614   431 raft_consensus.cc:2243] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 14:56:34.350214   431 raft_consensus.cc:2272] T 5ff4b0d5d0544e378da6c1d1b5f1f4ec P 65dabe7272bd41ec849103a082ddd59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.353159   431 tablet_replica.cc:333] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a: stopping tablet replica
I20260709 14:56:34.353753   431 raft_consensus.cc:2243] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 2 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.354751   431 raft_consensus.cc:2272] T 4c7a9981ec004884ab14f39794183aba P 65dabe7272bd41ec849103a082ddd59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.405222   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 14:56:34.443181   431 master.cc:562] Master@127.0.107.254:34179 shutting down...
I20260709 14:56:34.473019   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 1 LEADER]: Raft consensus shutting down.
I20260709 14:56:34.473814   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 14:56:34.474260   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P f97fae1c76d3474ca9e61e0940d3bedc: stopping tablet replica
I20260709 14:56:34.498584   431 master.cc:584] Master@127.0.107.254:34179 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12360 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25695 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (395607 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 1 FAILED TEST
I20260709 14:56:34.562021   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 392 messages since previous log ~15 seconds ago
I20260709 14:56:34.562323   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 390 messages since previous log ~15 seconds ago
I20260709 14:56:34.562561   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 383 messages since previous log ~15 seconds ago
I20260709 14:56:34.562849   431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 3 messages since previous log ~206 seconds ago
I20260709 14:56:34.563110   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~34 seconds ago
I20260709 14:56:34.563287   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~34 seconds ago
I20260709 14:56:34.563524   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20260709 14:56:34.563867   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
